López-García, GuillermoJerez, José M.Ribelles, NuriaAlba, EmilioVeredas, Francisco J.2022-07-282022-07-282021-05-13López-García G, Jerez JM, Ribelles N, Alba E, Veredas FJ. Transformers for Clinical Coding in Spanish. IEEE Access. 2021;9:72387-72397http://hdl.handle.net/10668/3836Automatic clinical coding is an essential task in the process of extracting relevant information from unstructured documents contained in electronic health records (EHRs). However, most research in the development of computer-based methods for clinical coding focuses on texts written in English due to the limited availability of medical linguistic resources in languages other than English. With nearly 500 million native speakers, there is a worldwide interest in processing healthcare texts in Spanish. In this study, we sys tematically analyzed transformer-based models for automatic clinical coding in Spanish. Using a transfer learning-based approach, the three existing transformer architectures that support the Spanish language, namely, multilingual BERT (mBERT), BETO and XLM-RoBERTa (XLM-R), were first pretrained on a corpus of real-world oncology clinical cases with the goal of adapting transformers to the particularities of Spanish medical texts. The resulting models were fine-tuned on three distinct clinical coding tasks, following a multilabel sentence classification strategy. For each analyzed transformer, the domain-specific version out performed the original general domain model across those tasks. Moreover, the combination of the developed strategy with an ensemble approach leveraging the predictive capacities of the three distinct transformers yielded the best obtained results, with MAP scores of 0.662, 0.544 and 0.884 on CodiEsp-D, CodiEsp-P and Cantemist-Coding shared tasks, which remarkably improved the previous state-of-the-art performance by 11.6%, 10.3% and 4.4%, respectively. We publicly release the mBERT, BETO and XLMR transform ers adapted to the Spanish clinical domain at https://github.com/guilopgar/ClinicalCodingTransformerES, providing the clinical natural language processing community with advanced deep learning methods for performing medical coding and other tasks in the Spanish clinical domain.enAtribución 4.0 InternacionalAtribución 4.0 InternacionalAtribución 4.0 InternacionalAtribución 4.0 Internacionalhttp://creativecommons.org/licenses/by/4.0/Clinical codingDeep learningNatural language processingText classificationTransformersCodificación clínicaAprendizaje profundoProcesamiento de lenguaje naturalModelos epidemiológicosAnálisis y desempeño de tareasMedical Subject Headings::Information Science::Information Science::Computing Methodologies::Artificial Intelligence::Natural Language ProcessingMedical Subject Headings::Analytical, Diagnostic and Therapeutic Techniques and Equipment::Investigative Techniques::Epidemiologic Methods::Data Collection::Records as Topic::Medical Records::Medical Records Systems, Computerized::Electronic Health RecordsMedical Subject Headings::Health Care::Health Services Administration::Organization and Administration::Professional Practice::Practice Management::Office Management::Forms and Records Control::Clinical CodingMedical Subject Headings::Information Science::Information Science::Communication::Language::LinguisticsMedical Subject Headings::Health Care::Health Care Quality, Access, and Evaluation::Delivery of Health CareMedical Subject Headings::Information Science::Information Science::Computing Methodologies::Computer Systems::ComputersMedical Subject Headings::Psychiatry and Psychology::Behavior and Behavior Mechanisms::Motivation::GoalsMedical Subject Headings::Psychiatry and Psychology::Psychological Phenomena and Processes::Psychology, Applied::Human Engineering::Task Performance and Analysisresearch articleopen access10.1109/ACCESS.2021.30800852169-3536