Abstract
Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.
| Original language | English |
|---|---|
| Article number | 103299 |
| Journal | Artificial Intelligence in Medicine |
| Volume | 171 |
| DOIs | |
| State | Published - Jan 2026 |
Funding
This work was supported in part by the Joint Design of Advanced Computing Solutions for Cancer program established by the US Department of Energy (DOE), in part by the NCI of the National Institutes of Health, in part by Argonne National Laboratory, United States under DE-AC02-06-CH11357, in part by Lawrence Livermore National Laboratory, United States under DEAC52-07NA27344, in part by Los Alamos National Laboratory, United States under DE-AC5206NA25396, in part by Oak Ridge National Laboratory (ORNL) under DE-AC05-00OR22725 performed under the auspices of DOE, and in part by the UT-Battelle LCC under DE-ACO5-000R22725 with the DOE. Notice: This manuscript has been authored by UT-Battelle LLC under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan ( https://www.energy.gov/doe-public-access-plan ).
Keywords
- Automated medical encoding
- Common data models
- Electronic health records
- Machine learning
- Natural language processing