Use of free text in Electronic Health Records (EHRs) for clinical, administrative, and research purposes has proliferated in recent years. Using the Mobility domain of the ICF as a framework, we comprehensively analyze the structure and characteristics of mobility related concepts found in physical therapy notes from the National Institutes of Health’s Clinical Center. The result is a mobility entity framework comprised of 5 entities types, 3 relations, 8 attributes, and 33 attribute values. Two domain experts manually curated a gold standard corpus of 14,281 nested entity mentions from 400 clinical notes. Inter-annotator agreement (IAA) of exact matching averaged 92.3% F1-score on mention text spans, and 96.6% Cohen’s kappa on attributes assignments. A novel ensemble machine learning model for named entity recognition was trained and evaluated using the gold standard corpus. Average F1-score on exact entity matching of our ensemble method (83.31%) outperformed both baseline methods: a probabilistic graphical model (80.4%), and an artificial neural network (81.82%). Overcoming the irregularities and challenges in capturing functioning concepts, this work pioneers a representational framework, an annotated gold standard corpus, and a cutting-edge machine learning model that identify concepts in the Mobility domain of the ICF.

Presenter(s)

Thanh Thieu