We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Automating detection of diagnostic error of infectious diseases using machine learning.
- Authors
Peterson, Kelly S.; Chapman, Alec B.; Widanagamaachchi, Wathsala; Sutton, Jesse; Ochoa, Brennan; Jones, Barbara E.; Stevens, Vanessa; Classen, David C.; Jones, Makoto M.
- Abstract
Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings. Author summary: Identifying diagnostic error is challenging since it is often found only through review so time consuming not all patient data can be reviewed, let alone in a timely fashion to potentially prevent harm. In this work we address this gap by proposing machine learning methods which leverage millions of patient encounters. Since such methods could potentially be automated, they could scale to identify situations when the diagnosis may not be correct, timely, or when there may be a risk of death to the patient. This approach was validated by clinicians and shows promise for continued development. Future work will be needed to translate this work to protect patients and support clinicians.
- Subjects
COMMUNICABLE disease diagnosis; MORTALITY risk factors; RISK assessment; SCALE analysis (Psychology); DATA analysis; RECEIVER operating characteristic curves; PREDICTION models; RESEARCH funding; DIAGNOSTIC errors; HOSPITAL emergency services; DESCRIPTIVE statistics; ELECTRONIC health records; STATISTICS; MACHINE learning; AUTOMATION; COMPARATIVE studies; SENSITIVITY &; specificity (Statistics); INTER-observer reliability
- Publication
PLoS Digital Health, 2024, Vol 3, Issue 6, p1
- ISSN
2767-3170
- Publication type
Article
- DOI
10.1371/journal.pdig.0000528