We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Internal and External Validation of a Machine Learning Risk Score for Acute Kidney Injury.
- Authors
Churpek, Matthew M.; Carey, Kyle A.; Edelson, Dana P.; Singh, Tripti; Astor, Brad C.; Gilbert, Emily R.; Winslow, Christopher; Shah, Nirav; Afshar, Majid; Koyner, Jay L.
- Abstract
Key Points: Question: What is the accuracy of a single-center machine learning algorithm for predicting acute kidney injury (AKI) when internally and externally tested? Findings: In this multicenter diagnostic study of approximately 500 000 admissions from 6 hospitals in 3 health systems, the machine learning algorithm had similarly high discrimination in both internal and external validation cohorts. Alert thresholds fired nearly a day and a half before the event. Meaning: These findings demonstrate that the AKI algorithm is generalizable to patients in the center in which it was derived and to patients from other hospitals, suggesting that implementation could prompt early identification and therapy aimed at decreasing preventable AKI. This diagnostic study internally and externally validates a machine learning risk score for detecting acute kidney injury in hospitalized patients. Importance: Acute kidney injury (AKI) is associated with increased morbidity and mortality in hospitalized patients. Current methods to identify patients at high risk of AKI are limited, and few prediction models have been externally validated. Objective: To internally and externally validate a machine learning risk score to detect AKI in hospitalized patients. Design, Setting, and Participants: This diagnostic study included 495 971 adult hospital admissions at the University of Chicago (UC) from 2008 to 2016 (n = 48 463), at Loyola University Medical Center (LUMC) from 2007 to 2017 (n = 200 613), and at NorthShore University Health System (NUS) from 2006 to 2016 (n = 246 895) with serum creatinine (SCr) measurements. Patients with an SCr concentration at admission greater than 3.0 mg/dL, with a prior diagnostic code for chronic kidney disease stage 4 or higher, or who received kidney replacement therapy within 48 hours of admission were excluded. A simplified version of a previously published gradient boosted machine AKI prediction algorithm was used; it was validated internally among patients at UC and externally among patients at NUS and LUMC. Main Outcomes and Measures: Prediction of Kidney Disease Improving Global Outcomes SCr-defined stage 2 AKI within a 48-hour interval was the primary outcome. Discrimination was assessed by the area under the receiver operating characteristic curve (AUC). Results: The study included 495 971 adult admissions (mean [SD] age, 63 [18] years; 87 689 [17.7%] African American; and 266 866 [53.8%] women) across 3 health systems. The development of stage 2 or higher AKI occurred in 15 664 of 48 463 patients (3.4%) in the UC cohort, 5711 of 200 613 (2.8%) in the LUMC cohort, and 3499 of 246 895 (1.4%) in the NUS cohort. In the UC cohort, 332 patients (0.7%) required kidney replacement therapy compared with 672 patients (0.3%) in the LUMC cohort and 440 patients (0.2%) in the NUS cohort. The AUCs for predicting at least stage 2 AKI in the next 48 hours were 0.86 (95% CI, 0.86-0.86) in the UC cohort, 0.85 (95% CI, 0.84-0.85) in the LUMC cohort, and 0.86 (95% CI, 0.86-0.86) in the NUS cohort. The AUCs for receipt of kidney replacement therapy within 48 hours were 0.96 (95% CI, 0.96-0.96) in the UC cohort, 0.95 (95% CI, 0.94-0.95) in the LUMC cohort, and 0.95 (95% CI, 0.94-0.95) in the NUS cohort. In time-to-event analysis, a probability cutoff of at least 0.057 predicted the onset of stage 2 AKI a median (IQR) of 27 (6.5-93) hours before the eventual doubling in SCr concentrations in the UC cohort, 34.5 (19-85) hours in the NUS cohort, and 39 (19-108) hours in the LUMC cohort. Conclusions and Relevance: In this study, the machine learning algorithm demonstrated excellent discrimination in both internal and external validation, supporting its generalizability and potential as a clinical decision support tool to improve AKI detection and outcomes.
- Subjects
ILLINOIS; ACADEMIC medical centers; ACUTE kidney failure; ALGORITHMS; ANALYSIS of variance; CHI-squared test; CONFIDENCE intervals; CREATININE; HOSPITAL care; LONGITUDINAL method; MACHINE learning; RESEARCH methodology; MEDICAL cooperation; MEDICAL records; RESEARCH; RISK assessment; STATISTICAL hypothesis testing; SURVIVAL analysis (Biometry); T-test (Statistics); RETROSPECTIVE studies; RECEIVER operating characteristic curves; RESEARCH methodology evaluation; DATA analysis software; DESCRIPTIVE statistics; ACQUISITION of data methodology; MANN Whitney U Test; KRUSKAL-Wallis Test; DISEASE risk factors
- Publication
JAMA Network Open, 2020, Vol 3, Issue 8, pe2012892
- ISSN
2574-3805
- Publication type
Article
- DOI
10.1001/jamanetworkopen.2020.12892