We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults.
- Authors
Wilimitis, Drew; Turer, Robert W.; Ripperger, Michael; McCoy, Allison B.; Sperry, Sarah H.; Fielstein, Elliot M.; Kurz, Troy; Walsh, Colin G.
- Abstract
Key Points: Question: Does prediction of suicide risk improve when combining face-to-face screening with electronic health record–based machine learning models? Findings: In this cohort study of 120 398 adult patient encounters, an ensemble learning approach combined suicide risk predictions from the Columbia Suicide Severity Rating Scale and a real-time machine learning model. Combined models outperformed either model alone for risks of suicide attempt and suicidal ideation across a variety of time periods. Meaning: These findings suggest that health care systems should attempt to leverage the independent, complementary strengths of traditional clinician assessment and automated machine learning to improve suicide risk detection. This cohort study evaluates the respective and combined abilities of a real-time machine learning model and the Columbia Suicide Severity Rating Scale to predict suicide attempt and suicidal ideation among adults. Importance: Understanding the differences and potential synergies between traditional clinician assessment and automated machine learning might enable more accurate and useful suicide risk detection. Objective: To evaluate the respective and combined abilities of a real-time machine learning model and the Columbia Suicide Severity Rating Scale (C-SSRS) to predict suicide attempt (SA) and suicidal ideation (SI). Design, Setting, and Participants: This cohort study included encounters with adult patients (aged ≥18 years) at a major academic medical center. The C-SSRS was administered during routine care, and a Vanderbilt Suicide Attempt and Ideation Likelihood (VSAIL) prediction was generated in the electronic health record. Encounters took place in the inpatient, ambulatory surgical, and emergency department settings. Data were collected from June 2019 to September 2020. Main Outcomes and Measures: Primary outcomes were the incidence of SA and SI, encoded as International Classification of Diseases codes, occurring within various time periods after an index visit. We evaluated the retrospective validity of the C-SSRS, VSAIL, and ensemble models combining both. Discrimination metrics included area under the receiver operating curve (AUROC), area under the precision-recall curve (AUPR), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Results: The cohort included 120 398 unique index visits for 83 394 patients (mean [SD] age, 51.2 [20.6] years; 38 107 [46%] men; 45 273 [54%] women; 13 644 [16%] Black; 63 869 [77%] White). Within 30 days of an index visit, the combined models had higher AUROC (SA: 0.874-0.887; SI: 0.869-0.879) than both the VSAIL (SA: 0.729; SI: 0.773) and C-SSRS (SA: 0.823; SI: 0.777) models. In the highest risk-decile, ensemble methods had PPV of 1.3% to 1.4% for SA and 8.3% to 8.7% for SI and sensitivity of 77.6% to 79.5% for SA and 67.4% to 70.1% for SI, outperforming VSAIL (PPV for SA: 0.4%; PPV for SI: 3.9%; sensitivity for SA: 28.8%; sensitivity for SI: 35.1%) and C-SSRS (PPV for SA: 0.5%; PPV for SI: 3.5%; sensitivity for SA: 76.6%; sensitivity for SI: 68.8%). Conclusions and Relevance: In this study, suicide risk prediction was optimal when leveraging both in-person screening (for acute measures of risk in patient-reported suicidality) and historical EHR data (for underlying clinical factors that can quantify a patient's passive risk level). To improve suicide risk classification, prediction systems could combine pretrained machine learning with structured clinician assessment without needing to retrain the original model.
- Subjects
SUICIDE prevention; SUICIDE risk factors; NOSOLOGY; MACHINE learning; MEDICAL screening; RETROSPECTIVE studies; RISK assessment; SUICIDAL behavior; SUICIDAL ideation; DESCRIPTIVE statistics; PREDICTION models; RECEIVER operating characteristic curves; DATA analysis software; LONGITUDINAL method; ADULTS
- Publication
JAMA Network Open, 2022, Vol 5, Issue 5, pe2212095
- ISSN
2574-3805
- Publication type
Article
- DOI
10.1001/jamanetworkopen.2022.12095