We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Utility of the Simulated Outcomes Following Carotid Artery Laceration Video Data Set for Machine Learning Applications.
- Authors
Kugener, Guillaume; Pangal, Dhiraj J.; Cardinal, Tyler; Collet, Casey; Lechtholz-Zey, Elizabeth; Lasky, Sasha; Sundaram, Shivani; Markarian, Nicholas; Zhu, Yichao; Roshannai, Arman; Sinha, Aditya; Han, X. Y.; Papyan, Vardan; Hung, Andrew; Anandkumar, Animashree; Wrobel, Bozena; Zada, Gabriel; Donoho, Daniel A.
- Abstract
Key Points: Question: What is the utility of a data set that contains videos of surgeons managing hemorrhage? Findings: This quality improvement study of the Simulated Outcomes Following Carotid Artery Laceration (SOCAL), a public data set of surgeons managing catastrophic surgical hemorrhage in a cadaveric training exercise included 65 071 instrument annotations with recorded outcomes. Computer vision–based instrument detection achieved a mean average precision of 0.67 on SOCAL and a sensitivity of 0.77 and a positive predictive value of 0.96 at detecting surgical instruments from real intraoperative video. Meaning: A corpus of videos of surgeons managing catastrophic hemorrhage is a novel, valuable resource for surgical data science. This quality improvement study evaluates the utility of a publicly available surgical video data set of hemorrhage complication management with instrument annotations and task outcomes. Importance: Surgical data scientists lack video data sets that depict adverse events, which may affect model generalizability and introduce bias. Hemorrhage may be particularly challenging for computer vision–based models because blood obscures the scene. Objective: To assess the utility of the Simulated Outcomes Following Carotid Artery Laceration (SOCAL)—a publicly available surgical video data set of hemorrhage complication management with instrument annotations and task outcomes—to provide benchmarks for surgical data science techniques, including computer vision instrument detection, instrument use metrics and outcome associations, and validation of a SOCAL-trained neural network using real operative video. Design, Setting, and Participants: For this quailty improvement study, a total of 75 surgeons with 1 to 30 years' experience (mean, 7 years) were filmed from January 1, 2017, to December 31, 2020, managing catastrophic surgical hemorrhage in a high-fidelity cadaveric training exercise at nationwide training courses. Videos were annotated from January 1 to June 30, 2021. Interventions: Surgeons received expert coaching between 2 trials. Main Outcomes and Measures: Hemostasis within 5 minutes (task success, dichotomous), time to hemostasis (in seconds), and blood loss (in milliliters) were recorded. Deep neural networks (DNNs) were trained to detect surgical instruments in view. Model performance was measured using mean average precision (mAP), sensitivity, and positive predictive value. Results: SOCAL contains 31 443 frames with 65 071 surgical instrument annotations from 147 trials with associated surgeon demographic characteristics, time to hemostasis, and recorded blood loss for each trial. Computer vision–based instrument detection methods using DNNs trained on SOCAL achieved a mAP of 0.67 overall and 0.91 for the most common surgical instrument (suction). Hemorrhage control challenges standard object detectors: detection of some surgical instruments remained poor (mAP, 0.25). On real intraoperative video, the model achieved a sensitivity of 0.77 and a positive predictive value of 0.96. Instrument use metrics derived from the SOCAL video were significantly associated with performance (blood loss). Conclusions and Relevance: Hemorrhage control is a high-stakes adverse event that poses unique challenges for video analysis, but no data sets of hemorrhage control exist. The use of SOCAL, the first data set to depict hemorrhage control, allows the benchmarking of data science applications, including object detection, performance metric development, and identification of metrics associated with outcomes. In the future, SOCAL may be used to build and validate surgical data science models.
- Subjects
CAROTID artery surgery; CAROTID artery; MACHINE learning; HEALTH outcome assessment; HEMOSTASIS; DATABASE management; QUALITY assurance; DESCRIPTIVE statistics; ARTIFICIAL neural networks; SENSITIVITY &; specificity (Statistics); VIDEO recording
- Publication
JAMA Network Open, 2022, Vol 5, Issue 3, pe223177
- ISSN
2574-3805
- Publication type
Article
- DOI
10.1001/jamanetworkopen.2022.3177