We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Inter-observer variability of expert-derived morphologic risk predictors in aortic dissection.
- Authors
Willemink, Martin J.; Mastrodicasa, Domenico; Madani, Mohammad H.; Codari, Marina; Chepelev, Leonid L.; Mistelbauer, Gabriel; Hanneman, Kate; Ouzounian, Maral; Ocazionez, Daniel; Afifi, Rana O.; Lacomis, Joan M.; Lovato, Luigi; Pacini, Davide; Folesani, Gianluca; Hinzpeter, Ricarda; Alkadhi, Hatem; Stillman, Arthur E.; Sailer, Anna M.; Turner, Valery L.; Hinostroza, Virginia
- Abstract
Objectives: Establishing the reproducibility of expert-derived measurements on CTA exams of aortic dissection is clinically important and paramount for ground-truth determination for machine learning. Methods: Four independent observers retrospectively evaluated CTA exams of 72 patients with uncomplicated Stanford type B aortic dissection and assessed the reproducibility of a recently proposed combination of four morphologic risk predictors (maximum aortic diameter, false lumen circumferential angle, false lumen outflow, and intercostal arteries). For the first inter-observer variability assessment, 47 CTA scans from one aortic center were evaluated by expert-observer 1 in an unconstrained clinical assessment without a standardized workflow and compared to a composite of three expert-observers (observers 2–4) using a standardized workflow. A second inter-observer variability assessment on 30 out of the 47 CTA scans compared observers 3 and 4 with a constrained, standardized workflow. A third inter-observer variability assessment was done after specialized training and tested between observers 3 and 4 in an external population of 25 CTA scans. Inter-observer agreement was assessed with intraclass correlation coefficients (ICCs) and Bland-Altman plots. Results: Pre-training ICCs of the four morphologic features ranged from 0.04 (−0.05 to 0.13) to 0.68 (0.49–0.81) between observer 1 and observers 2–4 and from 0.50 (0.32–0.69) to 0.89 (0.78–0.95) between observers 3 and 4. ICCs improved after training ranging from 0.69 (0.52–0.87) to 0.97 (0.94–0.99), and Bland-Altman analysis showed decreased bias and limits of agreement. Conclusions: Manual morphologic feature measurements on CTA images can be optimized resulting in improved inter-observer reliability. This is essential for robust ground-truth determination for machine learning models. Key Points: • Clinical fashion manual measurements of aortic CTA imaging features showed poor inter-observer reproducibility. • A standardized workflow with standardized training resulted in substantial improvements with excellent inter-observer reproducibility. • Robust ground truth labels obtained manually with excellent inter-observer reproducibility are key to develop reliable machine learning models.
- Subjects
MACHINE learning; AORTIC dissection; COMPUTED tomography; BLAND-Altman plot; INTRACLASS correlation
- Publication
European Radiology, 2023, Vol 33, Issue 2, p1102
- ISSN
0938-7994
- Publication type
Article
- DOI
10.1007/s00330-022-09056-z