Development and validation of a deep learning model for detecting signs of tuberculosis on chest radiographs among US-bound immigrants and refugees.

Lee, Scott H.; Fox, Shannon; Smith, Raheem; Skrobarcek, Kimberly A.; Keyserling, Harold; Phares, Christina R.; Lee, Deborah; Posey, Drew L.

Immigrants and refugees seeking admission to the United States must first undergo an overseas medical exam, overseen by the US Centers for Disease Control and Prevention (CDC), during which all persons ≥15 years old receive a chest x-ray to look for signs of tuberculosis. Although individual screening sites often implement quality control (QC) programs to ensure radiographs are interpreted correctly, the CDC does not currently have a method for conducting similar QC reviews at scale. We obtained digitized chest radiographs collected as part of the overseas immigration medical exam. Using radiographs from applicants 15 years old and older, we trained deep learning models to perform three tasks: identifying abnormal radiographs; identifying abnormal radiographs suggestive of tuberculosis; and identifying the specific findings (e.g., cavities or infiltrates) in abnormal radiographs. We then evaluated the models on both internal and external testing datasets, focusing on two classes of performance metrics: individual-level metrics, like sensitivity and specificity, and sample-level metrics, like accuracy in predicting the prevalence of abnormal radiographs. A total of 152,012 images (one image per applicant; mean applicant age 39 years) were used for model training. On our internal test dataset, our models performed well both in identifying abnormalities suggestive of TB (area under the curve [AUC] of 0.97; 95% confidence interval [CI]: 0.95, 0.98) and in estimating sample-level counts of the same (-2% absolute percentage error; 95% CIC: -8%, 6%). On the external test datasets, our models performed similarly well in identifying both generic abnormalities (AUCs ranging from 0.89 to 0.92) and those suggestive of TB (AUCs from 0.94 to 0.99). This performance was consistent across metrics, including those based on thresholded class predictions, like sensitivity, specificity, and F1 score. Strong performance relative to high-quality radiological reference standards across a variety of datasets suggests our models may make reliable tools for supporting chest radiography QC activities at CDC. Author summary: After COVID-19, tuberculosis is the second leading cause of death from infectious disease in the world. The U.S. has relatively low rates of the disease—about 2.5 cases per 100,000 members of the population in 2022—but immigrants, refugees, and other migrants seeking entry into the U.S. often come from areas where the background rates are much higher. To help treat these populations and prevent disease from being imported into the U.S., the Division for Global Migration Health (DGMH) at the Centers for Disease Control and Prevention (CDC) oversees a health screening program for applicants seeking entry. To help detect tuberculosis, most of these applicants receive a chest x-ray, which is then checked for signs of the disease. DGMH receives around half a million of these x-rays each year and conducts ad-hoc quality control assessments to make sure the panel physicians overseas are interpreting the x-rays according to the screening program's standards. To make these assessments more efficient, we developed a machine learning algorithm that can reliably detect signs of tuberculosis in the x-rays. In testing, the algorithm worked well on a variety of datasets, suggesting it will be a good tool for supporting these important quality control efforts.

WEST Indies; AUSTRALIA; POLYNESIA; ASIA; AFRICA; EUROPE; NEW Zealand; UNITED States; TUBERCULOSIS diagnosis; IMMIGRANTS; COMPUTER-assisted image analysis (Medicine); PREDICTION models; RECEIVER operating characteristic curves; RESEARCH methodology evaluation; ARTIFICIAL intelligence; CLINICAL decision support systems; DIGITAL health; COMPUTED tomography; CHEST X rays; DESCRIPTIVE statistics; INFORMATION resources; TEACHING methods; EXPERIMENTAL design; SURVEYS; RESEARCH methodology; DEEP learning; ARTIFICIAL neural networks; DIGITAL image processing; CONFIDENCE intervals; TUBERCULOSIS; REFUGEES; LATENT tuberculosis; SENSITIVITY & specificity (Statistics); SYMPTOMS

PLoS Digital Health, 2024, Vol 3, Issue 9, p1

2767-3170

Academic Journal

10.1371/journal.pdig.0000612