We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Detection of Residents With Progress Issues Using a Keyword–Specific Algorithm.
- Authors
Tremblay, Gaby; Carmichael, Pierre-Hugues; Maziade, Jean; Grégoire, Mireille
- Abstract
The literature suggests that specific keywords included in summative rotation assessments might be an early indicator of abnormal progress or failure. This study aims to determine the possible relationship between specific keywords on in-training evaluation reports (ITERs) and subsequent abnormal progress or failure. The goal is to create a functional algorithm to identify residents at risk of failure. A database of all ITERs from all residents training in accredited programs at Université Laval between 2001 and 2013 was created. An instructional designer reviewed all ITERs and proposed terms associated with reinforcing and underperformance feedback. An algorithm based on these keywords was constructed by recursive partitioning using classification and regression tree methods. The developed algorithm was tuned to achieve 100% sensitivity while maximizing specificity. There were 41 618 ITERs for 3292 registered residents. Residents with failure to progress were detected for family medicine (6%, 67 of 1129) and 36 other specialties (4%, 78 of 2163), while the positive predictive values were 23.3% and 23.4%, respectively. The low positive predictive value may be a reflection of residents improving their performance after receiving feedback or a reluctance by supervisors to ascribe a "fail" or "in difficulty" score on the ITERs. Classification and regression trees may be helpful to identify pertinent keywords and create an algorithm, which may be implemented in an electronic assessment system to detect future residents at risk of poor performance.
- Subjects
REGRESSION trees; RECURSIVE partitioning; ELECTRONIC systems; RESIDENTS (Medicine); ALGORITHMS
- Publication
Journal of Graduate Medical Education, 2019, Vol 11, Issue 6, p656
- ISSN
1949-8349
- Publication type
Article
- DOI
10.4300/JGME-D-19-00386.1