We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
Reliability and validity of procedure-based assessments in otolaryngology training.
- Authors
Awad, Zaid; Hayden, Lindsay; Robson, Andrew K.; Muthuswamy, Keerthini; Tolley, Neil S.
- Abstract
Objectives/Hypothesis To investigate the reliability and construct validity of procedure-based assessment (PBA) in assessing performance and progress in otolaryngology training. Study Design Retrospective database analysis using a national electronic database. Methods We analyzed PBAs of otolaryngology trainees in North London from core trainees (CTs) to specialty trainees (STs). The tool contains six multi-item domains: consent, planning, preparation, exposure/closure, technique, and postoperative care, rated as 'satisfactory' or 'development required,' in addition to an overall performance rating (pS) of 1 to 4. Individual domain score, overall calculated score (cS), and number of 'development-required' items were calculated for each PBA. Receiver operating characteristic analysis helped determine sensitivity and specificity. Results There were 3,152 otolaryngology PBAs from 46 otolaryngology trainees analyzed. PBA reliability was high (Cronbach's α 0.899), and sensitivity approached 99%. cS correlated positively with pS and level in training ( rs: +0.681 and +0.324, respectively). ST had higher cS and pS than CT (93% ± 0.6 and 3.2 ± 0.03 vs. 71% ± 3.1 and 2.3 ± 0.08, respectively; P < .001). cS and pS increased from CT1 to ST8 showing construct validity ( rs: +0.348 and +0.354, respectively; P < .001). The technical skill domain had the highest utilization (98% of PBAs) and was the best predictor of cS and pS ( rs: +0.96 and +0.66, respectively). Conclusions PBA is reliable and valid for assessing otolaryngology trainees' performance and progress at all levels. It is highly sensitive in identifying competent trainees. The tool is used in a formative and feedback capacity. The technical domain is the best predictor and should be given close attention. Level of Evidence NA Laryngoscope, 125:1328-1335, 2015
- Subjects
LONDON (England); OTOLARYNGOLOGY education; MEDICAL equipment reliability; CURRICULUM; TISSUES; MEDICAL technology
- Publication
Laryngoscope, 2015, Vol 125, Issue 6, p1328
- ISSN
0023-852X
- Publication type
Article
- DOI
10.1002/lary.24983