We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A confidence and safety approach to multiple-choice question scoring.
- Authors
Tweed, M. J.; Thompson-Fawcett, M.; Schwartz, P.; Wilkinson, T. J.
- Abstract
Aims: This study explored the effect of a scoring system that incorporates a rating of the candidates' confidence in their responses and the safety of the chosen responses in practice. BackgrountL Multiple-choice question (MCQ) tests with number-correct scoring are widely used in high-stakes examinations. Concerns about numbercorrect scoring include the implication that guessing is acceptable practice and uncertainty whether an incorrect response reflecting an unsafe decision in practice is a true belief or a random guess. Methods: A scoring system was developed in which responses-included descriptors for levels of confidence, as defined by the likelihood of contacting a colleague/reference. Incorrect responses were also reviewed in advance fOr the level of safety in practice. An optional MCQ paper was then offered to medical students at the University of Otago. A randomized cross-over design with four versions of the paper was used. Each paper had two sets of 12 questions in different order, one set to be answered with number-correct scoring instructions and the other with the new system instructions. Results: The survey was completed by 372 students. Score reliability was sufficient, and the scoring system instructions and year of the student both led to differences in responses. Students gave fewer unsafe responses when scored using the safety and confidence instructions. Discussion: Making students aware of potential unsafe responses, and allowing them to admit a lack of confidence, had the effect of reducing answers that reflected unsafe decisions, without decreasing the number of correct responses.
- Subjects
MULTIPLE choice examinations; GUESSING (Educational tests &; measurements); TEST scoring; CONFIDENCE testing; UNIVERSITY of Otago
- Publication
Focus on Health Professional Education, 2012, Vol 13, Issue 3, p84
- ISSN
1442-1100
- Publication type
Article