Assessing the quality of multiple-choice questions in allied health science summative exams: A retrospective analysis

Abstract

Introduction: Multiple-choice questions are feasible, reproducible and cost-effective; hence, they are widely embraced in health professions education. The quality of multiple-choice questions is monitored statistically by item analysis. Item analysis confirms whether the questions measure the intended learning outcomes, ensuring fair and equitable assessments. To promote quality assessment, we analyzed the quality standards of multiple-choice questions in a summative examination using item analysis.

Methods: The multiple choice questions answered by 38 students in the first semester of the allied health science programme of Madan Bhandari Academy of Health Sciences in Bagmati Province were collected, examined, and analyzed. The data gathered were subjected to the performance of each multiple choice question by analysis of difficulty index, discrimination index, and distractor effectiveness.

Results: Items of most courses were acceptable (30% to 70%) as per the difficulty index with only three courses having (>70%) easy items. Poor discriminatory index <0.20=poor) was noted in seven courses, particularly discipline-specific courses. A significant proportion of excellent distractors were identified in two courses: 50% each in Medical Terminology & Musculoskeletal and Fundamentals of Pharmacy. Most of the courses had more than 50% of functional distractors.

Conclusions: The multiple-choice questions in most of the courses had poor discrimination index. These outcomes highlight the need for careful formulation of multiple-choice questions for authentic assessment.