Journal of Research in Medical Education & Ethics

For use of MCQ's for credible assessments, item analysis is widely recommended. However, it is difficult to perform manually. Computer-based assessments and analysis has made it possible to administer large number of items and report item statistics with much fewer resources. However, little is known about how educators are using item analysis guidelines within health professions education without negative consequences to examination reliability, length or difficulty.

We recently introduced online MCQ based within course assessments in the undergraduate medical curriculum in a batch of 250 students of pre-clinical year. Since educators may be tempted to use computer-based analysis for determining item statistics due to ease of its use, we determined the relationship between difficulty index and discrimination index of these items for two different subjects. After removing items with negative discrimination index and item writing flaws, the relationship between difficulty and discrimination index was described by a polynomial fit.

After removing very difficult (difficulty index < 0.2) and very easy questions (difficulty index > 0.8) the relationship between the two became linear suggesting that relatively easier questions had a higher discriminatory value. Since the quizzes were part of within-course assessments, most probably students had not achieved the desired level of learning.

Though, item analysis using computer-based assessments is apparently useful, care should be taken that they are taken from examinations which are administered under the similar testing conditions.