Assessing clinical reasoning skills using Script Concordance Test (SCT) and extended matching questions (EMQs): A pilot for urology trainees

Abstract

Introduction: Clinical reasoning skill is the core of medicalcompetence. Commonly used assessment methods for medicalcompetence have limited ability to evaluate critical thinking andreasoning skills. Script Concordance Test (SCT) and Extended Matching Questions (EMQs) are the evolving tests which areconsidered to be valid and reliable tools for assessing clinicalreasoning and judgment. We performed this pilot study to determinewhether SCT and EMQs can differentiate clinical reasoning abilityamong urology residents, interns and medical students. Methods: This was a cross-sectional study in which an examinationwith 48 SCT-based items on eleven clinical scenarios and four themedEMQs with 21 items were administered to a total of 27 learners atthree differing levels of experience i.e. 9 urology residents, 6 internsand 12 fifth year medical students. A non-probability conveniencesampling was done. The SCTs and EMQs were developed fromclinical situations representative of urological practice by 5 contentexperts (urologists) and assessed by a medical education expert.Learners' responses were scored using the standard and the graduatedkey. A one way analysis of variance (ANOVA) was conducted tocompare the mean scores across the level of experience. A p-value of <0.05 was considered statistically significant. Test reliability was estimated by Cronbach α. A focused group discussion with candidates was done to assess their perception of test. Results: Both SCT and EMQs successfully differentiated residents from interns and students. Statistically significant difference in mean score was found for both SCT and EMQs among the 3 groups using both the standard and the graduated key. The mean scores were higher for all groups as measured by the graduated key compared to the standard key. The internal consistency (Cronbach's α) was 0.53 and 0.6 for EMQs and SCT, respectively. Majority of the participants were satisfied with regard to time, environment, instructions provided and the content covered and nearly all felt that the test helped them in thinking process particularly clinical reasoning.<br />Conclusion: Our data suggest that both SCT and EMQs are capable of discriminating between learners according to their clinical experience in urology. As there is a wide acceptability by all candidates, these tests could be used to assess and enhance clinical reasoning skills. More research is needed to prove validity of these tests.