Using modified Direct Observation of Procedural Skills (DOPS) to assess undergraduate medical students

Abstract

Introduction: Nowadays according to competency basedcurriculum, selecting an appropriate assessment method isinevitable. This study aimed to investigate application of DirectObservation of Procedural Skills (DOPS) in undergraduatemedical students.Methods: This is a cross sectional study conducted duringemergency ward rotation in last year medical students usingconsensus sampling method. Each student performed 2 proceduresat least twice under the observation of 2 assessors using modifiedDOPS rating scales designed for each procedure simultaneously.Correlation between DOPS score and final routine exam wasmeasured. Face and content validity was determined by thepanel of experts. Moreover, through the test-retest and interraterreliability, the correlation of each score and total score wasinvestigated. The spent time was calculated too. The statisticalanalysis was carried out using SPSS version 18.Results: Totally 60 students did 240 procedures under DOPS.The face and content validity confirmed by an expert panel. Thefindings showed that there was a significant correlation betweenthe scores of each test and the total DOPS score (r1=0.736**,r2=0.793**, r3=0.564**, r4=0.685**; p<0.001). There was a significantcorrelation between the first and second scores of doing the sameprocedure (Pearson Cor.=0.74, p<0.001) and also between thescores of the two individual examiners when observing the sameprocedure (Pearson Cor.=0.84-0.94, p<0.001). The results showedthat there was no correlation (Pearson Correlation=0.018, p<0.89)between the scores of this test and the final routine ward examscores. The average time for doing DOPS test and the averagetime for providing feedback were 11.17±7.5 Max and 9.2±4.5 Min,respectively.Conclusion: The use of novel performance assessment methodssuch as DOPS is highly beneficial in order to ensure the adequacyof learning in medical students and assess their readiness foraccepting professional responsibilities. DOPS as a practical andreliable test with acceptable validation can be used to assessclinical skills of undergraduate medical students.Keywords: Reliability, Validity, Feasibility, Satisfaction, Undergraduate, Medicalstudent