Effect of formative evaluation using direct observation of procedural skills in assessment of postgraduate students of obstetrics and gynecology: Prospective study

Abstract

Introduction: Direct Observation of Procedural Skills (DOPS)is a way of evaluating procedural skills through observation inthe workplace. The purpose of this study was to assess the role ofDOPS in teaching and assessment of postgraduate students and toknow the effect of repeated DOPS on improvement of the skillsand confidence of the students.Methods: This prospective study was conducted in the Obstetricsand Gynecology department of a rural tertiary centre in NorthernIndia in two phases. All postgraduate students (n=20) wererandomly divided year-wise into two groups, using lottery systemsuch that both groups had equal number of first, second andthird year students for didactic lecture/simulation teaching usingmannequins, with hands-on exposure on Active management ofthird stage of labor (AMTSL) and Postpartum hemorrhage (PPH)management. In the first phase, Group 1 received simulationteaching using mannequins with hands on for 4 sittings, and Group2 received didactic lecture on AMTSL. Following the students’performance of AMTSL steps on delivering women, theircompetencies were assessed using DOPS structured checklist often points on 6 sessions. The students’ performance in six DOPSwas compared between the two groups. In the 2nd phase, afterflipping of the groups, group one received didactic lecture, andgroup 2 simulation teaching on PPH management with hands onfor 4 sittings, followed by comparison of six DOPS performancein both groups. The data were analyzed by Wilcoxon rank-sum(Mann–Whitney) test using SPSS software, version 20.Results: In both phases, significant difference was observedbetween the two groups on first DOPS comparison (1st phase:p=0.000; 2nd phase: p=0.002), with simulation group performingbetter. Comparison of sixth DOPS in the two groups revealed nodifference in both phases, but significant difference on first andsixth DOPS comparison in each group (p=0.000).Conclusion: Repeated DOPS results in improved skills andconfidence of students in managing real life obstetric emergenciesirrespective of the teaching modality.Keywords: Direct observation; Procedural skills; Postpartum hemorrhage; Clinical competency