Effectiveness of direct observation of procedural skills as an assessment tool in anesthesiology for postgraduate students

Background: Competency-based medical education highlights the importance of assessing practical procedural skills alongside theoretical knowledge. Direct observation of procedural skills (DOPS) is a validated workplace-based assessment (WPBA) tool, though its application in Indian anesthesiology training remains limited. Aims and Objectives: This study evaluated the feasibility, reliability, and effectiveness of DOPS for postgraduate anesthesiology students. Materials and Methods: A 2-year interventional study was conducted in the Department of Anesthesiology, Rajarshi Dashrath Autonomous State Medical College, Ayodhya, including nine postgraduate students from all training years and seven faculty members. Key anesthetic procedures-spinal anesthesia, epidural anesthesia, ultrasound-guided central venous catheterization, and laryngoscopy with endotracheal intubation – were assessed using a structured DOPS pro forma covering nine domains on a 5-point Likert scale. Faculty was trained to deliver standardized feedback. Statistical analysis employed analysis of variance, independent and paired t-tests, Chi-square tests, and intraclass correlation coefficient (ICC) models, with P<0.05 considered significant. Results: Baseline competency improved significantly across training years (P<0.05). Longitudinal paired analysis demonstrated consistent enhancement of procedural performance over 2 years (P<0.001). Faculty perceived DOPS as more effective than trainees in fostering skill acquisition, patient safety, and feedback quality (P<0.05). Safety compliance was high, with aseptic adherence (94.2%) and patient communication (91.7%) showing significant results. Inter-rater reliability was excellent, with ICC values ranging from 0.84 to 0.92 across domains. Conclusion: DOPS appears to be a feasible and reliable WPBA tool for postgraduate anesthesiology training, with potential to enhance procedural competence and structured feedback. However, given the small sample size and single-institution design, these findings should be considered preliminary, and larger multicentric studies are required to confirm its effectiveness and generalizability.