The Stanford Microsurgery and Resident Training (SMaRT) Scale: Validation of An On-Line Self-Directed Curriculum

Purpose:   Plastic surgery education has traditionally been built on the apprenticeship model, where experts teach skills to novices using real patients. While this is a well-established process, there are limitations and areas for improvement to maximize the educational benefit of residents and to optimize patient safety.   Our current study provides an objective means for expert evaluators to reliably rate trainees’ technical skills under the microscope.

Methods: “Microsurgery Essentials” (http://insideplastics.stanford.edu/microsurgery) is our novel on-line curriculum providing basic information and training on microsurgery.  The website has step-by-step instructions, as well as multiple intra-operative videos narrated by senior faculty. Residents were randomly divided into two groups:  one group reviewed this online resource and the other did not.  Pre- and post-tests consisted of videotaped microsurgical sessions in which the trainee performed “microsurgery”.  The videotapes were reviewed and graded by expert senior level surgeons using our novel global rating scale to determine technical improvement of each participant.  A 5-point Likert scale was employed, and results were analyzed with ANOVA and student’s T-test, with p<0.05 indicating statistical significance.

Results:  Fifteen residents participated in the study, ranging from PGY-1 to PGY-6.  There were no significant differences in scoring among all three evaluators (p>0.05).  The SMaRT scale adequately differentiated the performance of more experienced senior residents (PGY-4 to PGY-6, total average score = 3.43) from less experienced junior residents (PGY-1 to PGY-3, total average score = 2.10, p<0.0001), conferring validity.  Residents who had access to the website were more likely to perform microanastomoses on vessels faster with an average of 4.5-minute improvement compared with a 1.25-minute change among the control group.  Additionally, junior residents who had access to our website showed a significant increase in their graded technical performance by 0.7 points when compared to residents who did not have access to the website who showed an improvement of only 0.03 points (p=0.01).

Conclusions:  Our SMaRT scale is reliable and valid in assessing the microsurgical skills of residents and other trainees.  As self-directed online education becomes more and more the norm, an efficient and easy-to-use global rating scale is needed to ensure that residents are achieving appropriate technical milestones.  With appropriate standardization, web-based programs may ultimately be more cost-effective and allow more wide spread access than traditional microsurgical training sessions.