St. Michael’s doctor develops skills test for surgical residents By Greg Winson
A technical exam for colorectal surgery fellows piloted by a St. Michael’s physician could lead to a paradigm shift in certification for all surgical fellows. Surgical residents are currently assessed on their knowledge through written exams and for their judgment through oral exams. However, there is no formal assessment of technical skill at the time of certification. Dr. Sandra de Montbrun, a colorectal surgeon , has been working with the American Society of Colon and Rectal Surgeons to develop a technical skills exam for the purpose of certification in the United States. “This is the first time that any North American surgical society has moved forward with developing a technical skills exam with the purpose of certification for high stakes assessment,” said Dr. de Montbrun. Implementing this sort of test could lead to improved patient outcomes. “If we can identify the residents who show deficiencies in technical skill and remediate them during their training, there is a potential impact on patient care,” she said. The technical exam takes place in a surgical skills lab setting and is made up of eight different technical skill tasks. The students are observed by an examiner who evaluates their performance. She has led three pilot studies to prove the validity of the technical exam. The first study, held at the University of Toronto in 2011, compared general surgery residents to colorectal residents. “We found there was a difference in their performance, giving some initial evidence of validity to the test, said Dr. de Montbrun. The results from the second pilot study suggested that this
Dr. Sandra de Montbrun evaluates a surgical fellow in the Allan Waters Family Simulation Centre skills laboratory. (Photo by Katie Cooper, Medical Media Centre)
exam identifies technical deficiencies in people who would otherwise go on to be certified with the current board certification process. For 2014, the Colorectal Objective Structured Assessment of Technical Skill, or COSATS, exam became a mandatory component for certification for the American Board of Colon and Rectal Surgery. It was the first time in North America that a technical skills exam was a required component for certification. “The purpose of the exam was to collect data on the exam itself,” said Dr. de Montbrun. “Candidates were not assigned a pass/fail status, but we had to administer it to the entire cohort of people to get an idea of what the data would look like with the entire group of examinees taking their board exam.” The results of this most recent pilot have been submitted for publication. There is no timetable for the COSATS exam to become a permanent component of The American Board of Colon and Rectal Surgery exam. At the same time, the American College of Surgeons is interested in moving forward with a technical exam for general surgery training in the United States. The Royal College of Physicians and Surgeons of Canada does not yet have plans to incorporate the COSATS into Canadian certification. DECEMBER 2015 / JANUARY 2016 | IN TOUCH | 5