Cognitive assessment can help measure surgeons' expertise

What clues might brain metrics hold about the skill levels of surgeons who perform robot-assisted surgeries?

Looking for better ways to assess the proficiency of surgeons performing these complex procedures, researchers at UB and Roswell Park Cancer Institute (RPCI) set out to determine whether cognitive assessment can effectively measure the expertise of robotic surgeons with varying levels of experience. They found that assessment of robotic surgeons’ cognitive processes during surgery gives a fuller, more reliable picture than other measurable indicators and may be a valuable element to incorporate into robotic-surgery training.

The study, published online ahead of print in the journal BJUI, is the first published report to evaluate surgeons’ cognition during robot-assisted surgery.

The research team evaluated novice, competent and expert robotic surgeons using both tool-based measures available through the daVinci Surgical System and continuous wireless electroencephalography (EEG) recording. Most of the 10 surgeons enrolled — two residents, three fellows and five practicing surgeons — were assessed on basic, intermediate and advanced surgical tasks. Those assigned to the novice group were assessed only on basic and intermediate skills. Each participant completed a pre-recording session to establish baseline responses.

For the cognitive testing, participants’ cognitive engagement, mental workload and mental state were calculated from the measured EEG during each task. When researchers compared the results of both testing approaches among the three groups, they found that cognitive assessment detected significant differences that were not identified by the tool-based metrics available through the surgical robot.

“The real distinctions emerged when we looked at the cognitive differences between our intermediate and expert groups,” notes the study’s first author, Khurshid Guru, director of robotic surgery and the Robert P. Huben Endowed Professor of Oncology at RPCI.

“We saw that the middle group, those considered competent but not expert, demonstrated greater high-level engagement, higher mental state and more cognitive load to complete each task when performing intermediate and advanced tasks — in other words, they had to work harder and devote more mental resources than the expert group,” adds co-author Ehsan T. Esfahani, assistant professor in UB’s Department of Mechanical and Aerospace Engineering and director of the UB Brain Computer Interface Laboratory.

The researchers concluded that cognitive assessment is an effective complement to traditional methods of assessing the skill level of robotic surgeons at various stages of development and can be used to help tailor training programs to their individual needs. The authors acknowledge that further testing will be needed to validate the findings of this small study.

“Our results truly provided a window into how novice robotic surgeons develop into expert robotic surgeons, how they evolve to a point where they’re using their intuition, pre-planning their actions and demonstrating high comfort and competence in interacting with the machine,” says Guru, who is also serves as director of RPCI’s ATLAS Project, or Applied Technology Laboratory for Advanced Surgery. “The implications for patient safety are striking. I think in the future we’re going to see cognitive assessment widely incorporated into surgeon training programs, and the model our team developed is a good foundation for subsequent investigations.”

The testing was performed at RPCI in collaboration with UB’s Brain Computer Interface Laboratory. Assessments were conducted using training modules and surgeries performed on synthetic organ models; no live surgeries were involved.

The study, titled “Cognitive Skills Assessment During Robot-Assisted Surgery: Separating Wheat from Chaff,” was funded in part by the Roswell Park Alliance Foundation and the John R. Oishei Foundation.