Improving Patent Safety through Innovations in Surgical

Improving Patient Safety through Innovations in Surgical Training and Machine Learning

Simulated Inanimate Models combines augmented reality with synthetic anatomy to create a novel “Flight Simulator for Surgery.”

by Michael P. Wilson, PhD

The Luminate accelerator – an optics, photonics, and imaging (OPI) program – concluded its third cohort in September of this year, with the Rochester-based company Simulated Inanimate Models LLC (SIM) receiving the $400,000 Outstanding Graduate Award. The company’s ESIST™ surgical training platform is candidly described as a “flight simulator for surgery,” a technology that enables surgeons to acquire skill and certification without risking a patient’s life. The technology combines realistic, synthetic models of human anatomy (called “phantoms”) with educational software and augmented reality (AR) technology to deliver an immersive training experience. The ESIST™ platform can also be applied to broader medical training in operating rooms and hospitals, including for nurses, technicians, and other medical supportive staff.

The phantom technology and simulation work were originally developed in part by company founder Dr. Jonathan Stone, an attending neurosurgeon and engineer at the University of Neurosurgery residents at Strong Memorial Hospital resect synthetic Rochester Strong Memorial Hospital. Dr. Stone brain tumors from a SIM phantom using fluorescent microscopy as began creating these realistic phantoms during part of a training workshop. his residency to give surgeons a safer and more effective means of acquiring the skills required. SIM was launched as a startup company from the university in July 2018, with support from an NIH STTR grant award to commercialize the phantom technology. One of the greatest strengths of SIM is the cross disciplinary diversity of its founders, including CSO Dr. Nelson Stone’s 40 year clinical experience and serial entrepreneurship, COO Dr. Michael Wilson’s Ph.D. training as a research scientist, and CTO Steve Griffith 30+ year career as a electrical/computer engineer with flight simulation and medical device experience. With the diverse team assembled, SIM submitted and was awarded an NSF SBIR Phase I grant to integrate augmented reality, artificial intelligence and gaming technology into its patented ESIST platform.

Solving Shortcomings in Surgical Education

While simulation training has revolutionized the manner in which professionals in other fields acquire skills for complex tasks – the airline industry being the gold standard – these training innovations have been lacking for surgery. Modern surgical training remains centered around an apprenticeship-style of education that involves novice surgeons acquiring skills during procedures on live patients, under heavy 1-on-1 guidance and supervision from expert teaching surgeons; this dynamic persists due to the overall lack of effective training methods. Available surgical training tools all exhibit significant shortcomings, including existing phantoms, cadavers, animals, and virtual reality (VR) trainers. Perhaps most importantly, none of these tools provide the guidance and instruction critical for surgical education.

SIM phantoms replicate every step of complex surgical

SIM’s ESIST platform solves all of these issues by procedures, including integration with operating room systems combining physical realism with real-time instruction and software tools. and education. The company’s phantoms are made from a synthetic, water-based hydrogel material with lifelike biomechanical behavior. Unlike competing phantoms, SIM’s models enable hands-on cutting, suturing, needle insertions, tumor resections, etc., and they are able to replicate complex procedures normally performed in the operating room. These phantoms are combined with educational software running on AR hardware that guides a surgeon through a complete procedure.

ESIST uses machine learning to read the surgeons interactions with the phantom and deliver instruction, guidance, and feedback in a timely manner as the surgeon progresses through the simulated procedure. The software detects common errors in real-time and issues corrective instruction; the technology is meant to simulate the presence of an expert teaching surgeon standing beside the trainee and giving instruction. Additionally, the system tracks key metrics that are critical for the success of a procedure, scoring the performance of the surgeon and allowing them to train to a level of skill and become certified for a given medical device or procedure.

All of this information is overlaid in real-time through the see-through optic of the AR headset, freeing up the surgeon’s hands to perform “real” procedures on SIM’s phantoms in a comprehensive, immersive training experience. SIM’s team labels this a “flight simulator for surgery,” with the hopeful goal of transitioning the remarkably low error rate and training efficiency found in the airline industry to modern surgery.

Standardized Training

The instructorless education and performance-scoring features of the ESIST platform open up major possibilities for surgical simulation and broader medical education. SIM’s founders envision early-stage surgical residents acquiring “instructed” repetitions with the company’s products in a simulation setting, honing their skills until they are able to continue their training safely in the operating room, under instruction from experienced attending surgeons. On the other hand, as novel medical devices are released onto the market and new procedures become the standard of care, even experienced surgeons require hands-on training to bring them up to speed. The ESIST platform enables surgeons all over the world to obtain training that is standardized and graded according to the same criteria, regardless of which country the surgeon practices in, which medical center or clinic they’re at, etc. SIM envisions this training dynamic expanding beyond surgeons to cover additional hospital staff, and even medical and nursing students.

Following two years of consistent sales growth for its phantoms, the company continues to develop its ESIST technology and acquire pilot customers. With support from Luminate and the NSF’s SBIR program, SIM will be engaging the considerable engineering and programming resources in the Rochester area to commercialize this technology. The company will also build out its manufacturing and sales capabilities as it continues to grow. Overall, SIM is a great example of how an early stage startup can quickly gain traction with the right founding team and support from the local entrepreneurial community.

A video of SIM’s working ESIST prototype can be seen at: https://www.youtube.com/ watch?v=WWG8R4n67yk. q

Michael P. Wilson, PhD is the COO of Simulated Inanimate Models, LLC

ESIST delivers audiovisual instruction in real time.

The following screencaps illustrate how content using the see-through optic of an AR-headset as a surgeon progresses through a simulated partial nephrectomy procedure on a SIM phantom. The procedure is outlined in full, each step is explained and demonstrated through audio visual instruction, including picturein-picture video, and the system automatically detects and scores progress as the user interacts with the phantom. Error count and grade are delivered at the end to allow for performance evaluation and certification.

1. Procedural Outline

Instruction for Step 5, PiP video:

3. Step 5 Completed, Instructions for Step 6:

4. End of Procedure, Summary of Performance, Grade and Score:

Neurosurgery training workshop, demonstrating tumor resection using fluorescent microscopy on a SIM phantom.

Hydrogel kidney phantom (courtesy of URMC Strong Memorial Hospital)