White Paper
AI-Xperential Collaboration – Holographic Training and Autonomous Surgery Artificial Intelligence Virtual Reality platforms combined with the power of 3D content, digital health applications enable virtual training, patient engagement, education and health coaching Authors Navin Govind Aventyn, Inc.
Moh Barathan 8chili, Inc.
Current surgical training simulators and staging areas are expensive and fixed modalities in teaching hospitals, medical training facilities and high acuity surgical centers. Recently, many practice specific specialties in medical training have commercialized online training, offering certified education with hands on approaches. However, surgical skills training continue to lag because of a lack of portable, cost effective, standardized and connected innovative platforms to enable dynamic immersive experiences. Surgical specialties typically have hundreds of unique surgical procedures and a myriad of complications that are overlooked and missed, many of which can be effectively captured on Virtual Reality (VR) simulators. Truly connected digital health 3D surgical training platform capabilities are slim to non-existent other than internet video recordings. In a pandemic world, the market for VR is estimated to grow to over $60B by 2027 impacting global economies in a health crisis. Emerging long term challenges of managing chronic disease patients with post Covid-19 long haul complications can potentially be addressed by immersive VR and AR solutions as interactive ways to educate patients about disease progression, therapies and treatment methods. “There are 143 million surgical procedures missed globally which is a significant unmet need,” said Moh Barathan, Chief Operating Officer at 8chili. “Today, 97% of surgeons prefer to practice or rehearse surgical procedures prior to operating room surgery. Average cost of traditional surgical simulators in the hundreds of thousands make this a money pit and magnified cost center, we need innovative 3D connected solutions to overcome such barriers to cost and scale for wider adoption.”
White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
1
As published in the Harvard Business Review1, the present approach to training surgeons’ has placed barriers for some doctors having to perform complex surgical procedures with outdated infrastructure and resources. More importantly, patients may be placed in an undesirably risky situation pre and post op with skill shortages in surgeons and residents. VR training platforms provide an immersive experience to overcome many traditional training challenges by creating procedure specific operating room simulation modules to mimic hands-on training that assist surgeons and residents helping fine tune skills anytime, anyplace “With the pandemic environment, validated immersive virtual collaboration solutions are indispensable for surgical training to interact and arm the next generation of surgeons for anywhere, anyplace pre-op planning, surgical training, remote collaboration, and 3D automated guided surgeries,” said Barathan. “More importantly, innovative AR and VR applications in digital therapeutics solutions for virtual care with AI-Xperential Collaboration enabling patient engagement, health coaching and immersive education are needed urgently. VR is enabling care teams increase patient awareness by allowing patients to interact with 3D models of disease progression, help stroke rehabilitation patient’s measure and improve cognitive abilities”
Virtual Reality in Heart Failure and Atrial Fibrillation Artificial intelligence has shown the ability to detect and recognize patterns that elude humans, providing unique opportunities to further understand disease pathophysiology, diagnoses and treatments. AI has been used in AR and VR technology which has already shown tremendous promise, with the US Food and Drug Administration approving AI medical devices and clinically validated algorithms for diagnosis in the last year. Many health innovation centers and care delivery platforms across the USA are on the path to fully immerse in applying AI technology to the many different facets of its current practice such as detection of asymptomatic left ventricular dysfunction (ALVD) from the surface electrocardiogram (EKG), screening for occult atrial fibrillation (AF), training surgeons and structural heart disease specialists. Electrophysiologists (EP) in practice and those in training both in the western hemisphere and around the world are demonstrating hands on training with Mixed Reality (MR) technology in real time as well as educating and engaging the patient community. “An exciting opportunity exists in utilizing Mixed Reality technology to address both AF and Congestive Heart Failure (CHF) using virtual reality – what used to be complex technology that is now simplified for HF and AF treatment,” said Kris Vijay, MD, Medical Director at Abrazo Arizona Heart and board-certified heart failure specialist. “To this end, we are excited about the potential of validated VR solutions in cardiology and the intersection of HF and AF. Through visual images and description as well as surgical training demonstration, we aim to bring some astounding 3D content, simplify the underpinnings and capture the VR experience in profound ways that may translate to a better patient outcome from these disease states.”
White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
2
Studies have indicated the feasibility of applying VR in cardiac ablation, a procedure for restoring normal heart rhythm when medication therapy has not responded. In particular, VR recently has shown potential to improve analgesia and reduction of pain perception in a small sample of patients undergoing transcatheter aortic valve implantation (TAVR). Additionally, VR digital therapeutics applications evidenced for stroke rehabilitation have shown promise for improving limb recovery with immersive therapies and treatment. “About 20% of patients with a stroke from AF are unaware of AF prior to the stroke diagnosis. Although the U.S. Preventive Services Task Force has concluded that there is insufficient evidence to determine whether the benefits of screening asymptomatic individuals for AF outweigh the harms,” said Renzo Cataldo, MD and board-certified cardiac electrophysiologist with Northern Arizona Healthcare Cardiology. “Development of innovative methods to detect and treat those patients at risk of developing AF could provide an opportunity to implement primary and secondary education and treatment strategies with VR to reduce the risk of complications from AF and stroke.”
Immersive 3D Content Creation Enabling a truly immersive VR experience is critical in capturing meaningful content that can be curated and annotated by experts in specific practice areas. True 3D occlusion rendered immersive experiences help surgeons and residents “ReLive” the surgery and experience personalized “virtual teleportation” that is impactful, simple and easy to use. 3D content is captured by a surgeon wearing and positioning a purpose built 3D capture headband in a surgeon’s optimal field of view via wireless connectivity of the capture device to a compute client. The curated annotated procedures are launched on the secure client or cloud to experience the 3D content platform in a secure staging and training area. Clinical residents can view patient scans in 3D recon, trajectory planning, surgical procedure details and more that enrich the learning curve with hand tracking, eye tracking interactivity tools for trained surgeons to assess pre and post-op performance. This unique 3D content capture is made possible by 8chili with a purpose built HintVR 3D capture device which is light weight, comfortable and ergonomic. The 3D capture headband sits on the crown of the user similar to a headband, compatible with existing surgical attachments such as loupes and surgical headlamps and takes just minutes to setup. These features help the surgeon focus on the surgery rather than being distracted with headband adjustments, especially in a lit OR with surgical team and staff movements. “The VR surgical experience with HintVR is super sticky, interactive and immersive with true depth 3D. The ability to zoom into the surgeon’s field of view of the procedure and see granular details help surgeons and medical residents comprehend the nuances of a surgical procedure like never before,” said Barathan. “The sensation of hand tracking is a novel experience. HintVR occlusion rendering, true 3D spatial reconstruction with sub millimeter depth accuracies makes the entire hand tracking fun and exciting. Controlling the virtual tools console, rehearsing the actions superimposed on the original recorded White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
3
and annotated procedures deliver realism to the entire surgical training experience teleporting the user back in time to ReLive the procedure.” Access to high quality content never gets old in the eyes of experienced surgeons, residents or surgeons starting post residency and future surgical robots. These VR platforms are now capable of massive societal change by delivering quality training to surgeons, residents, medical professionals and more importantly for patient engagement towards wellness and healthy living, globally democratizing access to quality content across the health care ecosystem.
Autonomous Surgery The capability in being able to place numerous surgical procedures - open, endoscopic, laparoscopic, robotic surgeries - into the hands of residents to experience and train in full 3D is implemented in a matter of a few hours post-surgery capture is extraordinarily innovative and revolutionary. The platform is medical specialty agnostic – the capture device enables unique training content and learning materials to be created in 3D for multiple modalities and specialty such as Cardiology, Orthopedics, Neurology, ENT, General Surgery, Urology, Vision, Dental and more. The 3D content management system is easy to use with a rich set of annotation tools powered with AI that enable annotations easier and accurate and multilingual support. The content management system is extensible to add patient diagnosis, procedure abstract, complications, pre-op plan, patient scans that make learning more engaging. 4K and higher definition resolution 3D surgical content is used to train robust machine learning models in assisting heart surgeons as well as enable a new class of robots perform minimally invasive, key hole surgeries, replacement surgeries with sutures within spatial boundaries guided in a virtual environment with computer vision and clinically evidenced rich data lake. While there are several clinical trials and ongoing studies in comparing different modalities of robotic surgery, it is imminent rich 3D imaging VR content that is curated and annotated demonstrate the significant benefits of virtual reality in robotic simulation training, autonomous surgery with expert machine learning methods.
VR Patient Engagement, Coaching and Education Aventyn, Inc with AI-Xperential Collaboration is forging broader uses of VR that benefit the digital health ecosystem. The HintVR capture hardware is simple and easy to use in that the HintVR platform can be extended with the Vitalbeat® Digital Therapeutics2 platform into patient education, nurse care team training, paramedic training, technical training of medical staff. The staging area is immersive and powerful facilitating development and deployment of clinically validated digital therapeutic 3D applications for personalized immersive patient engagement, pre-op planning and post-op recovery and path to wellness for healthy living.
White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
4
Innovative VR enabled content for mental health and wellness, applications for managing and treating long term chronic pain, stress management and post-acute rehabilitation therapy. VR provides effective ways to treat PTSD and delivering therapeutic value with full immersion. With more clinical evidence from validated IRB approved studies for VR effectiveness in health professionals training proven to be 540% more effective, pre-op planning improving patient outcomes, patient engagement delivering value based care by bringing immersive engagement to patients to help better understand diagnosis and manage long term conditions post-acute care. The AI-Xperential Collaboration platform delivers content not just in VR but also with 5G connectivity more efficiently on smartphones, laptops and desktops making it truly ubiquitous. A connected future with edge computing, 5G bringing high definition holograms in real time for immersive, productive collaborative experience for experiential learning. Furthermore, remote collaboration in a Covid-19 world is accelerating VR application platforms and devices as a growing unmet need fueled by domain specific requirements for secure, rich data environments, advanced 3D devices and application delivery platforms
AI-Xperential VR solutions are being deployed in the digital health ecosystem from partners and collaborators such as Aventyn, Inc. in the digital health ecosystem. Today, these leading innovation driven companies are showing what is REAL in Virtual Reality using state of the art technology in digital therapeutics, patient engagement and immersive training and education for the next generation. Using the Intel® OpenVINO™ toolkit with AI-Xperential Collaboration for patient engagement Aventyn applications built on Intel® oneAPI programming model enable intelligent health powered by Intel technologies. These proven technologies deliver ideal combination of power and performance for holographic collaboration in immersive surgical training and patient engagement applications on enabled by Aventyn along with AI at the edge, vision and deep learning capabilities enabling innovative use cases for rapid deployment
White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
5
Now truly immersive technology components have seamless connectivity across devices, secure health cloud and remote care enabling data sharing and anywhere, anyplace response swiftly to life-threatening events Right now, digital technologies are already driving the most significant change in the health and life sciences industries since the establishment of institutional public health systems over two centuries ago. AI, vision and deep learning are being applied by these companies transforming the disciplines of drug discovery and clinical trials, diagnosis, treatment, recovery, and health maintenance of both young and old. Collectively, these advances can break down data silos, increase time to insight, and drive greater cross-industry collaboration—all with the ultimate result of improving people’s lives around the world towards healthy living.
Learn More: AI-Xperential
1
https://hbr.org/2019/10/research-how-virtual-reality-can-help-train-surgeons
2
https://marketplace.intel.com/s/offering/a5b3b000000TecnAAC/vitalbeat-remote-care-solution-for-chronic-diseasemanagement?language=en_US&wapkw=aventyn Copyright © 2021 Aventyn, Inc. All rights reserved. *All marks, images, data are properties of their respective owners. +Products and services in whole or in part are CE, FDA 510(k) approved
White Paper | AI-Xperential Collaboration – Holographic Training and Autonomous Surgery
6