2021 Biannual Report
Center for Women’s Health Data Science
Message from the Director, Data Science for Women’s Health Welcome to the inaugural biannual report from the Center for Women’s Health Data Science in the Duke Department of Obstetrics and Gynecology. Data are everywhere in healthcare and are produced faster than we can comprehend. Providers are attempting to harness the power of these data to make better clinical decisions and benefit our patients. The Center focuses on leveraging the power of data and analytics to improve the health of women in our community by developing tools that rely on data, statistical analysis, and computer-generated algorithms to inform important and sometimes confusing clinical decisions.
J. Eric Jelovsek, MD, MMEd, MSDS Director, Data Science for Women’s Health
Since prediction is at the heart of most clinical decisions, this report provides examples of how our team is committed to building, evaluating, and utilizing prediction models ranging in use from how many pain pills to prescribe a patient after surgery to predicting who is a risk for obstetric perineal lacerations after a vaginal birth. One of the Department’s leading faculty is urogynecologist Alison Weidner, MD, MMCI, Director of Informatics, who is working on development of a Patient Record Universe that captures the breadth of clinical encounters for women’s health conditions across the Duke University Health System. This Data Universe allows faster identification of women seeking care for specific conditions, specific surgical cases and stratification by comorbid conditions. The Patient Record Universe, along with the Stork Universe for obstetrics patients, serves as a foundation for using data quickly to make decisions.
2 Message From the Director
Appropriate data visualization aims to provide busy clinical leaders insight into quality of care using the least amount of space. Minimally Invasive Gynecologic Surgeon Craig Sobolewski, MD, leads the development of the Hysterectomy Dashboard, which aims to monitor quality and costs of performing hysterectomy across Duke Health. The tool serves to guide evidence-based pathways for performing this common surgical procedure in women. Maternal-Fetal Medicine Specialist Brenna Hughes, MD, MSc, leads the development of the Maternal Morbidity Dashboard, which allows obstetric providers a way to visualize, understand and develop interventions for reducing severe maternal morbidity and health disparities. Finally, Gynecologic Oncologist Laura Havrilesky, MD, MHSc, developed the Venous Thromboembolic Event (VTE) Dashboard to track blood clots during hospitalization and after discharge in women after undergoing gynecologic surgery. We are proud to highlight just a few of the many important ways that our Data Science team has developed useful practical solutions for improving care for women at Duke. We hope you enjoy our summary of accomplishments. There is no better time to be in Data Science in healthcare.
J. Eric Jelovsek, MD, MMEd, MSDS
Meet the Center for Women’s Health Data Science Team J. Eric Jelovsek, MD, MMEd, MSDS
Kaye Schlitz, BSN, RN-BC
Professor, Vice Chair for Education Duke Ob/Gyn
Business Analyst, Nursing Information System Specialist lll
Director of Data Science for Women’s Health
Team Lead: Research, Departmental Analytics and Resource Team (DART)Women’s Health, Population Health Sciences, Biostats & Informatics Analytics Center of Excellence (ACE), Duke Health Technology Solution
Alison Weidner, MD, MMCI
Jamie Tyler-Walker, MIS, CSSGB, ITIL V4-foundation
Professor, Duke Ob/Gyn
Departmental Analytics and Resource Team (DART)-Women’s Health
Director of Informatics
Analytics Center of Excellence (ACE), Duke Health Technology Solutions
Jennifer Gagnon, BS, RN
Asad Sherwani, BS, ITIL Foundation, SAP Certified
Associate Director of IT, Duke Transplant Center
Departmental Analytics and Resource Team (DART)-Women’s Health
Analytics Center of Excellence (ACE)-PORT, ACE-Departmental Analytics and Resource Team (DART), and ACE-Research Teams, Duke Health Technology Solutions
Analytics Center of Excellence (ACE), Duke Health Technology Solutions
Additional Steering Committee Members Matthew Barber, MD, MHS
Evan Myers, MD, MPH
Kristin Weaver, BS, CCRC
Alison C. Weidner, MD, MMCi
Kelli Kurtovic, MS, CSSBB
Joe English, MBA, MHA
3 Meet the Team
Medical Risk Prediction Models The Foundation of Medical Decision Making Prediction is the foundation of medical decision making. Clinicians use their knowledge, experience and scientific literature to predict what disorder a patient has and how likely they are to respond to treatment. • A prediction model is most useful for patient counseling. Patients want to know what their chances are for developing a condition. This allows for a prognostic prediction tailored for the individual patient. • This tool is also used for treatment decision making. It helps patients understand what their best treatment is likely to be, since this cannot be answered with certainty. A patient’s personal preferences can be weighed with knowledge from a prediction model. • Additionally, a prediction model can be used in selecting whether new diagnostic tests or new biomarkers of disease are useful when compared to routinely collected clinical information. A tool is only as good as the data it uses, and its ease of access and use by providers. It must also be easy to interpret by patients. Additional work is planned to incorporate these algorithms into the Electronic Medical Record (EMR) to allow providers and patients to benefit from their use. - Dr. Eric Jelovsek
4 Medical Risk Prediction Models
Completed* Prediction Models • Postpartum Hemorrhage • De Novo Stress Urinary Incontinence After Pelvic Organ Prolapse Surgery • Recurrence, Complications And Health Status In Women After Pelvic Organ Prolapse Surgery • Vaginal Birth After Cesarean Birth • Postoperative Opioid Use Following Surgery In Progress* Prediction Models • Urinary Tract Infection After Urogynecologic Surgery • Outcomes After Intradetrusor OnabotulinumtoxinA For Non-Neurogenic Urgency Incontinence In Women • Mixed Urinary Incontinence • Complications In Pregnant Women Undergoing Non-obstetric Surgery • Obstetric Anal Sphincter Injuries At The Time Of Admission For Labor • Preterm Delivery Within Seven Days
* Please refer to page 7 for citations
Prediction Model Spotlight: Gynecologic Oncology Prediction of Opioid Pills (GO-POP) calculator
with Dr. Eric Jelovsek
Why was this tool created? How was the need identified? What was the goal? The GO-POP calculator predicts the number of narcotic pills a woman is likely to use after gynecologic surgery. This provides a tool for providers to control pain and reduce overprescribing of narcotic medication after surgery.
Snapshot of the GO-POP Prediction Model
5 We Medical know our Riskcompany Prediction summary Models
with Dr. Eric Jelovsek
Prediction Model Spotlight: Obstetric Anal Sphincter Injuries (OASIS) Calculator
Why was this tool created? How was the need identified? What was the goal? Obstetric Anal Sphincter Injuries (OASIS), sometimes called 3rd or 4th degree perineal tears, have long-term consequences/aftereffects (sequelae) for patients after vaginal birth, however clinically useful tools to predict and counsel patients about these outcomes are lacking. Additionally, risks of a vaginal birth are often minimized during patient counseling. Duke’s OASIS model was designed for this purpose.
Snapshot of the OASIS Calculator Prediction Model
6 Medical Risk Prediction Models
Citations for Completed Prediction Models
Citations for In Progress Prediction Models
Postpartum Hemorrhage Machine Learning and Statistical Models to Predict Postpartum Hemorrhage. Venkatesh KK, Strauss RA, Grotegut CA, Heine RP, Chescheir NC, Stringer JSA, Stamilio DM, Menard KM, Jelovsek JE. Obstet Gynecol. 2020 Apr;135(4):935-944. doi: 10.1097/ AOG.0000000000003759. PMID: 32168227
Urinary Tract Infection After Urogynecologic Surgery External Validation and Updating of a Model to Predict Urinary Tract Infection After Urogynecologic Surgery. M. O’Shea, J. Dillon, W. Hendrickson, J. Jelovsek. Society of Gynecologic Surgeons 2022 Annual Meeting (SUBMITTED)
De Novo Stress Urinary Incontinence After Pelvic Organ Prolapse Surgery Validation of a Model Predicting De Novo Stress Urinary Incontinence in Women Undergoing Pelvic Organ Prolapse Surgery. Jelovsek JE, van der Ploeg JM, Roovers JP, Barber MD. Obstet Gynecol. 2019 Apr;133(4):683-690. doi: 10.1097/ AOG.0000000000003158. PMID: 30870279 Recurrence, Complications And Health Status In Women After Pelvic Organ Prolapse Surgery Models for Predicting Recurrence, Complications, and Health Status in Women After Pelvic Organ Prolapse Surgery. Jelovsek JE, Chagin K, Lukacz ES, Nolen TL, Shepherd JP, Barber MD, Sung V, Brubaker L, Norton PA, Rahn DD, Smith AL, Ballard A, Jeppson P, Meikle SF, Kattan MW; NICHD Pelvic Floor Disorders Network. Obstet Gynecol. 2018 Aug;132(2):298-309. doi: 10.1097/ AOG.0000000000002750. PMID: 29995735 Vaginal Birth After Cesarean Birth Are prediction models for vaginal birth after cesarean accurate? Harris BS, Heine RP, Park J, Faurot KR, Hopkins MK, Rivara AJ, Kemeny HR, Grotegut CA, Jelovsek JE. Am J Obstet Gynecol. 2019 May;220(5):492.e1-492.e7. doi: 10.1016/j. ajog.2019.01.232. Epub 2019 Feb 1. PMID: 30716285 Postoperative Opioid Use Following Gynecologic Surgery Development of a prediction model for postoperative opioid use following gynecologic surgery. Rodriguez, I. V., J. R. Salinaro, O. Kohrman, A. S. Habib, L. J. Havrilesky, J. E. Jelovsek, and B. A. Davidson. In Gynecologic Oncology, 159:274–75. Elsevier BV, 2020. https://doi. org/10.1016/j.ygyno.2020.05.479.
Outcomes After Intradetrusor OnabotulinumtoxinA For Non-Neurogenic Urgency Incontinence In Women Predicting Outcomes After Intradetrusor OnabotulinumtoxinA for Non-Neurogenic Urgency Incontinence in Women. Whitney K. Hendrickson MD1, Gongbo Xie MS2, David D. Rahn MD3, Megan Bradley MD4, Vivian W. Sung MD5, James A. Hokanson, PhD6, Ariana L. Smith MD7, Anthony G. Visco MD1, Cindy L. Amundsen MD1, Shen Luo PhD2, J. Eric Jelovsek MD MMEd MSDS1 Neurourology and Urodynamics (SUBMITTED) Mixed Urinary Incontinence Development and Validation of Models to Predict Persistent Stress and Urgency Urinary Incontinence and Need for Additional Treatment in Women Planning Midurethral Sling for Mixed Urinary Incontinence. Jelovsek, J., V. Sung, B. Carper, M. Gantz, H. Richter, L. Barden, E. Lukacz, H. Heidi, L. Burkett, and D. Mazloomdoost. In Neurourology and Urodynamics, 40:S110–11, 2021. (SUBMITTED) Complications In Pregnant Women Undergoing Nonobstetric Surgery Predictive performance of the American College of Surgeons risk calculator in pregnant patients undergoing non-obstetric surgery. A Adesomo, J DiMari, L Roby, M Costantine, M Landon, T Pawlik, C Lynch, J Jelovsek, K Venkatesh. Society for Maternal-Fetal Medicine 42ND Annual Meeting. Orlando, FL. February 2022. Obstetric Anal Sphincter Injuries At The Time Of Admission For Labor Predicting Risk of Obstetric Anal Sphincter Injuries at the Time of Admission for Labor Meekins AR, Zhoa C, Luchristt D, Grotegut C, Siddiqui NY, Ashanti B, Jelovsek JE. American Urogynecologic Society 2021. Preterm Delivery Within Seven Days Development and Validation of a Model to Predict Preterm Delivery Within Seven Days. Reiff, Emily S., Xuehan Ren, Dipali Pandya, Sheng Luo, Brenna Hughes, and Eric Jelovsek. In Reproductive Sciences, 27:306A-307A. SPRINGER HEIDELBERG, 2020.
7 We Medical know our Riskcompany Prediction summary Models
DATA UNIVERSES Data Universes Explained A Universe is a vast collection of clinically relevant data, logically organized to allow users to easily build operational or quality improvement reports.
Stork Universe The Stork Universe permits flexible reporting on pregnancies and deliveries across the Health System. This includes prenatal, delivery, newborn and postpartum care with linkages to providers, as well as group practices. • Source for linking mother’s and baby’s data • Source for maternal morbidity data, including race and ethnicity for identifying healthcare disparities Use Examples: • Monitoring of rates of vaginal vs. cesarean deliveries • Monitoring/reducing 3rd/4th degree obstetrical tears • Monitoring/reducing morbidity associated with severe blood loss • Helping to ensure adequate staffing levels by monitoring delivery patterns (day of week, time of day)
8 Stork Universe
Stork Universe By the Numbers (6/22/2013-5/19/2021)
Distinct mothers 53,874
Distinct newborns 43,136
Demographic Characteristics Patient ethnicity (pregnancies resulting in delivery)
• Hispanic: 6,797 (16%)
• Non-Hispanic: 34,786 (82%)
• Unknown/unavailable: 1,053 (2%)
Data elements 1,800
Births by location
• Duke Regional Hospital: 17,589 (42%)
• Duke University Hospital: 24,705 (58%)
Births by type
• Vaginal: 28,250 (67%)
• C-Sections: 13,817 (33%)
• Caucasian/White: 19,892
• Black or African American: 13,086
• Newborns: 5,494 (13%)
• American Indian / Alaskan: 192
• Pregnant mothers: 293 (1%)
• Asian: 2,625
• Native Hawaiian or Other Pacific Islander: 114
• Other: 3,221
Transfers to Intensive Care Unit
9 Stork Universe
Patient Record Universe The Patient Record Universe permits flexible reporting on encounter-based data. The basic unit is an encounter (inpatient, outpatient, emergency visit, etc.). Duke Women’s Health spearheaded its development, as well as the first project to leverage its foundation. The design is flexible enough to allow reporting by any clinical department in the Health System. • Way to identify Ob/Gyn visits based on either diagnoses or provider • Way to identify hysterectomy surgical encounters • Way to identify encounters involving an Intensive Care Unit (ICU) stay • Way to identify cases utilizing robotic technology • Way to identify specific comorbidities Use Examples • Hysterectomy cost comparison by surgical approach (e.g. robot vs. others) • Visit analysis by patient geographic location • Visit analysis by patient comorbid conditions • Visit analysis by clinic location
10 Patient Record Universe
Universe Champion: Dr. Alison Weidner
Patient Record Universe By the Numbers (6/22/2013-5/19/2021)
Distinct Women’s Health patients 432,643
Outpatient Visits 3,062,781
Hospital Admission Encounters 331,321
Demographic Characteristics • Hispanic: 30,214 (7%)
• Non-Hispanic: 377,271 (87%)
• Unknown/unavailable: 25,158 (6%)
Total Encounters 3,478,482
Data elements 1,800
Patient with the following comorbid conditions can be easily extracted from the Universe
Emergency Room Visits 376,878
• Breast cancer
• Chronic heart disease
• General malignancy
• Caucasian/White: 246,336
• Black or African American: 117,626
• Uterine leiomyoma
• American Indian / Alaskan: 1,814
• Asian: 19,154
• Native Hawaiian or Other Pacific Islander: 702
• Other: 47,011
11 Patient Record Universe
DATA VISUALZATIONS OPERATIONAL Hysterectomy Dashboard The Hysterectomy Dashboard provides demographic, outcome and cost analysis for a rolling two years of hysterectomies. • Way to trend patient characteristics over time • Way to compare outcomes by hysterectomy route • Way to track transfers to intensive care unit • Way to manage cost of care information • Way to monitor adherence to hysterectomy care pathway
Dashboard Champion: Dr. Craig Sobolewski Snapshot of the Hysterectomy Dashboard Executive Summary. The power of this project is both its broad scope of tracking demographics as well as its focus on the ability to determine value-based healthcare delivery in a Population Health model.
12 Hysterectomy Dashboard
with Dr. Craig Sobolewski Why was this dashboard created? How was the need identified? What was the goal? The Hysterectomy Dashboard was created to enable the Department to ensure that we are continuously striving for the highest value procedures. Value in medicine is defined as quality over cost. As such, the ability to monitor the direct and indirect costs associated with hysterectomies performed at Duke hospitals is critical. Coupling this with a variety of quality measures makes for a robust data set to assist in the endeavor. How does your team use that data? Data from the dashboard is shared monthly with Division Chiefs, who also can access the dashboard themselves at any time. Identifying outliers will provide opportunities for training and sharing best practices. In addition, a clinical pathway decision tool for choosing the route of hysterectomy for benign indications has been developed and integrated with the electronic medical record. Compliance with the utilization of this decision tool and the impact that it may have on hysterectomy value will be monitored over time. It is expected that Division Chiefs will share this data with their faculty and monitor trends over time. What do you hope the data will assist with for improving patient care in the future? The hope is that the combination of an evidence-based clinical pathway that assists clinicians in choosing the route of hysterectomy combined with a more available and transparent understanding of their individual costs and quality outcomes will drive improvements in the overall value of hysterectomy provided to our patients at Duke.
13 We know Hysterectomy our company Dashboard summary
Maternal Morbidity Dashboard The Maternal Morbidity Dashboard provides overall and annual analysis of morbidity events, including days since last events, counts by morbidity indicator, analysis of patients requiring four or more units of blood products and types of products transfused. The dashboard’s executive overview also allows analysis of data by patient delivery type, race, ethnicity and payor. • Provides the ability to quantify the number of morbidities and mortalities by race, ethnicity and payor • Provides insight into potential biases in care
Dashboard Champion: Dr. Brenna Hughes
We hope to create interventions that will allow us to reduce or eliminate certain types of severe maternal morbidity as well as decrease or eliminate any potential disparities in outcomes.
The Maternal Morbidity Dashboard presents data such as Deliveries by Ethnicity, Race, Insurance or type, as well as for Morbidity Indicators.
14 Maternal Morbidity Dashboard
with Dr. Brenna Hughes Why was this dashboard created? How was the need identified? What was the goal? Severe maternal morbidity (any physical or mental illness or disability directly related to pregnancy and/or childbirth) is known to predict the likelihood of maternal mortality. The U.S. has the highest risk of maternal mortality among high income countries. We wanted to create a dashboard that could demonstrate how frequently we see severe maternal morbidity at Duke with the hope that we will be able to develop interventions to decrease the occurrence of certain types of severe maternal morbidity. How does your team use that data? We looked at all of the CDC-defined types of severe maternal morbidity across several years. We are able to visualize each type of morbidity as well as evaluate their occurrence by race, ethnicity and health insurance status. The dashboard was recently created, and we are using it to evaluate whether rates of morbidity differ by race or ethnicity or insurance status. What do you hope the data will assist with for improving patient care in the future? We hope to create interventions that will allow us to reduce or eliminate certain types of severe maternal morbidity, as well as decrease or eliminate any potential disparities in outcomes.
15 WeMaternal know our Morbidity company Dashboard summary
Venous Thromboembolic Event (VTE) Dashboard The VTE dashboard displays the number and percentage of gynecologic oncology surgeries resulting in a post-op VTE within 30 days and compliance with various prophylactic process measures. Some of these measures include order/administration of pre-op Heparin, post-op in-house anticoagulation therapy and prescription of at-home anticoagulants. • Provides the ability to track blood clots within 30 days following surgery over time • Provides the ability to track adherence to gynecologic surgery care pathways • Enables identification of cases that require deeper chart review
Dashboard Champion: Dr. Laura Havrilesky
Every blood clot diagnosed after surgery is one too many. Our VTE dashboard is more accurate than the data being tracked hospitalwide, because unlike the hospital, our Department is tracking events that happen even after hospital discharge.
16 VTE Dashboard
Above images represent data sets from the VTE Dashboard.
with Dr. Laura Havrilesky Why was this dashboard created? How was the need identified? What was the goal? The Venous Thromboembolic Event (VTE) dashboard was created after we began a Quality Improvement (QI) project to reduce the number of blood clots diagnosed after gynecologic surgery. A VTE is a blood clot that develops in the leg and can move to the lung and be life-threatening. When we began the project, we realized that we had no way to track the total number of women who were diagnosed with postoperative blood clots or the number who received preventive medicines such as Heparin before and after surgery. The VTE dashboard provides an easy way to track VTEs in real time. We designed this dashboard to keep track of VTE events, as well as the appropriate use of medicines that can be given before, during and after gynecologic surgery to prevent VTE. How does your team use that data? We frequently look at the VTE dashboard as part of our QI meetings to visualize trends in the use of preventive medicines, as well as the VTE event rate. The dashboard was also used to track and report on trends over time in a recent publication, “A quality improvement initiative to reduce venous thromboembolism on a gynecologic oncology service,” (Rafael Gonzalez et al, Gynecologic Oncology 2021), describing our VTE prevention project. What do you hope the data will assist with for improving patient care in the future? Every blood clot diagnosed after surgery is one too many. Our VTE dashboard is more accurate than the data being tracked hospitalwide, because unlike the hospital, our Department is tracking events that happen even after hospital discharge. This allows us to look closely at every blood clot that happens in the month after surgery to troubleshoot and brainstorm ways to prevent the next one.
17 We know our company VTE Dashboard summary
DART Contributions to the Health System & Return on Investment (ROI) Project Summaries Reducing Blood Loss The Departmental Analytics and Resource (DART) team recognized that blood loss data in the medical record was, in some cases, not accurate. The team developed a process to allow data from the medical record to be compared to data from the blood bank. By comparing the two data sources, the measure of blood loss is more accurate and allows quality improvement teams to know if their interventions are working to reduce blood loss. This improvement not only benefits Duke Ob/Gyn but also all other departments tracking blood loss.
18 DART Project Summaries
Cost of Robotic Surgery The institution needed a way to appropriately capture the number of surgeries utilizing robotic equipment. No standard indicator was available from the medical record software vendor, as processes vary greatly between hospitals. A process was developed to identify cases using robotic technology, which allows the Health System to more effectively plan for future robotic equipment needs.
19 We knowDART our company Project Summaries summary
Center for Women’s Health Data Science
Produced by the Duke Ob/Gyn Department of Communications and Marketing Some photos taken prior to COVID-19 pandemic
Designed by Blue House Creative