Volume 25 Issue 2

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health Indexed in MEDLINE

Behavioral Health

144

Characteristics for Low, High and Very High Emergency Department Use for Mental Health Diagnoses from Health Records and Structured Interviews

Marie-Josée Fleury, Zhirong Cao, Guy Grenier

Cardiology

155 Bridging the Gap: Evaluation of an Electrocardiogram Curriculum for Advanced Practice Clinicians

Steven Lindsey, Tim P. Moran, Meredith A. Stauch, Alexis L. Lynch, Kristen Grabow Moore

160 Stage B Heart Failure Is Ubiquitous in Emergency Patients with Asymptomatic Hypertension

Kimberly Souffront, Bret P. Nelson, Megan Lukas, Hans Reyes Garay, Lauren Gordon, Thalia Matos, Isabella Hanesworth, Rebecca Mantel, Claire Shubeck, Cassidy Bernstein, George T. Loo, Lynne D. Richardson

166

Performance of Intra-arrest Echocardiography: A Systematic Review

Yi-Ju Ho, Chih-Wei Sung, Yi-Chu Chen, Wan-Ching Lien, Wei-Tien Chang, Chien-Hua Huang

Education

175 Staffing Patterns of Non-ACGME Fellowships with 4-Year Residency Programs: A National Survey

David A. Haidar, Laura R. Hopson, Ryan V. Tucker, Rob D. Huang, Jessica Koehler, Nik Theyyunni, Nicole Klekowski, Christopher M. Fung

181 Changes in Residency Applicant Cancellation Patterns with Virtual Interviews: A Single-site Analysis

Meryll Bouldin, Carly Eastin, Rachael Freeze-Ramsey, Amanda Young, Meredith von Dohlen, Lauren Evans, Travis Eastin, Sarah Greenberger

Western Journal of Emergency Medicine:

Andrew W. Phillips, MD, Associate Editor DHR Health-Edinburg, Texas

Edward Michelson, MD, Associate Editor Texas Tech University- El Paso, Texas

Dan Mayer, MD, Associate Editor

Retired from Albany Medical College- Niskayuna, New York

Wendy Macias-Konstantopoulos, MD, MPH, Associate Editor Massachusetts General Hospital- Boston, Massachusetts

Gayle Galletta, MD, Associate Editor

University of Massachusetts Medical SchoolWorcester, Massachusetts

Yanina Purim-Shem-Tov, MD, MS, Associate Editor Rush University Medical Center-Chicago, Illinois

Resident Editors

AAEM/RSA

John J. Campo, MD

Harbor-University of California, Los Angeles Medical Center

ACOEP

Justina Truong, DO Kingman Regional Medical Center

Section Editors

Behavioral Emergencies

Leslie Zun, MD, MBA Chicago Medical School

Marc L. Martel, MD Hennepin County Medical Center

Cardiac Care

Fred A. Severyn, MD University of Colorado School of Medicine

Sam S. Torbati, MD

Cedars-Sinai Medical Center

Clinical Practice

Cortlyn W. Brown, MD Carolinas Medical Center

Casey Clements, MD, PhD Mayo Clinic

Patrick Meloy, MD Emory University

Nicholas Pettit, DO, PhD Indiana University

David Thompson, MD University of California, San Francisco

Kenneth S. Whitlow, DO Kaweah Delta Medical Center

Critical Care

Christopher “Kit” Tainter, MD University of California, San Diego

Gabriel Wardi, MD

University of California, San Diego

Joseph Shiber, MD University of Florida-College of Medicine

Matt Prekker MD, MPH Hennepin County Medical Center

David Page, MD University of Alabama

Erik Melnychuk, MD Geisinger Health

Quincy Tran, MD, PhD University of Maryland

Disaster Medicine

John Broach, MD, MPH, MBA, FACEP University of Massachusetts Medical School UMass Memorial Medical Center

Mark I. Langdorf, MD, MHPE, Editor-in-Chief University of California, Irvine School of MedicineIrvine, California

Shahram Lotfipour, MD, MPH, Managing Editor

University of California, Irvine School of MedicineIrvine, California

Michael Gottlieb, MD, Associate Editor Rush Medical Center-Chicago, Illinois

Niels K. Rathlev, MD, Associate Editor Tufts University School of Medicine-Boston, Massachusetts

Rick A. McPheeters, DO, Associate Editor Kern Medical- Bakersfield, California

Gentry Wilkerson, MD, Associate Editor University of Maryland

Christopher Kang, MD Madigan Army Medical Center

Education

Danya Khoujah, MBBS

University of Maryland School of Medicine

Jeffrey Druck, MD University of Colorado

John Burkhardt, MD, MA

University of Michigan Medical School

Michael Epter, DO Maricopa Medical Center

ED Administration, Quality, Safety

Tehreem Rehman, MD, MPH, MBA Mount Sinai Hospital

David C. Lee, MD Northshore University Hospital

Gary Johnson, MD Upstate Medical University

Brian J. Yun, MD, MBA, MPH Harvard Medical School

Laura Walker, MD Mayo Clinic

León D. Sánchez, MD, MPH

Beth Israel Deaconess Medical Center

William Fernandez, MD, MPH University of Texas Health-San Antonio

Robert Derlet, MD

Founding Editor, California Journal of Emergency Medicine

University of California, Davis

Emergency Medical Services

Daniel Joseph, MD Yale University

Joshua B. Gaither, MD

University of Arizona, Tuscon

Julian Mapp

University of Texas, San Antonio

Shira A. Schlesinger, MD, MPH Harbor-UCLA Medical Center

Geriatrics

Cameron Gettel, MD Yale School of Medicine

Stephen Meldon, MD Cleveland Clinic

Luna Ragsdale, MD, MPH Duke University

Health Equity

Emily C. Manchanda, MD, MPH Boston University School of Medicine

Faith Quenzer

Temecula Valley Hospital San Ysidro Health Center

Shadi Lahham, MD, MS, Deputy Editor

Kaiser Permanente- Irvine, California

Susan R. Wilcox, MD, Associate Editor

Massachusetts General Hospital- Boston, Massachusetts

Elizabeth Burner, MD, MPH, Associate Editor

University of Southern California- Los Angeles, California

Patrick Joseph Maher, MD, MS, Associate Editor Ichan School of Medicine at Mount Sinai- New York, New York

Donna Mendez, MD, EdD, Associate Editor

University of Texas-Houston/McGovern Medical School- Houston Texas

Danya Khoujah, MBBS, Associate Editor

University of Maryland School of Medicine- Baltimore, Maryland

Mandy J. Hill, DrPH, MPH

UT Health McGovern Medical School

Payal Modi, MD MScPH

University of Massachusetts Medical

Infectious Disease

Elissa Schechter-Perkins, MD, MPH Boston University School of Medicine

Ioannis Koutroulis, MD, MBA, PhD

George Washington University School of Medicine and Health Sciences

Kevin Lunney, MD, MHS, PhD University of Maryland School of Medicine

Stephen Liang, MD, MPHS Washington University School of Medicine

Victor Cisneros, MD, MPH Eisenhower Medical Center

Injury Prevention

Mark Faul, PhD, MA

Centers for Disease Control and Prevention

Wirachin Hoonpongsimanont, MD, MSBATS

Eisenhower Medical Center

International Medicine

Heather A.. Brown, MD, MPH Prisma Health Richland

Taylor Burkholder, MD, MPH

Keck School of Medicine of USC

Christopher Greene, MD, MPH University of Alabama

Chris Mills, MD, MPH

Santa Clara Valley Medical Center

Shada Rouhani, MD Brigham and Women’s Hospital

Legal Medicine

Melanie S. Heniff, MD, JD Indiana University School of Medicine

Greg P. Moore, MD, JD

Madigan Army Medical Center

Statistics and Methodology

Shu B. Chan MD, MS Resurrection Medical Center

Stormy M. Morales Monks, PhD, MPH Texas Tech Health Science University

Soheil Saadat, MD, MPH, PhD University of California, Irvine

James A. Meltzer, MD, MS

Albert Einstein College of Medicine

Musculoskeletal

Juan F. Acosta DO, MS Pacific Northwest University

Rick Lucarelli, MD Medical City Dallas Hospital

William D. Whetstone, MD

University of California, San Francisco

Neurosciences

Antonio Siniscalchi, MD

Annunziata Hospital, Cosenza, Italy

Pediatric Emergency Medicine

Paul Walsh, MD, MSc

University of California, Davis

Muhammad Waseem, MD Lincoln Medical & Mental Health Center

Cristina M. Zeretzke-Bien, MD University of Florida

Public Health

Jacob Manteuffel, MD

Henry Ford Hospital

John Ashurst, DO Lehigh Valley Health Network

Tony Zitek, MD

Kendall Regional Medical Center

Trevor Mills, MD, MPH

Northern California VA Health Care

Erik S. Anderson, MD

Alameda Health System-Highland Hospital

Technology in Emergency Medicine

Nikhil Goyal, MD

Henry Ford Hospital

Phillips Perera, MD

Stanford University Medical Center

Trauma

Pierre Borczuk, MD

Massachusetts General Hospital/Havard Medical School

Toxicology

Brandon Wills, DO, MS

Virginia Commonwealth University

Jeffrey R. Suchard, MD

University of California, Irvine

Ultrasound

J. Matthew Fields, MD Thomas Jefferson University

Shane Summers, MD Brooke Army Medical Center

Robert R. Ehrman

Wayne State University

Ryan C. Gibbons, MD Temple Health

Official Journal of the California Chapter of the American College of Emergency Physicians, the America College of Osteopathic Emergency Physicians, and the California Chapter of the American Academy of Emergency Medicine

Western Journal of Emergency Medicine:

Amin A. Kazzi, MD, MAAEM

The American University of Beirut, Beirut, Lebanon

Brent King, MD, MMM University of Texas, Houston

Christopher E. San Miguel, MD Ohio State University Wexner Medical Center

Daniel J. Dire, MD University of Texas Health Sciences Center San Antonio

Douglas Ander, MD Emory University

Edward Michelson, MD Texas Tech University

Edward Panacek, MD, MPH University of South Alabama

Francesco Della Corte, MD Azienda Ospedaliera Universitaria “Maggiore della Carità,” Novara, Italy

Gayle Galleta, MD

Editorial Board

Sørlandet Sykehus HF, Akershus Universitetssykehus, Lorenskog, Norway

Hjalti Björnsson, MD

Icelandic Society of Emergency Medicine

Jaqueline Le, MD Desert Regional Medical Center

Jeffrey Love, MD

The George Washington University School of Medicine and Health Sciences

Katsuhiro Kanemaru, MD University of Miyazaki Hospital, Miyazaki, Japan

Kenneth V. Iserson, MD, MBA University of Arizona, Tucson

Leslie Zun, MD, MBA Chicago Medical School

Linda S. Murphy, MLIS

University of California, Irvine School of Medicine Librarian

Niels K. Rathlev, MD

Tufts University School of Medicine

Pablo Aguilera Fuenzalida, MD

Pontificia Universidad Catolica de Chile, Región Metropolitana, Chile

Peter A. Bell, DO, MBA Baptist Health Sciences University

Peter Sokolove, MD University of California, San Francisco

Rachel A. Lindor, MD, JD Mayo Clinic

Robert Suter, DO, MHA UT Southwestern Medical Center

Robert W. Derlet, MD University of California, Davis

Rosidah Ibrahim, MD Hospital Serdang, Selangor, Malaysia

Scott Rudkin, MD, MBA University of California, Irvine

Scott Zeller, MD

University of California, Riverside

Steven H. Lim, MD

Changi General Hospital, Simei, Singapore

Terry Mulligan, DO, MPH, FIFEM ACEP Ambassador to the Netherlands Society of Emergency Physicians

Wirachin Hoonpongsimanont, MD, MSBATS

Siriraj Hospital, Mahidol University, Bangkok, Thailand

Elena Lopez-Gusman, JD

California ACEP

American College of Emergency Physicians

Jennifer Kanapicki Comer, MD FAAEM

California Chapter Division of AAEM Stanford University School of Medicine

DeAnna McNett, CAE

American College of Osteopathic Emergency Physicians

Kimberly Ang, MBA

UC Irvine Health School of Medicine

Randall J. Young, MD, MMM, FACEP

California ACEP

American College of Emergency Physicians

Kaiser Permanente

Mark I. Langdorf, MD, MHPE, MAAEM, FACEP

UC Irvine Health School of Medicine

Robert Suter, DO, MHA

American College of Osteopathic Emergency Physicians

UT Southwestern Medical Center

Shahram Lotfipour, MD, MPH FAAEM, FACEP

UC Irvine Health School of Medicine

Jorge Fernandez, MD, FACEP

UC San Diego Health School of Medicine

Isabelle Nepomuceno, BS Executive Editorial Director

Visha Bajaria, BS WestJEM Editorial Director

Emily Kane, MA WestJEM Editorial Director

Stephanie Burmeister, MLIS WestJEM Staff Liaison

Cassandra Saucedo, MS Executive Publishing Director

Nicole Valenzi, BA WestJEM Publishing Director

June Casey, BA Copy Editor

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

JOURNAL FOCUS

Emergency medicine is a specialty which closely reflects societal challenges and consequences of public policy decisions. The emergency department specifically deals with social injustice, health and economic disparities, violence, substance abuse, and disaster preparedness and response. This journal focuses on how emergency care affects the health of the community and population, and conversely, how these societal challenges affect the composition of the patient population who seek care in the emergency department. The development of better systems to provide emergency care, including technology solutions, is critical to enhancing population health.

Table of Contents

186

191

197

Virtual Interviews and the Pediatric Emergency Medicine Match Geography: A National Survey Aline Baghdassarian, Jessica A. Bailey, Derya Caglar, Michelle Eckerle, Andrea Fang, Katherine McVety, Thuy Ngo, Jerri A. Rose, Cindy Ganis Roskind, Melissa M. Tavarez, Frances Turcotte Benedict, Joshua Nagler, Melissa L. Langhan

Analysis of Anonymous Student Narratives About Experiences with Emergency Medicine Residency Programs

Molly Estes, Jacob Garcia, Ronnie Ren, Mark Olaf, Shannon Moffett, Michael Galuska, Xiao Chi Zhang

Rapid Cycle Deliberate Practice Training for Simulated Cardiopulmonary Resuscitation in Resident Education

Jaron D. Raper, Charles A. Khoury, Anderson Marshall, Robert Smola, Zachary Pacheco, Jason Morris, Guihua Zhai, Stephanie Berger, Ryan Kraemer, Andrew D. Bloom

205

209

213

221

Simulation Improves Emergency Medicine Residents’ Clinical Performance of Aorta Point-of-Care Ultrasound

Brandon M. Wubben, Cory Wittrock

Foundations of Emergency Medicine: Impact of a Standardized, Open-access, Core Content Curriculum on In-Training Exam Scores

Jaime Jordan, Natasha Wheaton, Nicholas D. Hartman, Dana Loke, Nathaniel Shekem, Anwar Osborne, P. Logan Weygandt, Kristen Grabow Moore

Integrating Hospice and Palliative Medicine Education Within the American Board of Emergency Medicine Model

Rebecca Goett, Jason Lyou, Lauren R. Willoughby, Daniel W. Markwalter, Diane L. Gorgas, Lauren T. Southerland

The Effect of a Simulation-based Intervention on Emergency Medicine Resident Management of Early Pregnancy Loss

Shawna D. Bellew, Erica Lowing, Leah Holcomb

Emergency Department Operations

226 Root Cause Analysis of Delayed Emergency Department Computed Tomography Scans Arjun Dhanik, Bryan A. Stenson, Robin B. Levenson, Peter S. Antkowiak, Leon D. Sanchez, David T. Chiu

Geriatrics

230 Usability of the 4Ms Worksheet in the Emergency Department for Older Patients: A Qualitative Study Mackenzie A. McKnight, Melissa K. Sheber, Daniel J. Liebzeit, Aaron T. Seaman, Erica K. Husser, Harleah G. Buck, Heather S. Reisinger, Sangil Lee

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Table of Contents continued

Pediatrics

237 National Characteristics of Emergency Care for Children with Neurologic Complex Chronic Conditions

Kaileen Jafari, Kristen Carlin, Derya Caglar, Eileen J. Klein, Tamara D. Simon

246 Pediatric Outcomes of Emergency Medical Services Non-Transport Before and During the COVID-19 Pandemic

Lori Pandya, Brandon Morshedi, Brian Miller, Halim Hennes, Mohamed Badawy

Research Methodology

254 Development and Validation of a Scoring Rubric for Editorial Evaluation of Peer-review Quality: A Pilot Study

Jeffrey N. Love, Anne M. Messman, Jonathan S. Ilgen, Chris Merritt, Wendy C. Coates, Douglas S. Ander, David P. Way

Ultrasound

264 Novel Scoring Scale for Quality Assessment of Lung Ultrasound in the Emergency Department

Jessica R. Balderston, Taylor Brittan, Bruce J. Kimura, Chen Wang, Jordan Tozer

268 Diagnostic Accuracy of a Handheld Ultrasound vs a Cart-based Model: A Randomized Clinical Trial

Ryan C. Gibbons, Daniel J. Jaeger, Matthew Berger, Mark Magee, Claire Shaffer, Thomas G. Costantino

275 Space Ultrasound: A Proposal for Competency-based Ultrasound Training for In-flight Space Medicine Chanel Fischetti, Emily Frisch, Michael Loesche, Andrew Goldsmith, Ben Mormann, Joseph S. Savage, Roger Dias, Nicole Duggan

282 Ultrasound Performed by Emergency Physicians for Deep Vein Thrombosis: A Systematic Review

Daniel Hercz, Oren J. Mechanic, Marcia Varella, Francisco Fajardo, Robert L. Levine

Women’s Health

291 User Experience of Access to Sexual Assault Nurse Examiner and Emergency Contraception in Emergency Departments in the United States: A National Survey

Colleen Cowdery, Diana Halloran, Rebecca Henderson, MA Kathleen M. Allen, Kelly O’Shea, Kristen Woodward, Susan Rifai, Scott A. Cohen, Muhammad Abdul Baker Chowdhury, Cristina Zeretzke-Bien, Lauren A. Walter, Marie-Carmelle Elie-Turenne

Letters to the Editor

301 Factors Associated with Overutilization of Computed Tomography Cervical Spine Imaging

Tessy La Torre Torres, Jonathan McGhee

302 Reply to “Factors Associated with Overutilization of Computed Tomography Cervical Spine Imaging” Karl Chamberlin

Western Journal of Emergency Medicine:

Integrating Emergency Care with Population Health

This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.

Professional Society Sponsors

American College of Osteopathic Emergency Physicians

California American College of Emergency Physicians

Academic Department of Emergency Medicine Subscriber

Albany Medical College Albany, NY

Allegheny Health Network Pittsburgh, PA

American University of Beirut Beirut, Lebanon

AMITA Health Resurrection Medical Center Chicago, IL

Arrowhead Regional Medical Center Colton, CA

Baylor College of Medicine Houston, TX

Baystate Medical Center Springfield, MA

Bellevue Hospital Center New York, NY

Beth Israel Deaconess Medical Center Boston, MA

Boston Medical Center Boston, MA

Brigham and Women’s Hospital Boston, MA

Brown University Providence, RI

Carl R. Darnall Army Medical Center Fort Hood, TX

Cleveland Clinic Cleveland, OH

Columbia University Vagelos New York, NY

State Chapter Subscriber

Arizona

Florida

International

Lebanese

California Chapter Division of American Academy of Emergency Medicine

Conemaugh Memorial Medical Center Johnstown, PA

Crozer-Chester Medical Center Upland, PA

Desert Regional Medical Center Palm Springs, CA

Detroit Medical Center/ Wayne State University Detroit, MI

Eastern Virginia Medical School Norfolk, VA

Einstein Healthcare Network Philadelphia, PA

Eisenhower Medical Center Rancho Mirage, CA

Emory University Atlanta, GA

Franciscan Health Carmel, IN

Geisinger Medical Center Danville, PA

Grand State Medical Center Allendale, MI

Healthpartners Institute/ Regions Hospital Minneapolis, MN

Hennepin County Medical Center Minneapolis, MN

Henry Ford Medical Center Detroit, MI

Henry Ford Wyandotte Hospital Wyandotte, MI

INTEGRIS Health Oklahoma City, OK

Kaiser Permenante Medical Center San Diego, CA

Kaweah Delta Health Care District Visalia, CA

Kennedy University Hospitals Turnersville, NJ

Kent Hospital Warwick, RI

Kern Medical Bakersfield, CA

Lakeland HealthCare St. Joseph, MI

Lehigh Valley Hospital and Health Network Allentown, PA

Loma Linda University Medical Center Loma Linda, CA

Louisiana State University Health Sciences Center New Orleans, LA

Louisiana State University Shreveport Shereveport, LA

Madigan Army Medical Center Tacoma, WA

Maimonides Medical Center Brooklyn, NY

Maine Medical Center Portland, ME

Massachusetts General Hospital/Brigham and Women’s Hospital/ Harvard Medical Boston, MA

Mayo Clinic Jacksonville, FL

Mayo Clinic College of Medicine Rochester, MN

Mercy Health - Hackley Campus Muskegon, MI

Merit Health Wesley Hattiesburg, MS

Midwestern University Glendale, AZ

Mount Sinai School of Medicine New York, NY

New York University Langone Health New York, NY

North Shore University Hospital Manhasset, NY

Northwestern Medical Group Chicago, IL

NYC Health and Hospitals/ Jacobi New York, NY

Ohio State University Medical Center Columbus, OH

Ohio Valley Medical Center Wheeling, WV

Oregon Health and Science University Portland, OR

Penn State Milton S. Hershey Medical Center Hershey, PA

Mediterranean Academy of Emergency Medicine

To become a WestJEM departmental sponsor, waive article processing fee, receive electronic copies for all faculty and residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:

Stephanie Burmeister

WestJEM Staff Liaison

Phone: 1-800-884-2236

Email: sales@westjem.org

Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias

Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine

Uniformed Services Chapter Division of the American Academy of Emergency Medicine

Virginia Chapter Division of the American Academy of Emergency Medicine

Western Journal of Emergency Medicine:

This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.

Professional Society Sponsors

American College of Osteopathic Emergency Physicians

California American College of Emergency Physicians

Academic Department of Emergency Medicine Subscriber

Prisma Health/ University of South Carolina SOM Greenville Greenville, SC

Regions Hospital Emergency Medicine Residency Program St. Paul, MN

Rhode Island Hospital Providence, RI

Robert Wood Johnson University Hospital New Brunswick, NJ

Rush University Medical Center Chicago, IL

St. Luke’s University Health Network Bethlehem, PA

Spectrum Health Lakeland St. Joseph, MI

Stanford Stanford, CA

SUNY Upstate Medical University Syracuse, NY

Temple University Philadelphia, PA

Texas Tech University Health Sciences Center El Paso, TX

The MetroHealth System/ Case Western Reserve University Cleveland, OH

UMass Chan Medical School Worcester, MA

University at Buffalo Program Buffalo, NY

State Chapter Subscriber

University of Alabama Medical Center Northport, AL

University of Alabama, Birmingham Birmingham, AL

University of Arizona College of Medicine-Tucson Tucson, AZ

University of California, Davis Medical Center Sacramento, CA

University of California, Irvine Orange, CA

University of California, Los Angeles Los Angeles, CA

University of California, San Diego La Jolla, CA

University of California, San Francisco San Francisco, CA

UCSF Fresno Center Fresno, CA

University of Chicago Chicago, IL

University of Cincinnati Medical Center/ College of Medicine Cincinnati, OH

University of Colorado Denver Denver, CO

University of Florida Gainesville, FL

University of Florida, Jacksonville Jacksonville, FL

California Chapter Division of American Academy of Emergency Medicine

University of Illinois at Chicago Chicago, IL

University of Iowa Iowa City, IA

University of Louisville Louisville, KY

University of Maryland Baltimore, MD

University of Massachusetts Amherst, MA

University of Michigan Ann Arbor, MI

University of Missouri, Columbia Columbia, MO

University of North Dakota School of Medicine and Health Sciences Grand Forks, ND

University of Nebraska Medical Center Omaha, NE

University of Nevada, Las Vegas Las Vegas, NV

University of Southern Alabama Mobile, AL

University of Southern California Los Angeles, CA

University of Tennessee, Memphis Memphis, TN

University of Texas, Houston Houston, TX

University of Washington Seattle, WA

University of WashingtonHarborview Medical Center Seattle, WA

University of Wisconsin Hospitals and Clinics Madison, WI

UT Southwestern Dallas, TX

Valleywise Health Medical Center Phoenix, AZ

Virginia Commonwealth University Medical Center Richmond, VA

Wake Forest University Winston-Salem, NC

Wake Technical Community College Raleigh, NC

Wayne State Detroit, MI

Wright State University Dayton, OH

Yale School of Medicine New Haven, CT

International

Lebanese

Mediterranean

To become a WestJEM departmental sponsor, waive article processing fee, receive electronic copies for all faculty and residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact:

Stephanie Burmeister

WestJEM Staff Liaison

Phone: 1-800-884-2236

Email: sales@westjem.org

Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine

ORIGINAL RESEARCH

CharacteristicsforLow,HighandVeryHighEmergency DepartmentUseforMentalHealthDiagnosesfromHealth RecordsandStructuredInterviews

Marie-JoséeFleury,PhD*†

ZhirongCao,MSc†

GuyGrenier,PhD†

*McGillUniversity,DepartmentofPsychiatry,Montreal,Canada

† DouglasMentalHealthUniversityResearchCentre,Montreal,Canada

SectionEditors:BradBobrin,MD,andYaninaPurim-Shem-Tov,MD,MS

Submissionhistory:SubmittedMay24,2023;RevisionreceivedNovember17,2023;AcceptedNovember22,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18327

Introduction: Patientswithmentalhealthdiagnoses(MHD)areamongthemostfrequentemergency department(ED)users,suggestingtheimportanceofidentifyingadditionalfactorsassociatedwiththeir EDusefrequency.Inthisstudyweassessedvariouspatientsociodemographicandclinical characteristics,andserviceuseassociatedwithlowEDusers(1–3visits/year),comparedtohigh(4–7) andveryhigh(8+)EDuserswithMHD.

Methods: OurstudywasconductedinfourlargeQuebec(Canada)EDnetworks.Atotalof299patients withMHDwererandomlyrecruitedfromtheseEDin2021–2022.Structuredinterviewscomplemented datafromnetworkhealthrecords,providingextensivedataonparticipantprofilesandtheirqualityof care.WeusedmultivariablemultinomiallogisticregressiontocomparelowEDusetohighandveryhigh EDuse.

Results: Overa12-monthperiod,39%ofpatientswerelowEDusers,37%high,and24%veryhighED users.ComparedwithlowEDusers,thoseatgreaterprobabilityforhighorveryhighEDuseexhibited moreviolent/disturbedbehaviorsorsocialproblems,chronicphysicalillnesses,andbarrierstounmet needs.Patientspreviouslyhospitalized1–2timeshadlowerriskofhighorveryhighEDusethanthose notpreviouslyhospitalized.ComparedwithlowEDusers,highandveryhighEDusersshowedhigher prevalenceofpersonalitydisordersandsuicidalbehaviors,respectively.Womenhadgreaterprobability ofhighEDusethanmen.PatientslivinginrentalhousinghadgreaterprobabilityofbeingveryhighED usersthanthoselivinginprivatehousing.Usingatleast5+ primarycareservicesandbeingrecurrentED userstwoyearspriortothelastyearofEDusehadincreasedprobabilityofveryhighEDuse.

Conclusion: FrequencyofEDusewasassociatedwithcomplexissuesandhigherperceived barrierstounmetneedsamongpatients.VeryhighEDusershadmoresevererecurrentconditions,such asisolationandsuicidalbehaviors,despiteusingmoreprimarycareservices.Results suggestedsubstantialreductionofbarrierstocareandimprovementonbothaccessandcontinuity ofcareforthesevulnerablepatients,integratingcrisisresolutionandsupportedhousing services.Limitedhospitalizationsmaysometimesbeindicated,protectingagainstED use.[WestJEmergMed.2024;25(2)144–154.]

Keywords: emergencydepartment;frequencyofemergencydepartmentvisits;lowserviceusers; highserviceusers;veryhighserviceusers;mentalhealthdiagnoses;probabilityfactors; associatedvariables.

INTRODUCTION

Emergencydepartment(ED)crowdingisamajor impedimenttotheefficacyofhealthcaresystems,1 causedin partbyaminorityofpatientswhousetheEDfrequently.2

Accordingtoa2019systematicreview,theestimated prevalenceofhighEDuserswas4-16%,yetthesepatients accountedfor14–47%ofallEDvisits,averaging6.9ED visitsperyear.3 HighEDusers,commonlydefinedashaving 4+ EDvisitsina12-monthperiod,4,5 aremorelikelythan otherpatientstobehospitalizedfrequently6 andhave2.2 timesgreaterprobabilityofdeaththanotherEDusers accordingtoa2015systematicreview.7 Mentalhealth diagnoses(MHD),includingsubstance-relateddisorders (SRD),areveryprevalentamonghighEDusers.1,4,8 Another 2013reviewreportedthatbetween0.3–18%ofpatientswith MHDwerefrequentEDusers.8 A2019Canadianstudy showedthatQuebecpatientswithMHDhadusedtheED roughlytwiceasoftenaspatientswithoutMHD,and17%of thesepatientswerehighEDusersin2015-16.9 AstheEDis notanappropriatesettingfortreatingrecurrentpatientswith MHD,theidentificationofhighEDusersandtheir characteristicsiskeytoimprovingcareamongthese vulnerablepatientsandforreducingcrowdingand healthcarecostsintheED,giventhatEDuseisoneofthe costliestcomponentsofhealthcare.10

Severalstudieshaveassessedpatientcharacteristics associatedwithhighEDuseamongpatientswithMHD, mostcomparinghighEDusersvsotherEDusers.11–17 The sociodemographiccharacteristicsdistinguishinghighED usersfromotherEDusersincludedbeingmale,15 younger, 14 single,16 havingpublichealthinsurance,11,12 andlivingin moresociallyormateriallydeprived15,18 ormetropolitan15 areas.Personalitydisorders,11,13,15,16 seriousMHD15,17 or SRD,5,17 andhavingchronicphysicalillnesses12 werethe mainclinicalcharacteristicsassociatedwithhighEDuse. HighEDusersalsodifferedfromotherEDusersintermsof higheroveralluseofmentalhealthservices.15,19,20 Toour knowledge,fewstudieshavecomparedsubgroupsoflow, high,andveryhighEDusersamongpatients.1,21 Those studieshavefocusedonMHDtoexplainthefrequencyof EDuse,includingpatientswithmultipleconditionsand withSRD,asthemainfactorleadingtoincreaseduse. VeryhighEDusersalsoreportedmorerecurrentEDusein previousyears.22 Yet,howthefrequencyofEDusewas categorizeddifferedgreatlyamongthesestudies: “veryhighEDuse” couldbeanywherebetween 8+1 and18+ visits/year.21

Abetterunderstandingofpatientcharacteristics associatedwithlow,high,andveryhighEDusersmayhelp tailorinterventionsandprogramstoEDprofilesandreduce EDuse,particularlyforhighandveryhighusers.Wefound nopreviousresearchcomparinglowEDuserstohighand veryhighusersamongpatientswithMHDorSRD.Also, moststudieswerebasedsolelyonsingle-sitehospitalhealth

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Emergencydepartment(ED)crowdingisa majorimpedimenttotheef fi cacyof healthcaresystems,causedinpartby aminorityofpatientswhousethe EDfrequently.

Whatwastheresearchquestion?

Wesoughttoassesspatients ’ characteristics andserviceusepatternsassociatedwithlow, highandveryhighEDusers.

Whatwasthemajor findingofthestudy?

Violent/disturbedbehaviorsorsocial problemsincreased5.55timestheprobability ofveryhighEDuse.

Howdoesthisimprovepopulationhealth?

Areductionofbarrierstocareandbetter accessandcontinuityofoutpatientcare shouldbeprovidedforthemost vulnerablepatients.

records.Ourstudyisoriginalinthatitintegratespatient structuredinterviewswithhealthrecordsfromfourlarge mentalhealthnetworksthatincludehospitalsand community-basedservices.VeryfewstudiesonEDuse integrateoveralloutpatientserviceuse,fromprimaryto specializedcare,andassesshowtheseservicesrelateto patientEDusefrequency.22 Moreover,fewstudieshave testedassociationsbetweenEDusefrequencyandquality ofoutpatientcareormotivationalbehaviors,suchas satisfactionwithcare,unmetneedsorperceivedstigma thatmaytriggerEDuse.

Basedontheliterature,wehypothesizedthatveryhigh EDusers,followedbyhighEDusers,wouldbemorelikely thanlowEDuserstohavecomplexhealthandsocialissues andunmetneeds,andtouseoutpatientcaremorefrequently. Weassessedvariouspatientsociodemographicandclinical characteristics,andserviceusepatternsassociatedwithlow EDuserswithMHD(1–3visits/year),comparedwithhigh EDusers(4–7visits)andveryhighEDusers(8+ visits)in fourlargeEDnetworksinQuebec(Canada).

METHODS

DescriptionoftheQuebecMentalHealthSystem

InCanada,allresidentsarecoveredbyauniversalhealth insurancemanagedattheprovinciallevel.23 Mentalhealth

Fleuryetal.

services,includingmedication,aremainlypublic,except servicessuchaspsychologicalservices,whichareusually paidbytheuserorcoveredbysomeemployers.Quebec publichealthcareservicesaremainlymanagedthrough22 largenetworks,integratinghospitals,long-termand addictionfacilities,andcommunityhealthcarecenters.24 In thesenetworks,specializedmentalhealthcareisprovidedin psychiatricdepartmentsofgeneralhospitalsorinpsychiatric hospitals,orinspecializedaddictiontreatmentcenters.25 HospitalEDstaffincludespecializedorgeneralemergency physicians,psychiatrists,andpsychosocialclinicians mostlynursesandsomesocialworkersandaddiction specialists.Primarymentalhealthcareisofferedinmedical clinicsstaffedbygeneralpractitioners,incommunity healthcarecentersmainlyprovidingpsychosocialservices, andbypsychologistsmostlyworkinginprivatepractice. Community-basedorganizations,thevoluntarysector, integratecrisisandsuicidepreventioncenters,detoxcenters, andpeersupportgroups.

StudySettingsandDataCollection

ThestudywasconductedinfourEDnetworksserving abouttwomillionpeople roughlyone-fourthofQuebec’ s population.StudyparticipantshadtobeEDusers,18+ years old,abletocompleteastructuredinterview,knowFrenchor English,andhadtogranttheresearchteamaccesstotheir healthrecords.Studyparticipantswererecruitedrandomly byEDstaffbasedonahealthrecordlistof1,751EDusers whohadMHD,includingSRD,andhadusedtheEDatleast oncewithinthefourEDnetworksinthe12monthspreceding recruitment.Ofthe first563eligiblepatientsreached,450 (80%)agreedtobereferredtotheresearchteamfor considerationasstudyparticipants.Theywerethen contactedbytheresearchcoordinatorandaskedtotakepart inastructuredtelephoneinterview,donebytrained interviewerscloselymonitoredbytheresearchteam.

TheseinterviewswereadministeredbetweenMarch1, 2021–May13,2022.Averagecompletiontimewas45 minutes.Healthrecordsforthe12monthspriortointerviews werecollectedtocomplementinterviewdata,exceptfor previousEDuse,whichwasmeasuredwithinthetwoyears priortothelastyearofEDuse.Healthrecordsdata concernedEDuse(Banquededonnéescommunesdes urgences[BDCU]database),psychiatricoutpatientservices used,hospitalization(MED-ÉCHOdatabase),and psychosocialservicesfromcommunityhealthcarecenters(ICLSCdatabase).PatientdiagnoseswereincludedinBDCU andMED-ÉCHO,andframedbytheInternational ClassificationofDiseases,Canada,10th Rev(Appendix).All healthrecordsincludedinformationonpatientserviceuse (eg,type,frequency)butexclusivelywithintheEDnetwork. Validatedbyasteeringcommitteeintegratingclinicians, structuredinterviewdataconsideredserviceuseoutsideED networksandservicesnotincludedinhealthrecords

(eg,medicalclinics,psychologists).Thesemergeddata allowedforabroaddatasetonpatientserviceuseandother patientcharacteristicspriortorecruitment.Participationin thestudywasvoluntary.Patientswhoprovidedconsent receivedamodest financialcompensation.Themultisite protocolwasapprovedbytheethicsreviewboardofthe DouglasMentalHealthUniversityInstitute.

StudyVariables

ThedependentvariablewasEDusefrequencyformental healthreasonsamongpatientswithMHD,measured12 monthspriortointerviews.Patientswerecategorizedaslow EDusers(1–3visits/year),highEDusers(4–7visits/year)or veryhighEDusers(8+ visits/year).Thestandarddefinition ofhighEDuseis4+ times/year,11,12,26 whileveryhighuse wasdefinedas8+ times/yearbasedonprevious1,27 studies andonaminimaldistributionofveryhighEDvisitsinthe studysample.Independentvariablesweresociodemographic characteristics,clinicalcharacteristics,andserviceuse patterns,againbasedonpreviousresearch.21,28

Sociodemographiccharacteristicsincludedthefollowing: sex;agegroup;educationlevel;civilstatus;employment status(eg,worker,unemployed);householdincome($Can); typeofhousing(eg,supervised);numberofsignificantsocial supportnetwork;andstigma.Allexcept “ agegroup ” were determinedbyinterviewdata.BasedontheCanadian CommunityHealthSurvey(CCHS),socialsupportwas measuredwiththefollowingquestion: “Doyouhaveoneor morepeoplearoundyouonwhomyoucanrelyforhelpwith problems?Ifyes,howmanypeople?” Alsobasedonthe CCHS,ona5-pointscale,withresponsesrangingfrom “totallydisagree” to “totallyagree” (greatest stigmatization),stigmawasmeasuredwiththefollowing affirmation: “Mostpeopleinmycommunitytreataperson withaMHDorSRDinthesamemannerastheywouldtreat anyotherperson.”

Clinicalcharacteristicsincludedthefollowing:MHD; SRD;suicidalbehaviors(suicideideationorattempt); violent/disturbedbehaviorsorsocialproblems;chronic physicalillnesses(eg,heartdiseases,diabetes);co-occurring MHD-SRD;andhightriagepriorityamongEDusers.All thesevariableswerebasedonhealthrecords,exceptSRD, whichwasbasedonbothhealthrecordsandthestructured interviews.TheMHDincludedseriousMHD(schizophrenia spectrumandotherpsychoticdisorders,andbipolar disorders),personalitydisorders,andcommonMHD (anxiety,depressiveandadjustmentdisorders;attention deficit/hyperactivitydisorder).TheSRDintegratedalcoholanddrug-relateddisorders(use,induced,intoxicationand withdrawal),measuredusinghealthrecordsalongwiththe AlcoholUseDisordersIdentificationTest29 andtheDrug AbuseScreeningTest-20.30 Thesewereincludedinthe structuredinterviews,asSRDareoftenunderdiagnosedin healthrecords.31 Weidentifiedchronicphysicalillnessesand

theirseverity(0to2+)basedonanadaptedversion integratingboththeCharlsonandElixhausercomorbidity indexes.32 TheEDtriageprioritywasbasedontheCanadian TriageAcuityScale,33 consistingof fiveprioritylevelsor illnessseverity,withlevels4–5consideredtreatablein outpatientcare.33 Inthisstudy,hightriagepriorityEDuse (1–3)wasconsideredaproxyforfunctionaldisability,based onmeanofnumberofEDvisitsperpatient,with1–3triage prioritydividedbytotalofEDvisitsperpatient(1–5).

Patientserviceuseincludedthefollowing:knowledgeof mentalhealthoraddictionservices;havingafamilydoctoror otherregularcareclinician;frequencyofprimarycare, community-based,andspecializedoutpatientservicesused; overallsatisfactionwithoutpatientservicesused;numberof barriersrelatedtounmetneeds;frequencyofhospitalization, andfrequencyofpreviousEDuse.Patientserviceuseinthe EDnetworks,mostlymentalhealthspecializedcareand someprimarycareservices(communityhealthcarecenters), wasbasedonhealthrecords,andservicesoutsidetheED networkswerereportedinthestructuredinterviews mostly primarycare,community-based,orspecializedaddiction services.Serviceusemeasuredwithbothtypesofdata integratedonlythehighestfrequencyofserviceuse patientsreported.Asaproxyofcontinuityofcare,patients wereaskediftheywerefollowedregularlybyafamilydoctor orotherclinicians.Basedonapreviousstudy,34 the benchmarkforfrequentserviceuse,orminimalintensityof optimalcare,was5+ follow-upappointments/year.Primary careincludedservicesreceivedfromfamilydoctors,general practitionersinwalk-inclinics,psychologistsinprivate practice,andpsychosocialcliniciansincommunity healthcarecenters.

Community-basedorganizationsintegratedcrisisand suicidepreventioncenters,etc.Specializedoutpatientcare includedpsychiatricservices(eg,treatmentfrompsychiatrist teams,assertivecommunitytreatment,andintensivecase managementprograms),andservicesfromaddiction treatmentcenters.Patientswereaskedtoindicateona 5-pointscaletheiryearlysatisfactionwitheachoutpatient servicereceived.Wecalculatedthemeansatisfactionscore, withhigherscoresindicatinggreatersatisfaction.Unmet needsweremeasuredthroughthefollowingCCHSquestion:

“Couldyouexplainthereasonswhyservicesoutsideofthe EDdidnotrespondtoyourneeds?” includingmultiple choiceofbarrierstocare(eg, “Iprefertomanagebymyself;” “Thehelpisnotreadilyavailable”).Thenumberofbarriers wascountedas0,1–2,or3+.FrequencyofpreviousEDuse included4–7(highEDusers)and8+ EDvisits(veryhighED users),measuredforthetwo-yearperiodprecedingthe 12-monthinterviewperiod.

Analyses

Missingvalues(<1%)wereimputedbymeanfor continuousvariablesandmodeforcategoricalvariables.35

Descriptiveanalysesincludedpercentagesforcategorical variablesandmeanvaluesforcontinuousvariables.Weused bivariatemultinomiallogisticregressiontoexaminethe associationsbetweeneachindependentvariableandthe dependentvariable,frequencyofEDuse.Theintraclass correlationcoefficient(ICC)forthestudywassmall(<0.01), indicatinglowsharedvarianceamongpatientsfromtheED networks;multilevelanalysiswasnotrequired.Basedon criterionproceduresforforwardmodelselection, independentvariablesidentifiedassignificantinthebivariate analyses(Alpha:0.20)36 wereenteredsequentiallyintothe multivariablemultinomiallogisticregressionmodelfor frequencyofEDuse,withlowEDuse(1–3visits/year)asthe referencegroup.WeusedtheAkaikeInformationCriterion (AIC)37 tocomparetherelativegoodnessof fitamong differentmodelsbeforeselectingthe finalmultivariatemodel withthesmallestAICthatbest fitthedata.Wealsoused varianceinflationfactor(VIF)tomeasuretheamountof multicollinearityinregressionanalysisandfoundsmaller than4,indicatingthatmulticollinearitywasnotaconcern.38 Relativeriskratios(RRR)and95%confidenceintervals(CI) werecalculatedinthe finalmodel.Weperformedstatistical analysesusingStata17(StataCorpLLC,College Station,TX).

RESULTS

Ofthe450EDusersreferred,50couldnotbereachedand 300agreedtoparticipateinthestudy(75%responserate). Onepatientwaswithdrawn.Ofthe299patientsinthe final sample,amajority(55%)werewomen;39%were30–49years old,82%single,and57%unemployedorretired;47%hada householdincomeoflessthanCAN$20,000;57%hadpostsecondaryeducation,58%livedinrentalhousing,and50% perceivedhighstigma(Table1).Overhalf(57%)had commonMHD,44%seriousMHD,42%personality disorders,59%SRD,and45%chronicphysicalillnesses;38% hadco-occurringMHD-SRD,54%suicidalbehaviors,and 17%violent/disturbedbehaviorsorsocialproblems.Interms ofEDuse,39%werelowEDusers(1–3visits/year),37%high EDusers(4–7visits/year),and24%veryhighEDusers(8+ visits/year)(Table2).Nearlyhalf(46%)hadpoortofair knowledgeofmentalhealthoraddictionservices;88%hada familydoctor(74%)orotherregularcareclinician(58%).In thepreviousyear,58%hadused5+ primarycareservices, 26%5+ servicesfromcommunity-basedorganizations,and 65%5+ specializedoutpatientcare.Overallsatisfactionwith outpatientservicesaveraged4.02/5;37%ofparticipantshad unmetneeds,with15%identifying3+ barriers.Amajority (56%)werehospitalized,35%ofthose1–2times, and39%hadbeenveryhighEDusersovertheprevious two-yearperiod.

Wecomparedvariablesassociatedwithhighorveryhigh EDuserswithvariablesamonglowEDusers(Table 3). Womenhad1.30timesmoreprobabilityofbeinghighED

Table1.

1Patientstructuredinterviews. 2Banquededonnéescommunesdesurgences (BDCU,EDdatabase). 3Thesamplewastoosmalltoseparate unemployedfromretired. 4Supervisedhousingincludedgrouphomes,residentialcare,supportedapartments,etc. 5Maintenanceet exploitationdesdonnéespourl’étudedelaclientèlehospitalière (MED-ÉCHO,hospitalizationdatabase). 6Patientsmayhavemorethan oneMHD. 7AlcoholUseDisordersIdenti ficationTest(AUDIT). 8DrugAbuseScreeningTest-20(DAST-20).Detailsofdiagnosticcodes arepresentedinthe Appendix ED,emergencydepartment.

Serviceuse(measuredintheprevious12months,orotherasspecified)

Group

LowED users(1–3 visits/year)

HighED users(4–7 visits/year)

Veryhigh EDusers (8+ visits/

1Seenote 1belowTable 1 2Seenote 2belowTable 1 3Systèmed’informationpermettantlagestiondel’informationcliniqueetadministrative dansledomainedelasantéetdesservicessociaux (I-CLSC,communityhealthcarecenterdatabase). 4Psychiatricoutpatientservicesused database. 5BasedontheCCHS,barrierstocareexplainingunmetneedswerea)Ipreferredtomanagebymyself;b)Ihaven’tgottenaround toityet(eg,toobusy);c)Ididn’thaveenoughconfidenceinthehealthcaresystemorsocialservices;d)Iwasafraidaboutwhatotherswould thinkofme;e)Ipreferredtoaskmyfamilyorfriendsforhelp;f)Iamdissatisfiedwiththequalityofservices;g)Idon’tknowhoworwheretoget thiskindofhelp;h)Myjobinterferedwithpossibletreatment(eg,hoursofwork);i)Thehelpisnotreadilyavailable;j)Icouldnotaffordtopay; myinsurancedidn’tcoverthecost;andk)Servicesarenotofferedinmylanguage. 6Seenote 5belowTable 1 ED,emergencydepartment.

usersthanmen.Patientslivinginrentalhousinghad2.09 timesmoreprobabilityofbeingveryhighEDusersthan thoselivinginprivatehousing.Patientsexhibitingviolent/ disturbedbehaviorsorsocialproblems,orchronicphysical illnesses,respectively,showed2.87and1.02timesincreasein probabilityofhighEDuse,anda5.55and4.95timesgreater probabilityofveryhighEDuse.Patientswithpersonality disordershad1.06timesgreaterprobabilityofhighEDuse, andthosewithsuicidalbehaviors,a1.29increased

probabilityofveryhighEDuse.Patientswith3+ barriers relatedtounmetneedshad1.64and2.27timesgreater probabilityofbeinghighorveryhighEDusers, respectively.Patientswith5+ primarycareservicesandhigh recurrentEDusehad2.5and1.53timesgreaterprobability ofbeingveryhighEDusers.Patientshospitalized 1–2timeshadareducedprobabilityof54%forhigh and79%forveryhighEDuse,comparedwiththose nothospitalized.

Table2. Serviceuseofpatientsusingtheemergencydepartment(N=299).

HighEDusers (4–7visits/year)

VeryhighEDusers (8+ visits/year)

RRR* P-value95%CI*RRR* P-value95%CI* Sociodemographiccharacteristics(measuredintheprevious12months)

Womenvsmen2.300.0071.254.231.480.3070.703.16

Typeofhousing1

Rentalvsprivate1.430.3260.702.943.090.0361.088.85

Supervisedvsprivate0.810.6310.341.942.180.2000.667.18

Clinicalcharacteristics(measuredintheprevious12months)

Personalitydisorders2.040.0391.044.012.260.0550.985.18

Suicidalbehaviors(suicideideationorattempt)1.810.0630.973.382.290.0461.015.16

Violent/disturbedbehaviorsorsocialproblems3.870.0051.529.856.550.0012.2619.00

Chronicphysicalillnesses2.020.0431.024.005.950.0002.5014.13

Serviceuse(measuredintheprevious12months,orotherasspecified)

Frequencyofprimarycareserviceuse

1

FrequencyofpreviousEDuse(measuredwithinthe2yearspriortothe12-monthperiod inwhichinterviewswereconducted)

ED,emergencydepartment;*RRR,relativeriskratio; CI,confidenceinterval. 1Seenote 4belowTable 1 2Seenote 5belowTable 2

DISCUSSION

Inthisstudyweaimedtoidentifysociodemographicand clinicalcharacteristics,aswellasserviceuse,amongpatients withMHD,comparinglow(1–3visits/year)tohigh(4–7 visits)andveryhighEDuse(8+ visits)formentalhealth reasons.Mostpatientshadhigh(37%)orveryhigh(24%)ED use,whichmaybeexplainedbythesubstantialsocialand healthissuestheyfaced.Theirlevelsofsocialandmaterial deprivationwerehigh,aswastheirperceivedstigma.Nearly halfhadseriousMHD,personalitydisordersorchronic physicalillnesses,whilemostexperiencedSRDandsuicidal behaviors.About40%reportedunmetneedsorpoorto fairknowledgeofservices,whichmayexplaintheirhigh overallEDuse.Asfoundinotherstudies,13,28 mosthighED userswerealsohighusersofoutpatientcareandwere frequentlyhospitalized.

Findingspartlyconfirmedthehypothesesthatveryhigh EDusers,followedbyhighEDusers,weremorelikelythan

lowEDuserstohavecomplexhealthandsocialissues,unmet needs,andtomakemorefrequentuseofoutpatientcare.The result showingthatdisturbed/violentbehaviorsorsocial problemswerethepatientcharacteristicsmoststrongly associatedwithbothveryhighandhighEDuse underlined thespecialneedsofthesepatients,whoforsomewerelikely involuntaryEDusers.Policearefrequentlycalledintodeal withpeoplepresentingviolentorerraticbehaviorsandto transportthemtoED.39

Interventionplans40 integratingbehavioraltreatment41 andhelpincrisisresolution42,43 maybebetterdeployedfor thesehighandveryhighEDusers.Studieshaveshownthat fewoverallinterventionsarebeingdeployedintheEDfor highusers.44,45 Previousstudieshavealsoshownthat patientswithchronicphysicalillnessesmademoreED visits.21,26 Thosewithco-occurringissueshadpoorerhealth overall,higherriskofmedicationinteractions46 andmore distress,47 explainingtheirfrequentEDuse.Improving

Table3. Estimationsofmultivariablemultinomiallogisticregressionmodelonemergencydepartment(ED)visits(referencegroup: lowEDusers,1–3visits/year).

collaborativecare48 betweenpsychiatristsandprimarycare servicesforbettertreatmentofpatientswithco-occurring issuesmayalsoreducetheirEDuse.

Higherperceivedbarriersforunmetneedswerealso stronglyassociatedwithmoreEDuse.Barriersmaybe structural(eg,lackofaccesstoservices)ormotivational (eg,duetodistrustordissatisfactionwithservices).49 AUS studyonbarrierstocareamongfrequentEDusersfoundthat mostofthemperceivedtheEDastheonlyplacewheretheir healthproblemswouldbetreated.50 Theseresultshighlight theimportanceofacknowledgingbarrierstooutpatientcare anddevelopingmorepersonalizedpatientcarebasedon recovery-orientatedserviceswithpatient-centred interventions,51,52 oralternative “rapid” specialized responsesforpatientswithMHDincrisis.53,54 Evenifvery highEDusersreceivedprimarycaremorefrequently,it doesn’tmeanthoseserviceswereadequateorsufficientto reduceorpreventunmetneeds.

Our findingthatbeinghospitalized1-2times,butnot3+ times/year,wasprotectiveagainsthighorveryhighEDuse comparedwithnotbeinghospitalized,wasanoriginalresult. Mosthospitalizedpatientsarereferredbyemergency physicians,55 whichmightsuggestthattheserepeated hospitalizedpatientshaveveryserioushealthconditionsand thattheirinpatientcareepisodesmaybeunavoidable.Lack ofabilitytorefer(eg,timeofday)orpossibilitytorefer (eg,longwaitinglists)tooutpatientcare,lackofmental healthsupportintheED(eg,briefinterventionteams)56,57 or ofcomfortintreatingpatientswithmorecomplexMHD profilesinoutpatientcaremightalsoexplainfrequentpatient hospitalizations.Hospitalizationmaysometimesbethemost appropriatesolutionformaximizingpatientrecovery.58 For patientswith1–2hospitalizations/year,closefollow-up care,59,60 whichisincreasinglyrecommendedfollowing discharge,mayhavecontributedtoreducingtheirEDuse. Diversifiedstrategiessuchasassertivecommunitytreatment programs, 61 hometreatmentteams,62 short-staycrisis units,63 andcrisisinterventionteams64 arealsoincreasingly beingpromotedtohelpreduceacutecareuse.Althoughsuch interventionsremaininsufficientlydeployedinQuebec,the province’snewMentalHealthActionPlan(2022–2026) promisestoincreasetheiruse.25

ComparedtolowEDusers,veryhighEDusershada higherprobabilityofhavingsuicidalbehaviors,whilehigh usersshowedhigherprobabilityofhavingpersonality disorders.Previousstudieshavefoundassociationsforboth theseissueswithgreaterEDuse.13,16,28 Consideringthat healthcaresystemstendtorespondpoorlytocrisis situations,55 especiallythosethatoccuroutsideregular businesshours,thefactthatthesestudyparticipantswere veryhighEDuserswasnotsurprising.Greateravailabilityof sustainedpsychosocialprogramsinprimarycareandmore specializedcrisisandsuicidalpreventionservices65 mayhelp preventEDvisitsforsuicidalbehaviors.66 Dialectical

behaviortherapymayalsobepromotedmoreextensivelyto reducesymptomsofpersonalitydisorders,borderline personalitydisorderinparticular,asreportedinasystematic review.67 Ingeneral,theEDshouldnotreplaceoutpatient careforvulnerablepatients,astheircapacitytotreatsuch patientswasidentifiedaslimited.68,69

WomenhadagreaterprobabilityofhighEDusethan men,andpatientslivinginrentalhousingshowedagreater probabilityofveryhighEDusethanthoseinprivate housing.Womenreportedlyusemorehealthservicesthan men, 70 whichforhighEDusecontradictedpreviousstudies thatfoundmoremenwerehighEDusers.15,26 Becausehigh andveryhighEDusersweredifferentiatedinourstudy,it mayaccountforthisdivergentresult,withnodifference foundbetweenwomenandmeninveryhighEDusers.The compositionofourstudysamplecouldalsoexplainthis finding,asamajorityofparticipantsrecruitedrandomlyby EDstaffwerewomen.Concerningpatientsresidinginrental housing,theymayexperiencegreaterdeprivation,including inadequatehousingsupport,comparedwiththoselivingin privateorsupervisedhousing,whichmayaccountfortheir veryhighEDuse.Sometypeofsupportivehousingwithcase management71 mayhelpthesepatientsavoidfrequentED use.Difficultytoaccessoutpatientcarebecauseoflong waitinglistsortransportationissuesmightalsoexplainvery highEDuseamongthesepatients.

Using5+ primarycareservices/yearandrecurrenthigh EDusewereonlyassociatedwithveryhighEDusers comparedtolowEDusers,butnothighEDusers.Asfor highEDusers,studieshaveidentifiedthemashighservice usersingeneral,72 andasbeing “recurrent” EDusersover severalconsecutiveyears.6,28 Ourstudyaddedtothis literaturebyspecifyingthatonlypatientswhomadeatleast fiveprimarycareappointmentsinthepreviousyearandeight EDvisitsintheprevioustwoyearshadagreaterprobability ofbeingveryhighEDusers(8+ EDvisits/year).Thegreater useofprimarycareservicesamongveryhighEDusersmay beexplainedbytheirhigherratesofchronicphysicalillnesses andthegreaterseverityoftheseconditions,comparedwith ratesforlowandhighEDusers.Perhapsprimarycarewas notadequateorcontinuousenoughtopreventEDuse 22,73 or topreventorreduceunmetneeds.Generalpractitionershave beenshowntolacktrainingorsufficientteamcapacityto adequatelyfollowuponvulnerablepatientswithMHD.74,75 Collaborativecaremaybemorepromotedbetweenprimary andpsychiatriccareandteamworktoreduceEDuseand bettertreatthesepatients.76,77

LIMITATIONS

Thisstudyhadcertainlimitationsthatshouldbenoted. First,thereisnoconsensualdefinitionforlow,high,andvery highEDuse.Differentdefinitionsthanthosechosenhere couldhaveledtodifferent findings.Second,thestudyresults weredifficulttocomparewiththeliteratureasmoststudies

havecomparedhighEDusewithotherEDuse.Third, structuredinterviewsmaybebiasedduetothepatients’ abilitytorecall,andthehealthrecordsthatwereused reflectedserviceuseonlywithintheparticipatingnetworks. Finally,thediversityofhealthcaresystemsmaylimitthe generalizationofthestudy findings,especiallyincountries thatdon’thavepublichealthcarecoveragefor deprivedpopulations.

CONCLUSION

Thisstudywasinnovativeinthewayitcomparedlow, high,andveryhighEDusersamongpatientswithMHDin Canada,andbyusingbothpatientstructuredinterviewsand healthrecords.The findingsconfirmedthathigherEDuse wasassociatedwithcomplexpatienthealthissuesandhigher perceivedbarrierstounmetneeds.Patientswithveryhigh overallEDusehadthemostsevereconditions,including greaterhousingvulnerabilityandisolation,andmore suicidalbehaviors.Theyalsousedmoreprimarycare services,possiblybecauseoftheirseverechronicphysical healthconditions.

RecurrentEDuseovertheyearsalsodistinguishedvery highEDusersfromlowusers.Bycontrast,theriskofhigh andveryhighEDusewasreducedinpatientswith1–2 hospitalizations/year,whichunderlinesthepotentialbenefits andpertinenceofhospitalizationforsomepatients.Overall, barrierstocareshouldbereducedandbetteraccessand continuityofoutpatientcareprovidedforthemost vulnerablepatients,integratingcrisisresolutionand supportedhousingservices.Thismayreducethenumberof patientswithMHDintheED,decreasingwaittimesand improvingcareintheED.

2.MinassianA,VilkeGM,WilsonMP.Frequentemergencydepartment visitsaremoreprevalentinpsychiatric,alcoholabuse,anddual diagnosisconditionsthaninchronicviralillnessessuchashepatitisand humanimmunodeficiencyvirus. JEmergMed.2013;45(4):520–5.

3.GiannouchosTV,KumHC,FosterMJ,etal.Characteristicsand predictorsofadultfrequentemergencydepartmentusersintheUnited States:asystematicliteraturereview. JEvalClinPract 2019;25(3):420–33.

4.KriegC,HudonC,ChouinardMC,etal.Individualpredictorsoffrequent emergencydepartmentuse:ascopingreview. BMCHealthServRes 2016;16(1):594.

5.LaCalleEandRabinE.Frequentusersofemergencydepartments:the myths,thedata,andthepolicyimplications. Review.AnnEmergMed 2010;56(1):42–8.

6.BillingsJandRavenMC.Dispellinganurbanlegend:frequent emergencydepartmentusershavesubstantialburdenofdisease. HealthAff(Millwood).2013;32(12):2099–108.

7.MoeJ,KirklandS,OspinaMB,etal.Mortality,admissionratesand outpatientuseamongfrequentusersofemergencydepartments: asystematicreview. EmergMedJ.2016;33(3):230–6.

8.VandykAD,HarrisonMB,VanDenKerkhofEG,etal.Frequent emergencydepartmentusebyindividualsseekingmentalhealthcare: asystematicsearchandreview. ArchPsychiatrNurs.2013;27(4):171–8.

9.FleuryMJ,FortinM,RochetteL,etal.Assessingqualityindicators relatedtomentalhealthemergencyroomutilization. BMCEmergMed 2019;19(1):8.

10.OndlerC,HegdeGG,CarlsonJN.Resourceutilizationandhealthcare chargesassociatedwiththemostfrequentEDusers. AmJEmergMed 2014;32(10):1215–9.

11.ChangG,WeissAP,OravEJ,etal.Predictorsoffrequentemergency departmentuseamongpatientswithpsychiatricillness. GenHosp Psychiatry.2014;36(6):716–20.

AddressforCorrespondence:Marie-JoséeFleury,PhD,Douglas MentalHealthUniversityInstituteResearchCentre,6875,LaSalle Blvd.,Verdun,Canada.Email: flemar@douglas.mcgill.ca

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.ThisstudywasfundedbytheCanadian InstitutesofHealthResearch(CIHR,grantnumber:8400997).

Copyright:©2024Fleuryetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.LiuSW,NagurneyJT,ChangY,etal.FrequentEDusers:aremostvisits formentalhealth,alcohol,anddrug-relatedcomplaints? AmJEmerg Med.2013;31(10):1512–5.

12.BuhumaidR,RileyJ,SattarianM,etal.Characteristicsoffrequentusers oftheemergencydepartmentwithpsychiatricconditions. JHealthCare PoorUnderserved.2015;26(3):941–50.

13.Richard-LepourielH,WeberK,BaertschiM,etal.Predictorsofrecurrent useofpsychiatricemergencyservices. PsychiatrServ 2015;66(5):521–6.

14.SirotichF,DurbinA,DurbinJ.Examiningtheneedprofilesofpatients withmultipleemergencydepartmentvisitsformentalhealthreasons: across-sectionalstudy. SocPsychiatryPsychiatrEpidemiol 2016;51(5):777–86.

15.FleuryMJ,RochetteL,GrenierG,etal.Factorsassociatedwith emergencydepartmentuseformentalhealthreasonsamonglow, moderateandhighusers. GenHospPsychiatry.2019;60:111–9.

16.SlankamenacK,HeidelbergerR,KellerDI.Predictionofrecurrent emergencydepartmentvisitsinpatientswithmentaldisorders. FrontPsychiatry.2020;11:48.

17.VuF,DaeppenJB,HugliO,etal.Screeningofmentalhealthand substanceusersinfrequentusersofageneralSwissemergency department. BMCEmergMed.2015;15:27.

18.GaulinM,SimardM,CandasB,etal.Combinedimpactsof multimorbidityandmentaldisordersonfrequentemergencydepartment visits:aretrospectivecohortstudyinQuebec,Canada. CMAJ 2019;191(26):E724–32.

19.MitchellMS,LeonCLK,ByrneTH,etal.Costofhealthcareutilization amonghomelessfrequentemergencydepartmentusers. PsycholServ 2017;14(2):193–202.

20.KorczakV,ShanthoshJ,JanS,etal.Costsandeffectsof interventionstargetingfrequentpresenterstotheemergency department:asystematicandnarrativereview. BMCEmergMed 2019;19(1):83.

21.DoupeMB,PalatnickW,DayS,etal.Frequentusersofemergency departments:developingstandarddefinitionsanddefiningprominent riskfactors. AnnEmergMed.2012;60(1):24–32.

22.GentilL,GrenierG,VasiliadisHM,etal.Predictorsofrecurrenthigh emergencydepartmentuseamongpatientswithmentaldisorders.

IntJEnvironResPublicHealth.2021;18(9):4559.

23.MinistèredelaSantéetdesServicesSociaux(MSSS).Lesystèmede santéetdeServicesSociauxauQuébec,enbref.Quebec:Government ofQuebec.2017.Availableat: https://publications.msss.gouv.qc.ca

AccessedJanuary27,2022.

24.MinistèredelaSantéetdesServicessociaux(MSSS).Projetdeloino 10(2015,chapitre1).Loimodifiantl’organisationetlagouvernancedu réseaudelasantéetdesservicessociauxnotammentparl’abolitiondes agencesrégionales.Quebec:GovernmentofQuebec.2015.

Availableat: https://www2.publicationsduquebec.gouv.qc.ca

AccessedJanuary21,2022.

25.Ministèredelasantéetdesservicessociaux(MSSS).LePland’action interministérielensantémentale2022-2026-S’unirpourunmieux-être collectifPublicationsduministèredelaSantéetdesServicessociaux. Quebec:GovernmentofQuebec.2022.Availableat: https:// publications.msss.gouv.qc.ca/msss/document-003301 AccessedJanuary27,2022.

26.BrennanJJ,ChanTC,HsiaRY,etal.Emergencydepartmentutilization amongfrequentuserswithpsychiatricvisits. AcadEmergMed 2014;21(9):1015–22.

27.MoeJ,BaileyAL,OlandR,etal.Defining,quantifying,and characterizingadultfrequentusersofasuburbanCanadianemergency department. CJEM.2013;15(4):214–26.

28.ArmoonB,CaoZ,GrenierG,etal.Profilesofhighemergency departmentuserswithmentaldisorders. AmJEmergMed 2022;54:131–41.

29.BohnMJ,BaborTF,KranzlerHR.Thealcoholusedisorders identificationtest(AUDIT):validationofascreeninginstrumentforusein medicalsettings. JStudAlcohol.1995;56(4):423–32.

30.SkinnerHA.Thedrugabusescreeningtest. AddictBehav 1982;7(4):363–71.

31.HuynhC,KiselyS,RochetteL,etal.Usingadministrativehealthdatato estimateprevalenceandmortalityratesofalcoholandothersubstancerelateddisordersforsurveillancepurposes. DrugAlcoholRev 2021;40(4):662–72.

32.SimardM,SiroisC,CandasB.Validationofthecombinedcomorbidity indexofCharlsonandElixhausertopredict30-daymortalityacross ICD-9andICD-10. MedCare.2018;56(5):441–7.

33.CanadianAssociationofEmergencyPhysicians.Canadiantriageacuity scale.2012.Availableat: https://ctas-phctas.ca AccessedDecember18,2021.

34.HarrisMG,HobbsMJ,BurgessPM,etal.Frequencyandqualityof mentalhealthtreatmentforaffectiveandanxietydisordersamong Australianadults. MedJAust.2015;202(4):185–9.

35.DziuraJD,PostLA,ZhaoQ,etal.Strategiesfordealingwithmissing datainclinicaltrials:fromdesigntoanalysis. YaleJBiolMed 2013;86(3):343–58.

36.MickeyJandGeenlandS.Astudyoftheimpactofcounfounderselectioncriteriaoneffectestimation. AmJEpidemiol 1989;129(1):125–37.

37.AkaikeH.Informationtheoryandanextensionofthemaximum likelihoodprinciple.In:PetrovBNandCsakiF(Eds.), Proceedingsofthe 2ndInternationalSymposiumonInformationTheory (267–81). Hungary,Budapest:AkademiaiKiado,1973.

38.HairJ,BlackWC,BabinBJ,etal.(2001). MultivariateDataAnalysis (7thed.).NewJersey,UpperSaddleRiver:PearsonEducation.

39.ShortTBR.Thenatureofpoliceinvolvementinmentalhealthtransfers. PolicePractRes.2014;15(4):336–48.

40.AbelloA,Jr.,BriegerB,DearK,etal.Careplanprogramreduces thenumberofvisitsforchallengingpsychiatricpatientsintheED. AmJEmergMed.2012;30(7):1061–7.

41.FrazierSNandVelaJ.Dialecticalbehaviortherapyforthetreatmentof angerandaggressivebehavior:areview. AggressViolentBehav 2014;19(2):156–63.

42.WheelerC,Lloyd-EvansB,ChurchardA,etal.Implementationofthe crisisresolutionteammodelinadultmentalhealthsettings:asystematic review. BMCPsychiatry.2015;15:74.

43.VakkalankaJP,NeuhausRA,HarlandKK,etal.Mobilecrisisoutreach andemergencydepartmentutilization:apropensityscore-matched analysis. WestJEmergMed.2021;22(5):1086–94.

44.GabetM,ArmoonB,MengX,etal.Effectivenessofemergency departmentbasedinterventionsforfrequentuserswithmentalhealth issues:asystematicreview. AmJEmergMed.2023;74:1–8.

45.GabetMandFleuryM-J.Innovationsorganisationnellesauxurgences pouraméliorerlaqualitédessoinsdispensésauxpatientssouffrantde troublesmentaux:perspectivesinternationales. JGestÉconSanté 2022;40(2-3):100–15.

46.KangHJ,KimSY,BaeKY,etal.Comorbidityofdepressionwith physicaldisorders:researchandclinicalimplications. ChonnamMedJ 2015;51(1):8–18.

47.QinP,HawtonK,MortensenPB,etal.Combinedeffectsofphysical illnessandcomorbidpsychiatricdisorderonriskofsuicideinanational populationstudy. BrJPsychiatry.2014;204(6):430–5.

48.IvbijaroGO,EnumY,KhanAA,etal.Collaborativecare:modelsfor treatmentofpatientswithcomplexmedical-psychiatricconditions. CurrPsychiatryRep.2014;16(11):506.

49.MojtabaiR,OlfsonM,SampsonNA,etal.Barrierstomentalhealth treatment:resultsfromtheNationalComorbiditySurveyReplication. PsycholMed.2011;41(8):1751–61.

50.BirminghamLE,CochranT,FreyJA,etal.Emergencydepartmentuse andbarrierstowellness:asurveyofemergencydepartmentfrequent users. BMCEmergMed.2017;17(1):16.

51.ChesterP,EhrlichC,WarburtonL,etal.Whatistheworkofrecovery orientedpractice?Asystematicliteraturereview. IntJMentHealthNurs 2016;25(4):270–85.

52.CareyTA.Beyondpatient-centeredcare:enhancingthepatient experienceinmentalhealthservicesthroughpatient-perspectivecare. PatientExpJ.2016;3(2):46–9.

53.HubbelingDandBertramR.CrisisresolutionteamsintheUKand elsewhere. JMentHealth.2012;21(3):285–95.

54.BoucheryEE,BarnaM,BabalolaE,etal.Theeffectivenessofapeerstaffedcrisisrespiteprogramasanalternativetohospitalization. PsychiatrServ.2018;69(10):1069–74.

55.JohnsonS,Dalton-LockeC,BakerJ,etal.Acutepsychiatriccare: approachestoincreasingtherangeofservicesandimprovingaccess andqualityofcare. WorldPsychiatry.2022;21(2):220–36.

56.GabetM,GrenierG,CaoZ,etal.Implementationofthreeinnovative interventionsinapsychiatricemergencydepartmentaimedatimproving serviceuse:amixed-methodstudy. BMCHealthServRes 2020;20(1):854.

57.TurnerSBandStantonMP.Psychiatriccasemanagementinthe emergencydepartment. ProfCaseManag.2015;20(5):217–27; quiz228-9.

58.JunWHandYunSH.Mentalhealthrecoveryamonghospitalized patientswithmentaldisorder:associationswithangerexpressionmode andmeaninginlife. ArchPsychiatrNurs.2020;34(3):134–40.

59.GentilL,GrenierG,FleuryMJ.Factorsrelatedto30-dayreadmission followinghospitalizationforanymedicalreasonamongpatientswith mentaldisorders[Facteursliesalarehospitalisationa30jourssuivant unehospitalisationpouruneraisonmedicalechezdespatientssouffrant detroublesmentaux]. CanJPsychiatry.2021;66(1):43–55.

60.KurdyakP,VigodSN,NewmanA,etal.Impactofphysicianfollow-up careonpsychiatricreadmissionratesinapopulation-basedsampleof patientswithschizophrenia. PsychiatrServ.2018;69(1):61–8.

61.AddingtonD,AndersonE,KellyM,etal.Canadianpracticeguidelines forcomprehensivecommunitytreatmentforschizophreniaand schizophreniaspectrumdisorders. CanJPsychiatry 2017;62(9):662–72.

62.BoisvertA,BouffardA-P,PaquetK.Letraitementintensifbrefà domicile. SantéMentale.2016;204:68–72.

63.AndersonK,GoldsmithLP,LomaniJ,etal.Short-staycrisisunitsfor mentalhealthpatientsoncrisiscarepathways:systematicreviewand meta-analysis. BJPsychOpen.2022;8(4):e144.

64.MurphySM,IrvingCB,AdamsCE,etal.Crisisinterventionfor peoplewithseverementalillnesses. CochraneDatabaseSystRev 2015;2015(12):CD001087.

65.MisharaBL,CoteLP,DargisL.Systematicreviewofresearchand interventionswithfrequentcallerstosuicidepreventionhelplinesand crisiscenters. Crisis.2023;44(2):154–67.

66.GentilL,GrenierG,FleuryMJ.Determinantsofsuicidalideationand suicideattemptamongformerandcurrentlyhomelessindividuals. SocPsychiatryPsychiatrEpidemiol.2020;56(5):747–57.

67.BloomJM,WoodwardEN,SusmarasT,etal.Useofdialecticalbehavior therapyininpatienttreatmentofborderlinepersonalitydisorder: asystematicreview. PsychiatrServ.2012;63(9):881–8.

68.DeLeoK,MaconickL,McCabeR,etal.Experiencesofcrisiscare amongserviceuserswithcomplexemotionalneedsoradiagnosisof 'personalitydisorder',andotherstakeholders:systematicreviewand meta-synthesisofthequalitativeliterature. BJPsychOpen 2022;8(2):e53.

69.StanleyB,BrownGK,BrennerLA,etal.Comparisonofthesafety planninginterventionwithfollow-upvsusualcareofsuicidalpatients treatedintheemergencydepartment. JAMAPsychiatry 2018;75(9):894–900.

70.PinkhasovRM,ShteynshlyugerA,HakimianP,etal.Aremen shortchangedonhealth?Perspectiveonlifeexpectancy,morbidity,and mortalityinmenandwomenintheUnitedStates. IntJClinPract 2010;64(4):465–74.

71.McPhersonP,KrotofilJ,KillaspyH.Mentalhealthsupported accommodationservices:asystematicreviewofmentalhealthand psychosocialoutcomes. BMCPsychiatry.2018;18(1):128.

72.PinesJM,AsplinBR,KajiAH,etal.Frequentusersofemergency departmentservices:gapsinknowledgeandaproposedresearch agenda. AcadEmergMed. 2011;18(6):e64–9.

73.BarkerLC,SunderjiN,KurdyakP,etal.Urgentoutpatientcarefollowing mentalhealthedvisits:apopulation-basedstudy. PsychiatrServ. 2020;71(6):616–9.

74.FleuryMJ,BamvitaJM,FarandL,etal.GPgroupprofilesand involvementinmentalhealthcare.Researchsupport,non-U.S.Gov’t. JEvalClinPract.2012;18(2):396–403.

75.LoebDF,BaylissEA,BinswangerIA,etal.Primarycarephysician perceptionsoncaringforcomplexpatientswithmedicalandmental illness. JGenInternMed.2012;27(8):945–52.

76.AupontO,DoerflerL,ConnorDF,etal.Acollaborativecaremodelto improveaccesstopediatricmentalhealthservices. AdmPolicyMent Health.2013;40(4):264–73.

77.RaneyLE.Integratingprimarycareandbehavioralhealth:therole ofthepsychiatristinthecollaborativecaremodel. AmJPsychiatry 2015;172(8):721–8.

BridgingtheGap:EvaluationofanElectrocardiogramCurriculum forAdvancedPracticeClinicians

StevenLindsey,MD*

TimP.Moran,PhD*

MeredithA.Stauch,MSN,APRN,FNP-BC,ENP-C†

AlexisL.Lynch,MSN,APRN,ENP-C,FNP-BC,ANP-BC*

KristenGrabowMoore,MD,MED*

SectionEditor:NielsRathlev,MD

*EmoryUniversity,DepartmentofEmergency Medicine,Atlanta,Georgia

† EmoryUniversitySchoolofMedicine, Atlanta,Georgia

Submissionhistory:SubmittedApril21,2023;RevisionreceivedNovember17,2023;AcceptedNovember22,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18085

Background: Trainingprogramsforadvancedpracticeproviders(APP)oftenhavesignificantvariability intheircurriculum,includingelectrocardiogram(ECG)education.DespitelimitationsinformalECG training,APPsintheemergencydepartment(ED)maybethe firstpractitionertointerpretanECG. FoundationsofEmergencyMedicine(FoEM)offersfree,open-accesscurriculathatarewidelyused forresidenteducation.WesoughttoimproveAPPECGinterpretationskillsbyimplementingthe FoEMECGIcourse.

Methods: Thiswasasingle-site,pre-andpost-interventionstudyof23APPsatourhigh-acuity,urban ED.Inthefallof2020,APPlearnersenrolledinaFoEMECGIcourseledbyfacultyandseniorresident instructors.Thecourseconsistedofsixvirtual,small-group,active-learningECGworkshops. Participantscompleteda15-questionmultiple-choicetestbefore,immediatelyafter,andsixmonths post-interventiontoquantifyknowledgeacquisition.Additionally,apre-andpost-intervention knowledge,attitudes,andpracticessurveywasadministeredonECGinterpretationskillsandto evaluatethecourse.WeevaluatedchangeinECGknowledgescoresusingaWilcoxonsigned-ranktest. Changesinself-assessedknowledgewereevaluatedusinganordinallogisticmixed-effectsregression.

Results: Atotalof23APPsenrolledinthecourse.KnowledgeassessmentsshowedAPPsimproved fromthepre-test(median9/15,interquartilerange[IQR]7–11)tothepost-test(median12/15,IQR 10–13; P = 0.001).Testscoresdidnotsignificantlychangefromthepost-testtothedelayedpost-test (median12/15,IQR12–13; P = 0.30).Respondents’ subjectiveratingoftheirskilldidnotsignificantly change(P = 0.06).Respondentsreportednochangeintheirlikelihoodofapproachinganattending whenuncertainofthecorrectinterpretationofanECG(P = 0.16).Overall,91%weresatisfiedwiththe courseand96%agreedthatthecoursedifficultywasappropriate.

Conclusion: TheFoEMECGcourseprovidedastandardizedcurriculumthatimprovedAPPknowledge forinterpretingECGs.Despitethis,thecoursedidnotalterAPPs’ willingnesstoapproachphysiciansfor guidancewithinterpretationofabnormalECGs.These findingsmayinformexpansionofthisconceptfor otherprogramswhodesireformalizedAPPECGeducation.[WestJEmergMed.2024;25(2)155–159.]

INTRODUCTION

Advancedpracticeproviders(APP),comprisingphysician assistants(PAs)andnursepractitioners(NPs),havea substantialpresenceinemergencydepartments(ED)inthe

UnitedStates.Emergencydepartmentsemployed77%of APPsin2006,increasingfrom28%in1997.1 Thereareover 13,000PAsandover10,000NPscurrentlypracticinginthe acutecaresetting.2,3 ItisestimatedthatAPPssee21%ofall

EDvisitsandtheproportionofhighacuityservices independentlybilledbyAPPsisincreasing.4,5 Somemodels ofEDcare,suchaspractitioner-in-triage,oftenemploy APPsasthe firstpointofcontactforpatientsandaretasked withorderinginitialdiagnosticssuchas electrocardiograms(ECG).6,7

DespitethevolumeandacuityofpatientstreatedbyAPPs intheED,arelativelysmallproportionofAPPshave receivedformalizedpostgraduatetraininginemergency medicine(EM),with10%ofPAsand21%ofNPshaving completedsuchtraining.2,8 BoththeAmericanAcademyof EmergencyNursePractitioners(AAENP)9 andtheSociety ofEmergencyMedicinePhysicianAssistants(SEMPA)10 identifyECGinterpretationasarequisiteskillforAPPs practicinginEM.However,noconsistentapproachis appliednationwidetoaddressthislackof EM-specifictraining.11,12

FoundationsofEmergencyMedicine(FoEM)isafree, open-accesscurriculumthatiswidelyusedandvalidatedin EMresidenteducation. 13, 14 FoEMoffersstandardized, level-speci fi c,corecontentthatprimarilytargetsresident physiciansinEM.TheFoEMECGIcourseiscomposedof sixunitsthatreviewfundamentalconceptsinECG interpretationusinga fl ippedclassroomapproach ( Appendix1). 15 Implementationguidelines,unit summaries,challengeECGs,andinterpretationguidesare allfoundontheFoEMwebsite,whichmaybeaccessedby programleadersafterfreeregistration. 16 Wesoughtto addressagapintrainingandimproveAPPECG interpretationskillsbyimplementingtheFoEM ECGIcourse.

METHODS

StudyPopulationandDesign

WeincludedAPPsinthissingle-sitestudyiftheycurrently practicedatalarge,urban,countyhospitalandwereenrolled intheFoEMECGIcourseduringOctober2020-June2021. Whileenrollmentinthecoursewasrequiredtostaffhigher acuityEDzones,participationinthestudywasvoluntary. Participantsreviewedunitsummariesandpracticedselect ECGspriortoeachworkshop.Duringtheworkshop,APPs weredividedintosmallgroupstocollaborativelyreviewfour challengeECGswithinterpretationanddiscussionprompts. Uponcompletionofsmall-groupdiscussion,facultyor seniorresidentinstructorsfacilitatedinteractivesessionswith alllearners,reviewingcoreconceptsandECG challengeanswers.

Studyparticipantscompletedaknowledge,attitudes,and practices(KAP)surveyatthebeginningandcompletionof thecourse(Appendix2).Additionally,weobtainedobjective knowledgeacquisitionthrougha15-questionmultiplechoiceassessmentadministeredinOctober2020(preintervention),December2020(immediatepostintervention),andJune2021(delayedpost-intervention).

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Advancedpracticeproviders(APP)are responsibleforseeingasigni fi cantnumberof patientsinmanyEDsettings,yettheir in-trainingandpost-trainingcurricula arevariable.

Whatwastheresearchquestion?

WouldtheimplementationoftheFoundations ofEmergencyMedicine(FoEM)ECGI courseimproveelectrocardiogram interpretationskillsofAPPs?

Whatwasthemajor findingofthestudy?

Knowledgeassessmentsimprovedfromthe pre-test(median9/15,IQR7 – 11)totheposttest(median12/15,IQR10 – 13;P = 0.001).

Howdoesthisimprovepopulationhealth?

UseoftheFoEMECGIcurriculumforAPP learnersledtoanimprovementinECG knowledge,whilemaintainingattending physicianguidanceinthesetting ofuncertainty.

ThisstudywasdeemedexemptbytheInstitutionalReview BoardofEmoryUniversity.

StatisticalAnalysis

Wedescribedcategoricalvariablesusingfrequencies andpercentages.Continuousandscalevariableswere describedusingmediansandinterquartileranges(IQR). WeevaluatedthechangeinECGknowledgescores betweenthepre-test,post-test,anddelayedpost-testsessions usingtheFriedmanrepeated-measuresrank-order ANOVA.Ordinalself-assessmentvariableswereevaluated usingamixed-effectsordinallogisticregression.Weused mixedeffectstoaccountformultipleresponsesfrom individualstudyparticipants.Oddsratios(OR)and 95%confidenceintervals(CI)arepresentedfromthe regressions.Two-tailed P -values ≤0.05wereconsidered significant.WeconductedstatisticalanalysesusingRversion 4(RCoreTeam,FoundationforStatisticalComputing, Vienna,Austria).

RESULTS

Atotalof23APPsenrolled,withthemajorityidentifying asfemale(74%)withamedianageof37(IQR33–40)years

(Table1).Learnerswereprimarilyfamilynursepractitioner (FNP)(48%),followedbyphysicianassistrantPAs(26%) andFNP-emergencynursepractitionerENPs(22%).They reportedamedianof fiveyearsofpostgraduateexperiencein EM(IQR3–6),andasmallproportionreportedcompleting formalizedpostgraduatetraininginEM(13%).

Self-assessedconfidenceofECGinterpretationwas higherinthepost-testassessmentcomparedtothepre-test assessment;however,thedifferencewasnotsignificant (oddsratio[OR]2.94(95%confidenceintervalCI0.94–9.1), P = 0.06)(Figure1A).Incontrast,theobjective knowledgeassessments(Figure1B-D)indicatethatECG interpretationimproved(P < 0.001).Post-hoctestsindicated thatpost-testscores(median12/15,IQR10–13)were significantlygreaterthanpre-testscores(median9/15,IQR 7–11; P < 0.001).Delayedpost-testscores(median12/15, IQR12–13)didnotdifferfrompost-testscores(P = 0.30) indicatingthattheimprovedunderstandingwaslargely maintainedovertime.

OntheKAPsurvey,APPsreportedimprovedconfidence indetectinganST-segmentelevationmyocardialinfarction (STEMI)onECG(P = 0.01)(Appendix3).Nooverall changewasnotedinconfidencetointerpretalife-threatening arrhythmia(P = 0.27).Participantswerenomoreorless likelytoapproachanattendingphysicianforhelpregarding anuncertainECGbeforeandaftertheECGcurriculum (P = 0.16).Withrespecttoparticipants’ viewofthecourse, 21(91%)reportedbeingsatisfiedorhighlysatisfied;9(39%) satisfied;and12(52%)highlysatisfied.Onlyoneparticipant (4%)wasneutralandone(4%)wasunsatisfied.Twenty-two

participants(96%)believedthatthecoursewastaughtatthe correctlevelofdifficulty:13(57%)stronglyagreed;and9 (39%)agreed.Onlyoneparticipant(4%)wasneutral.No respondentdisagreed.

DISCUSSION

TheFoEMECGIcurriculumwasadministeredtoAPP learnersandevaluatedusingapre-andpost-intervention, self-reportedKAPsurveyinconjunctionwithanobjective measureofknowledgeacquisition.Therewasan improvementinobjectiveknowledgeandretention,atrend towardimprovedconfidenceinECGinterpretation,anda significantimprovementinSTEMIidentification.Despite theseimprovements,therewasnochangeintheAPPs’ likelihoodofreachingouttophysiciansforassistancewith ECGinterpretation.

CharacteristicValueN = 23

Age,median(IQR)*37(33–40)

Gender,n(%)

Female17(74%)

Male6(26%)

Certi

*IQR,interquartilerange;**AGNP,adultgerontologynurse practitioner; αFNP,familynursepractitioner; βFNP-ENP,familynurse practitioner-emergencynursepractitioner; ¥PA,physicianassistant.

Auniqueadvantageofourstudyisitsdemonstrationof measurableimprovementinclinicallyrelevantECG interpretation.Whilethisisencouragingwithrespecttothe easeandefficiencyofthecourse,whatismoreimpressiveis theretentionofknowledgeovertime.TheAPPscommonly workintriageandloweracuityareasandareoftenthe first practitionerstoevaluatepatientsintheED.6,7 Todetect manylife-threateningillnesses,promptECGacquisitionand interpretationisessential.Thus,APPsshouldbeableto interpretECGswhenaphysicianisnotimmediately available,asmaybethecaseinsomepractices.17 Despitethe importanceofECGinterpretation,APPsoften findthis clinicalskillchallenging,withonestudydemonstrating50% proficiencyofECGinterpretationamonggraduatingPAs,a metriccommensuratewiththebaselinecompetency demonstratedinourcohort.18 Inthisstudy,wewereableto increaseandmaintaincompetencyat80%,underscoringthe valueaddedbytheFoEMECGIcourse.Despitethis increasedknowledge,APPswerejustaslikelytoreachoutto aphysicianforECGinterpretationguidance,showingthat thecoursedidnotdecreaserelianceonphysicianknowledge andjudgment.

OurstudydemonstratedthatAPPs,despiteshowing increasedknowledgefollowingthecourse,onlygained confidenceinidentifyingSTEMIsonECG,butdidnot improveconfidenceinotherdomainsofECGinterpretation followingtheFoEMECGIcourse.Thisrepresentsa mismatchinperceivedandactualECGinterpretation competence.Thismayreflectlimitedindividualtimespenton eachmoduleand/ormodulesfocusedonmorespecificECG pathologies.Furtherstudiesmayevaluatewhether confidencemaybeimprovedwithcontinuedtrainingand exposuretomorediverseECG findings.

Whileimplementingthecourseandshowingitwas effectivefromaknowledge,attitudes,andbehaviors standpointisofparamountimportance,aneducational programmustalsobewell-receivedbythelearner.Ourstudy demonstratedveryhighlevelsofsatisfactionwiththecourse

Table1. Demographiccharacteristicsofadvancedpractice providersenrolledintheFoundationsofEmergencyMedicineECGI course,October2020–June2021.

Figure1. Advancedpracticeproviderperceivedandobjectiveknowledgeacquisitionbeforeandafterfoundationsofemergencymedicine (FoEM)ElectrocardiogramGram(ECG)Icourse.A)Self-assessmentofrespondents’ understandingofECGinterpretationasafunctionof time.B)Boxplotdepictingknowledgetestscoresasafunctionoftime.C)Waterfallplotdepictingthechangeinknowledgetestscoresfrom pre-testtopost-testforeachindividualrespondent.D)Waterfallplotdepictingthechangeinknowledgetestscoresfrompost-testtodelayed post-testforeachindividualrespondent.

amongourAPPs,alongwithAPPsreportingthatthe conceptstaughtwereappropriatefortheirleveloftraining. Thistranslatesintomoreengagementandknowledge acquisitionandretentioninthecurriculum,in fittingwithpriorstudieslookingatAPPsand case-basededucation.19

LIMITATIONS

Ourstudylimitationsincludedasmallsamplesizewitha relativelyhomogeneousstudypopulation(eg,primarilyNPs, allpracticingatasinglecountyhospital).Additionally,our studycohortdidnotreflecttheroughly50/50distributionof NPsandPAspracticinginacutecaresettings,withour grouponlyhaving26%PArepresentation.Finally, ourstudydidnotincludeacontrolgroupwhichdidnot receivetrainingtherebyallowingforthepossibilityoftest/ retesteffects.

CONCLUSION

FormalizedpostgraduateECGinterpretationtrainingfor APPsinEMisatbestinconsistent,yetbothSEMPAand AAENPlistECGinterpretationasanecessaryskillfor practicinginEM.9,10 Inresponsetothis,weimplementedthe FoEMECGIcourseandfoundthatitwaseasyto implement,ledtoimprovedECGknowledgeandconfidence inECGinterpretation,andwaswellreceivedbytheAPP group.Theseresultsmayinformtheuseofthisfree, structuredECGcurriculumatbothacademicand community-basedprogramsthatsupportcontinuing educationforAPPs.Futurestudiesshouldinvestigatethe impactofincreasedsamplesizes,morevariablepractice locationsanddepartmentaldesigns,andahigherproportion ofPAs,allofwhichwouldservetomakethedatamore reflectiveoftheAPPpopulationasawholeand,therefore, moregeneralizable.

AddressforCorrespondence:StevenLindsey,MD,FACEP, FACEM,EmoryUniversity,DepartmentofEmergencyMedicine, 531AsburyCircle,AnnexBuilding,SuiteN340,Atlanta,GA30322. Email: steven.m.lindsey@emory.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Lindseyetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.MenchineMD,WiechmannW,RudkinS.Trendsinmidlevelprovider utilizationinemergencydepartmentsfrom1997to2006. AcadEmerg Med.Oct2009;16(10):963–9.

2.StatisticalProfileofCertifiedPAsAnnualReport.2021:National CommissiononCertificationofPhysicianAssistantswebsite.2022. Availableat: https://www.nccpa.net/wp-content/uploads/2022/08/ 2021StatProfileofCertifiedPAs-A-3.2.pdf.AccessedApril12,2023.

3.NPFactSheet.AmericanAcademyofNursePractitionerswebsite. 2022.Availableat: https://www.aanp.org/about/all-about-nps/ np-fact-sheet.AccessedApril12,2023.

4.WuFandDarracqMA.Comparingphysicianassistantandnurse practitionerpracticeinU.S.emergencydepartments,2010-2017. West JEmergMed.2021;22(5):1150–5.

5.GettelCJ,SchuurJD,MullenJB,etal.Risinghigh-acuityemergency careservicesindependentlybilledbyadvancedpracticeproviders,2013 to2019. AcadEmergMed.2023;30(2):89–98.

6.MoretzJ.Emergencydepartmentcaremodels.BerkeleyResearch Groupwebsite.2018.Availableat: https://www.thinkbrg.com/insights/ publications/emergency-department-care-models AccessedApril11,2023.

7.WylieK,CrillyJ,TolooGS,etal.Reviewarticle:emergencydepartment modelsofcareinthecontextofcarequalityandcost:asystematic review. EmergMedAustralas.2015;27(2):95–101.

8.TheStateoftheNursePractitionerProfession.AmericanAcademyof NursePractitionerswebsite.2021.Availableat: https://www.aanp.org AccessedApril12,2023.

9.PracticeStandardsfortheEmergencyNursePractitionerSpecialty. AmericanAcademyofEmergencyNursePractitionerswebsite.2018. Availableat: https://www.aaenp-natl.org/assets/docs/ practice_standards_for_the_emergency_nurse_practitioner.pdf AccessedApril11,2023.

10.EmergencyMedicinePhysicianAssistantPracticeGuidelines.Society ofEmergencyMedicinePhysicianAssistants.2021.Availableat: https:// www.sempa.org/about-sempa/guidelines-and-statements/ empa-practice-guidelines.AccessedApril12,2023.

11.ChekijianSA,EliaTR,MontiJE,etal.Integrationofadvancedpractice providersinacademicemergencydepartments:bestpracticesand considerations. AEMEducTrain.2018;2(Suppl1):S48–55.

12.KrausCK,CarlisleTE,CarneyDM.Emergencymedicinephysician assistant(EMPA)postgraduatetrainingprograms:program characteristicsandtrainingcurricula. WestJEmergMed 2018;19(5):803–7.

13.BurnsWP,HartmanND,WeygandtPL,etal.Criticalelectrocardiogram curriculum:settingthestandardfor flipped-classroomEKGinstruction. WestJEmergMed.2019;21(1):52–7.

14.GrabowMooreK,KettererA,WheatonN,etal.Development, implementation,andevaluationofanopenaccess,level-specific,core contentcurriculumforemergencymedicineresidents. JGradMed Educ.2021;13(5):699–710.

15.EKGcourses.FoundationsofEmergencyMedicinewebsite.Available at: https://foundationsem.com/ekg-courses.AccessedMarch29,2023.

16.FoundationsofEmergencyMedicine.Availableat: https:// foundationsem.com.AccessedNovember6,2022.

17.KeoughVA,TellD,AndreoniC,etal.Uniqueeducationalneedsof emergencynursepractitioners. AdvEmergNursJ.2016;38(4):300–7.

18.LundbergEL,Stagnaro-GreenA,ZhangY,etal.Assessmentof competencyinelectrocardiograminterpretationofgraduatingphysician assistantstudents. JPhysicianAssistEduc.2013;24(4):15–8.

19.BreunigMJ,HerberAJ,JenkinsSM,etal.Characteristicsofeffective continuingmedicaleducationforphysicianassistantsandnurse practitionersinhospitalmedicine. JPhysicianAssistEduc 2020;31(1):2–7.

ORIGINAL RESEARCH

StageBHeartFailureIsUbiquitousinEmergencyPatientswith AsymptomaticHypertension

KimberlySouffront,PhD,RN,FNP-BC,FAAN*†‡

BretP.Nelson,MD,RDMS,FACEP*

MeganLukas,RDMS,MBA*

HansReyesGaray,MD,MPH§

LaurenGordon,MPH,CHES*‡∥

ThaliaMatos,BA*

IsabellaHanesworth,BS*

RebeccaMantel,BA*

ClaireShubeck,BA*

CassidyBernstein,BA†

GeorgeT.Loo,DrPH,MPH,MPA,CHPCP*∥

LynneD.Richardson,MD,FACEP*‡∥¶

SectionEditor:FredSeveryn,MD

*IcahnSchoolofMedicineatMountSinai,Departmentof EmergencyMedicine,NewYork

† CenterforNursingResearchandInnovation,MountSinai HealthSystem,NewYork

‡ IchanSchoolofMedicineatMountSinai,InstituteforHealth EquityResearch,NewYork

§ MountSinaiMorningside,DepartmentofCardiology,NewYork

∥ IcahnSchoolofMedicineatMountSinai,Departmentof PopulationHealthScienceandPolicy,NewYork

¶ IcahnSchoolofMedicineatMountSinai,Departmentof ArtificialIntelligenceandHumanHealth,NewYork

Submissionhistory:SubmittedMarch23,2023;RevisionreceivedNovember21,2023;AcceptedDecember19,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17990

Introduction: Hypertensionistheleadingriskfactorformorbidityandmortalitythroughouttheworldand ispervasiveinUnitedStatesemergencydepartments(ED).Thisstudydocumentsthepointprevalence ofsubclinicalheartdiseaseinemergencypatientswithasymptomatichypertension.

Method: ThiswasaprospectiveobservationalstudyofEDpatientswithasymptomatichypertension conductedattwourbanacademicEDsthatbelongtoaneight-hospitalhealthcareorganizationin NewYork.Adult(≥18yearsofage)English-orSpanish-speakingpatientswhohadaninitialblood pressure(BP) ≥160/100millimetersofmercury(mmHg)andsecondBP ≥140/90mmHg,andpending discharge,wereinvitedtoparticipateinthestudy.Weexcludedpatientswithcongestiveheartfailure, renalinsufficiency,andatrial fibrillation,orwhowerepregnant,aprisoner,cognitivelyunableto provideinformedconsent,orexperiencingsymptomsofhypertension.Weassessedechocardiographic evidenceofsubclinicalheartdisease(leftventricularhypertrophy,anddiastolicand systolicdysfunction).

Results: Atotalof53patientswereincludedinthestudy;amajoritywereyoung(mean49.5yearsold, [SD14–52]),self-identifiedasBlackorOther(n = 39;73.5%),andfemale(n = 30;56.6%).Meaninitial bloodpressurewas172/100mmHg,and24patients(45.3%)self-reportedahistoryofhypertension. Fiftypatientscompletedanechocardiogram.All(100%)hadevidenceofsubclinicalheartdisease, with41(77.4%)displayingleftventricularhypertrophyand31(58.5%)diastolicdysfunction.There wasasignificantrelationshipbetweendiastolicdysfunctionandfemalegender[x2 (1,n = 53) = 3.98; P = 0.046];Blackorotherrace[x2 (3,n = 53) = 9.138; P = 0.03]andHispanicorotherethnicity [x2 (2,n = 53) = 8.03; P = 0.02].Lessthanonethirdofpatientsdemonstratedsystolic dysfunctiononechocardiogram,andthiswasmorelikelytooccurinpatientswithdiabetesmellitus [x2 (1,n = 51) = 4.84; P = 0.02].

Conclusion: ThereisahighprobabilitythatBlack,Hispanic,andfemalepatientswithasymptomatic hypertensionareonthecontinuumfordevelopingovertheartfailure.Emergencycliniciansshould provideindividualizedcarethatconsiderstheiruniquehealthneeds,culturalbackgrounds,andsocial determinantsofhealth.[WestJEmergMed.2024;25(2)160–165.]

INTRODUCTION

Hypertensionistheleadingriskfactorformorbidityand mortalitythroughouttheworldandispervasiveinUnited Statesemergencydepartments(ED).1,2 Itisacommon misconceptionthathighbloodpressurefoundduringtheED visitisrelatedtopainoranxiety;however,studiesshowthat elevatedbloodpressureintheEDreliablyidentifies hypertensioninanoverwhelmingmajorityofpatients3 andis anindependentriskfactorforhavingamajorcardiovascular eventafterdischarge.4

TheclinicalpolicyrecommendedbytheAmericanCollege ofEmergencyPhysicians(ACEP)forpatientswhohave persistentasymptomatichypertensionintheED without signsandsymptomsofacutetargetorganinjuryincludes promptreferraltoprimarycare.5 However,thisclinical policyisoutdatedandinconsistentlyadheredto.6

Furthermore,ofthepatientswhoarereferred,lessthanhalf adheretorecommendationsregardingfollow-up,regardless oftheirinsurancestatusoraccesstoroutinemedicalcare.7

Automatingrecommendationsforfollow-upusingthe electronichealthrecordisonewaytoensureadherencetothe ACEPpolicy;8 however,thisisnotusefulormeaningfulto patientswhenitisnotfollowedbyanyclinician-to-patient communicationabouttheimportanceoffollow-up.Besides, thispolicydoesnotconsidertheemergingevidencethatan overwhelmingmajorityofemergencypatients,particularly Blacks,withasymptomatichypertensionshowevidenceof subclinicalheartdisease,9,10 withupperlimitsreachingmore than90.7%inonesentinelstudybyLevyetal.9

Thisstudydocumentsthepointprevalenceofsubclinical heartdiseaseinthe first50EDpatientsenrolledinan ongoingstudy.Thiswasour firststeptoexploreuniqueways forimprovinghealthcaredeliveryforthishigh-riskpatient populationinourED.

METHODS

StudyDesign,Sample,andSetting

Thiswasaprospectiveobservationalstudyofemergency patientswithasymptomatichypertension.Datacollection beganafterreceivinginstitutionalreviewboardapproval (#18-00197).Thestudywasconductedattwourban academicEDsthatprovidecaretoadiversepatient population,servingthelocalcommunitiesofHarlem, NewYorkCity.Harlemhasaconcentratedburdenof hypertensionandisamongthepoorestneighborhoodsin NewYorkCity.Morethantwo-thirdsoftheresidentsarea racialorethnicminority.11

Datacollectionbeganin2018atoneEDsiteandstopped twicedueto1)lackoffundingtocontinueparticipant recruitment,and2)apauseinresearchactivitiesatthe recruitinginstitutioninresponsetotheCOVID-19 pandemic.Inlate2021,anadditionalsitewasaddedtoboost patientrecruitment.Annually,thereareabout100,000 patientvisitsineachED,andthereisastate-of-the-art

electronicinformationsystemtofacilitatedatacollection acrosssites.Theestimatedprevalenceofpatientswhovisit theEDateachsitewhohaveuncontrolledasymptomatic hypertensionis48%–50%.12

Adult(≥18yearsofage)English-orSpanish-speaking patients,whohadaninitialbloodpressure(BP) ≥160/ 100millimetersofmercury(mmHg)andsecond BP ≥140/90mmHgandwerependingdischarge,were invitedtoparticipateinthestudy.Weexcludedpatientswith congestiveheartfailure,renalinsufficiency,andatrial fibrillation,orwhowerepregnant,aprisoner,cognitively unabletoprovideinformedconsent,orexperiencing symptomsofhypertension,suchaschestpain,paresthesia,or shortnessofbreath.

ProtocolChange

Initiallywealsoexcludedpatientsiftheyhadtakenblood pressuremedicationwithinthepriorthreemonths;however, beginningatstudyparticipantnumber44,wemodifiedour protocoltoincludeallpatientsregardlessofwhetherthey hadtakenmedicationwithinthepreviousthreemonths.We revisedthestudyprotocoltobemoreinclusive,specificallyto avoidexcludingpatientswhomighthaveuntreated conditionsandtoensurethatallindividuals,includingthose potentiallyinneedofintervention,couldbeconsideredinour research.Thischangeinprotocolallowedforamore comprehensiveassessmentofthestudypopulationandalso assistedwithourrecruitment.

DataCollectionProcedure

Approximatelythreedays/weekfrom8 AM to4 PM,a researchcoordinatorenrolledandobtainedinformed consentelectronicallyfrompatientswhometinclusion criteria.Studydatawerecollectedandmanagedusing REDCap(ResearchElectronicDataCapture)13,14 electronicdatacapturetoolshostedatourinstitution.1,2 REDCapisasecure,web-basedsoftwareplatformdesigned tosupportdatacaptureforresearchstudies,providing1)an intuitiveinterfaceforvalidateddatacapture;2)audittrails fortrackingdatamanipulationandexportprocedures; 3)automatedexportproceduresforseamlessdatadownloads tocommonstatisticalpackages;and4)proceduresfordata integrationandinteroperabilitywithexternalsources.13,14

Eachmorningofpatientrecruitment,theresearch coordinatorusedMicrosoftTeams(MicrosoftCorp, Redmond,WA)toinformthesonographerthatrecruitment hadbegun.15 BeforeapproachinganypatientintheED,the researchcoordinatorusedtheEpicchatfeature(Epic SystemsCorp,Verona,WI)toinformtheEDattending physicianthathis/herpatientmetinclusionforthestudyand wouldbeapproached.15 Ifanelectrocardiogram(ECG)was notconductedaspartofusualcare,theECGwasconducted bytheresearchcoordinator.Thebedsideechocardiogram wasconductedbyonememberofthestudyultrasoundteam,

whichincludedoneEDsonographer,fouremergency physicians,andonecardiologist,eachtrainedonthestudy protocol.Eachechocardiogramtookapproximately 10minutestocompleteatbedside.

Measures

DemographicMeasures

Studymeasureswereobtainedbyelectronicdata extractionandincludedthepatient’ s firstandsecondsystolic anddiastolicBPlevelsincearrivaltotheED,age,gender, race/ethnicity,pastmedicalhistory,smokinghistory,and accesstoprimarycare.Alldataweredoubleenteredintothe REDCapdatabasebytwomembersoftheresearchteamto assureaccuracyofchartreviewanddataentry.Ourprotocol wassetsuchthatanydifferenceswerereconciledbythe principalinvestigator(KS);however,nodifferences wereidentified.

Echocardiogram

Threeendpointswereobtainedandmodeledafterthe AmericanCollegeofCardiology,AmericanHeart Association(ACC/AHA),HeartFailureSocietyofAmerica, andLevyetal.9,20:

1. Leftventricularhypertrophy(LVH) .Definedas presentifleftventricularseptalthicknesswas ≥ 1.1 centimeter(cm)orabsentifleftventricularseptal thicknesswas <1.1cm.(Leftventricularfreewall thicknesswasmeasuredintheparasternallongview atenddiastole.)

2. Systolicdysfunction. Ejectionfractionnotedas normal(≥55%)orabnormal(<55%).

3. Diastolicfunction. Evaluatedbyestimatingleftatrial pressureusingtheE/e’ ratio.Diastolicdysfunctionwas presentifE/e’(septal)was ≥15,ifE/e’(lateral)was ≥12, ifseptale’ was <8cm/sec,oriflaterale’ was <10cm/sec.3 Gradeofdiastolicdysfunctionwasnot determinedforthisinitialanalysis.

Electrocardiogram

ThestudydiagnosedLVHwithECG findingsthat satisfiedtheCornellvoltagecriteria(whenthesumoftheR waveinleadaVLandtheSwaveinleadV3is >20mmin womenand23mminmen).DataobtainedfromtheECG includedheartrate,PRinterval,andtheQRS intervalnumber.

DataAnalysisPlan

WeexporteddatafromREDCaptoSASanalytic softwareversion9.4(SASInc,Cary,NC)fordataanalysis. Demographicvariablesarepresentedaspercentages(%)or means(M )andstandarddeviations.Evidenceofsubclinical heartdiseasewasdichotomizedasabnormal/normaland

presentedaspercentages.Weconductedbivariateanalysesto testforsignificantrelationshipsbetweenindependent variables(age,gender,race/ethnicity,pastmedicalhistory, bodymassindex,smokinghistory,andaccesstoprimary care)andsubclinicalheartdisease.

RESULTS

CharacteristicsofSample

Atotalof53patientswereexamined.(Twopatientsdid notcompleteanechocardiogramandonepatientdidnot completeanechocardiogramandECGduetotime constraints.)Themajorityofthesamplewereyoung (mean49.5yearsold,[SD14–52]),self-identifiedasBlackor Other(n = 39;73.5%),andfemale(n = 30;56.6%). 31wereHispanicrace(58.5%)and30female(56.6%).Mean initialBPwas172/100mmHg,and24patients(45.3%) self-reportedahistoryofhypertension;36(68%)hadtaken theirantihypertensivemedicationwithinthepriorthree months.Meanbodymassindexwas31.9,whichcorrelatesto ObesityClassIormoderatelyobese.17 Fifteenpatients (28.3%)werecurrentsmokers,and12(28.3%)]hadahistory ofdiabetesmellitus.Thirty-one(58.5%)reportedtheyhada primarycarephysician(Table1).

Echocardiogram

Ofthe50patientswhocompletedanentire echocardiogram,100%hadevidenceofsubclinicalheart disease;41(77.36%)showedevidenceofLVHand31 (58.49%)diastolicdysfunction.Weperformedachi-square testofindependencetoassesstherelationshipbetween subclinicalheartdiseaseandourindependentvariables. Therewasasignificantrelationshipbetweendiastolic dysfunctionandfemalegender[x2 (1,n = 53) = 3.98; P = 0.046];Blackorotherrace[x2 (3,n = 53) = 9.138; P = 0.03] andHispanic[x2 (2,n = 53) = 8.03; P = 0.02].Lessthanonethirdofpatientsdemonstratedsystolicdysfunctionon echocardiogram,andthiswasmorelikelytooccurinpatients withdiabetesmellitus[x2 (1,n = 51) = 4.84; P = 0.02].There werenootherdifferencesbetweenourindependentvariables andourmainoutcome,subclinicalheartdisease.

OneparticipantwasmissinganECG.Of49participants, fivedemonstratedevidenceofLVH.

DISCUSSION

Wefoundthatsubclinicalheartdiseaseisubiquitousin EDpatientswithasymptomatichypertension.Our findings areconsistentwiththoseofLevyetal(2012),whowerethe firsttodocumentthealarmingprevalenceofstructuralheart changesinEDpatientswithasymptomatichypertension, particularlyamongBlackswhosoughtcareintheEDofan innercity.9 Ourstudyaddstotheexistingliteraturethat BlackandHispanicpatientswhohaveasymptomatic hypertensiondiagnosedintheEDhaveahighprobabilityof havingmyocardialtargetorgandamage,thatisconsistent

MeasureVariableStatisticN = 53(%)

Independentvariables – demographiccharacteristicsAge,mean(SD)49.47(14.5)

Gender,n(%)

Female30(56.6)

Male23(43.4)

Race,n(%)

White5(9.4)

Blackorother48(90.6)

Ethnicity,n(%)

Non-Hispanic27(49.1)

Hispanicorother31(58.5)

Bloodpressure,meanmmHg

1st systolicbloodpressure172.13(15.1)

1st diastolicbloodpressure100.74(15.0)

2nd systolicbloodpressure162.27(17.5)

2nd diastolicbloodpressure95.37(16.5)

Hypertensionhistory,yes24(45.3%)

Tookbloodpressuremedicinewithin3months36(68.0%)

Smoker,yes15(28.3%)

Diabetesmellitushistory,yes12(22.6%)

Bodymassindex,mean(SD)31.93(8.9)

Primarycarephysician,yes31(58.5)

Dependentvariable – subclinicalheartdiseaseSubclinicalheartdisease,yes50(100)

Diastolicdysfunction,yes31(58.5)

Leftventricularhypertrophy,yes41(77.4)

Systolicdysfunction,yes10(19.6)

withadiagnosisofStageBheartfailureaccordingto theACC/AHA.18

Congestiveheartfailureisaprogressivedisorder(Stages A,B,C,andD)thatoftenbeginswithleftventricularsystolic dysfunctionandresultsinsymptomsfrom fluidoverloadand poorend-organperfusion.18 StageBheartfailure encompassespatientswhoareasymptomaticandhave evidenceofstructuralheartabnormalities,suchasLVH, systolicordiastolicdysfunction.18 Asymptomaticleft ventricularsystolicdysfunctionhasanestimatedprevalence of3–6%inthecommunity,19 whichissignificantlylessthan ourstudy findingsof19%.Prendergastetal10 (2015)found thepointprevalenceofsubclinicalheartdiseaseinemergency patientswithasymptomatichypertensiontobenearly40%, alsosignificantlylessthanour findingsof100%.Toour knowledge,nootherstudyexiststhatdocumentsthepoint prevalenceofsubclinicalheartdiseaseinEDpatientswith asymptomatichypertension.BothLevyetal(2012)and Prendergastetal(2015)conductedtheirstudiesininner-city

Detroitandinner-cityChicago,respectively settings similartoours.9,10

Inrandomizedtrials,individualswithasymptomaticleft ventricularsystolicdysfunctionhavehighratesofincident heartfailureanddeath,increasedcardiovascular mortality19–21;all-causemortality20;andnonfatal cardiovascularevents,suchasmyocardialinfarctionand stroke.20,22 Whiletherelativecontributionofeachcondition (systolicanddiastolicdysfunction,orLVH)toovert congestiveheartfailureisunknown,fromthelensof emergencymedicineandpopulationhealth,identifying patientswithasymptomatichypertensionisofutmost importance,regardless.

OurstudydemonstratesthatEDpatientswith asymptomatichypertensionareonthecontinuumtoovert heartfailure,whichhasseriousconsequences.Aconsiderable numberofmissedopportunitiesoccur,whichcontributeto delaysindiagnosis,suboptimaltreatment,increased morbidityandmortalityand,aboveall,anexacerbationof

Table1. Demographiccharacteristicsandsubclinicalheartdisease.

Souffrontetal. StageBHeartFailureinPatientswithAsymptomaticHypertension

disparitiesrelatedtohypertensionamongBlacks.Studies suggestreferralratesare5%.12 Despitethemanyknowledge, attitudinal,andorganizationalbarriersthatexist,6,23,24 theidentificationandmanagementofasymptomatic hypertensioniscriticaltopreventtheprogression ofthediseaseanditscomplications.

LIMITATONS

Theresultsofourstudymustbeinterpretedconsideringits limitations,whichincludeitssmallsamplesizeandstudy design.Weanalyzedonlythe first50patientsofanongoing study,whichlimitstheabilitytogeneralizetolarger populations.Additionally,ourprospectiveobservational studydesignlimitstheabilitytodrawconclusionsabout theprogressionofcongestiveheartfailureorother cardiovasculareventsovertime.Lastly,wedidnotevaluate subclinicalheartdiseaseinnormotensivepatients. Nonetheless,thisstudyhasstrengthsinthatitdocumentsthe significantburdenofStageBheartfailureintheemergency populationwithasymptomatichypertension.

IMPLICATIONS

Emergencycliniciansshouldrecognizetheprevalenceof asymptomatichypertensioninBlackpatientsandprovide individualizedcarethatconsiderstheiruniquehealthneeds, culturalbackgrounds,andsocialdeterminantsofhealth. Thisincludesprovidingadequatefollow-upcareandsupport topreventhypertension-relatedcomplicationsandimprove overallhealthoutcomes.

CONCLUSION

Inthisstudywedocumentedthepointprevalenceof subclinicalheartdiseaseinthe first50EDpatientswhowere enrolledinanongoingstudy,asour firststeptoexplore uniquewaysforimprovinghealthcaredeliveryforthishighriskpatientpopulationinouremergencydepartment.While subclinicalheartdiseasewasubiquitous,weconcludethat female,Black,andHispanicpatientswhohave asymptomatichypertensiondiagnosedintheEDhaveahigh probabilityofalreadybeingonthecontinuumfordeveloping overtheartfailure.

ACKNOWLEDGMENTS

Weexpressoursincereappreciationtotheancillarystaff, staffnurses,andmedicalstaffoftheMountSinaiEmergency Departmentfortheirinvaluablecontributionstothisstudy. Ourgratitudeextendstothepatientswhoseparticipation madethisresearchpossible.SpecialthankstotheSouffront Collaboratorymembers,includingnursingleadership AshleyCacerasMSN,RN,LucioBarettoDNP,AGACNPBC,APRN-BC,RN,CCRN,NE-BC,CPXP,CLSSBB, MICN,andOliviaSgambelloneBSN,RN,CENwhose guidanceandsupportenrichedthecollaborativenatureof thisstudy.

AddressforCorrespondence:KimberlySouffrontPhD,IcahnSchool ofMedicineatMountSinai,DepartmentofEmergencyMedicine, 555West57th St.,5th Floor,Room5-33,NewYork,NY10019.

Email: Kimberly.souffront@mountsinai.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Thisprojectwasfundedbythe DepartmentofEmergencyMedicineattheIcahnSchoolofMedicine atMountSinai,theCenterforNursingResearchandInnovationat theMountSinaiHealthSystem,andtheGordonandBettyMoore FoundationthroughtheBettyIreneMooreFellowshipforNurse LeadersandInnovators(GBMF9048).Therearenootherconflictsof interestorsourcesoffundingtodeclare.

Copyright:©2024Souffrontetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.CenterforDiseaseControl.Factsabouthypertension.2023. Availableat: https://www.cdc.gov/bloodpressure/facts.htm AccessedFebruary1,2023.

2.McNaughtonCD,SelfWH,ZhuY,etal.IncidenceofhypertensionrelatedemergencydepartmentvisitsintheUnitedStates,2006to2012. AmJCardiol. 2015;116(11):1717–23.

3.GoldbergEM,WilsonT,SaucierC,etal.AchievingtheBpTRUth: emergencydepartmenthypertensionscreeningandtheCentersfor Medicare&MedicaidServicesqualitymeasure. JAmSocHypertens. 2017;11(5):290–4.

4.OrasP,HäbelH,SkoglundPH,etal.Elevatedbloodpressureinthe emergencydepartment:ariskfactorforincidentcardiovasculardisease. Hypertension. 2020;75(1):229–36.

5.WolfSJ,LoB,ShihRD,etal.AmericanCollegeofEmergency PhysiciansClinicalPoliciesCommittee.Clinicalpolicy:criticalissuesin theevaluationandmanagementofadultpatientsintheemergency departmentwithasymptomaticelevatedbloodpressure. AnnEmerg Med. 2013;62(1):59–68.

6.SouffrontK,ChyunD,KovnerC.Barrierstoreferralforelevatedblood pressureintheemergencydepartmentanddifferencesbetween providertype. JClinHypertens(Greenwich). 2015;17(3):207–14.

7.CornellSD,ValerioMA,KrauseT,etal.Lowadherencetopost emergencydepartmentfollow-upamonghypertensivepatientswith medicalinsurance. JEmergMed. 2020;58(2):348–55.

8.SouffrontK,RiveraL,LooG,etal.Integratingaclinicaldecisionsupport remindertoimprovebloodpressurereassessmentforpatientswith uncontrolledhypertension. ClinCardiolCardiovascInterv. 2019;2:01–3.

9.LevyP,YeH,ComptonS,etal.Subclinicalhypertensiveheartdisease inblackpatientswithelevatedbloodpressureinaninner-city emergencydepartment. AnnEmergMed. 2012;60(4):467–74.e1.

10.PrendergastHM,CollaJ,PatelN,etal.Correlationbetweensubclinical heartdiseaseandcardiovascularriskprofilesinanurbanemergency departmentpopulationwithelevatedbloodpressures:apilotstudy. JEmergMed. 2015;48(6):756–61.

11.FurmanCenterforRealEstateandUrbanPolicy.Harlemstatistics. Availableat: https://furmancenter.org/neighborhoods/view/ central-harlem.AccessedFebruary1,2023.

12.SouffrontK,GestalC,MelkusGD,etal.Recognitionofasymptomatic hypertensioninanurbanemergencydepartmentwherearewenow? AdvEmergNursJ. 2016;38(4):320–6.

13.HarrisPA,TaylorR,ThielkeR,etal.Researchelectronicdatacapture (REDCap)–ametadata-drivenmethodologyandworkflowprocessfor providingtranslationalresearchinformaticssupport. JBiomedInform. 2009;42(2):377–81.

14.HarrisPA,TaylorR,MinorBL,etal.TheREDCapconsortium:building aninternationalcommunityofsoftwareplatformpartners. JBiomed Inform. 2019;95:103208.

15.EPICResearch.Availableat: https://www.epicresearch.org/ AccessedFebruary1,2023.

16.CenterforDiseaseControl.DefiningAdultOverweight&Obesity.2022. Availableat: https://www.cdc.gov/obesity/basics/adult-defining.html AccessedFebruary1,2023.

17.GoldbergLRandJessupM.StageBheartfailure:managementof asymptomaticleftventricularsystolicdysfunction. Circulation. 2006;113(24):2851–60.

18.WangTJ,LevyD,BenjaminEJ,etal.Theepidemiologyof “asymptomatic” leftventricularsystolicdysfunction:implicationsfor screening. AnnInternMed. 2003;138(11):907–16.

19.SaraJD,ToyaT,TaherR,etal.Asymptomaticleftventriclesystolic dysfunction. EurCardiol. 2020;15:e13.

20.AmmarKA,JacobsenSJ,MahoneyDW,etal.Prevalenceand prognosticsignificanceofheartfailurestages:applicationofthe AmericanCollegeofCardiology/AmericanHeartAssociationheart failurestagingcriteriainthecommunity. Circulation. 2007;115(12):1563–70.

21.Echouffo-TcheuguiJB,ErqouS,ButlerJ,etal.Assessingtheriskof progressionfromasymptomaticleftventriculardysfunctiontoovertheart failure:asystematicoverviewandmeta-analysis. JACCHeartFail. 2016;4(4):237–48.

22.SouffrontK,ShastryS,BennettC,etal.Emergencynursingpolicyand hypertensionawareness:anintegrativereviewoftheliterature. CurrHypertensRep. 2019;21(7):55.

23.SouffrontK,ChyunD,KovnerC,etal.Barriersandfacilitators toreferralforasymptomatichypertensionintheemergencydepartment: amultidisciplinarysurveyofEDproviders. JHypertensManag. 2016;2(1).

24.BrodyA,TwinerM,KumarA,etal.Surveyofemergencyphysician approachestomanagementofasymptomatichypertension. JClin Hypertens(Greenwich).2017;19(3):265–9.

REVIEW

PerformanceofIntra-arrestEchocardiography: ASystematicReview

Yi-JuHo,MD*

Chih-WeiSung,MD*†

Yi-ChuChen,PhD‡

Wan-ChingLien,MD,PhD*§

Wei-TienChang,MD,PhD*§

Chien-HuaHuang,MD,PhD*§

*NationalTaiwanUniversityHospital,DepartmentofEmergencyMedicine, Taipei,Taiwan

† NationalTaiwanUniversityHsin-ChuHospital,DepartmentofEmergency Medicine,Hsinchu,Taiwan

‡ NationalTaiwanUniversity,InstituteofEpidemiologyandPreventiveMedicine, CollegeofPublicHealth,Taipei,Taiwan

§ NationalTaiwanUniversity,CollegeofMedicine,DepartmentofEmergency Medicine,Taipei,Taiwan

SectionEditor:JosephShiber,MD

Submissionhistory:SubmittedAugust21,2023;RevisionreceivedDecember8,2023;AcceptedDecember21,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18440

Introduction: Intra-arresttransthoracicechocardiography(TTE)andtransesophageal echocardiography(TEE)havebeenintroducedinadultpatientswithcardiacarrest(CA).Whetherthe diagnosticperformanceofTTEorTEEissuperiorduringresuscitationisunclear.Weconducteda systematicreviewfollowingPRISMAguidelines.

Methods: WesearcheddatabasesfromPubMed,Embase,andGoogleScholarandevaluatedarticles withintra-arrestTTEandTEEinadultpatientswithnon-traumaticCA.Twoauthorsindependently screenedandselectedarticlesforinclusion;theythendual-extractedstudycharacteristicsandtarget conditions(pericardialeffusion,aorticdissection,pulmonaryembolism,myocardialinfarction, hypovolemia,leftventriculardysfunction,andsonographiccardiacactivity).Weperformedquality assessmentusingtheQualityAssessmentofDiagnosticAccuracyStudiesVersion2criteria.

Results: Atotalof27studieswereincluded:14studieswith2,145patientsassessedTTE;and16with 556patientsassessedTEE.Ahighriskofbiasorapplicabilityconcernsinatleastonedomainwas presentin20studies(74%).BothTTEandTEEfoundpositive findingsinnearlyone-halfofthepatients. TheetiologyofCAwasidentifiedin13%(271/2,145),andinterventionwasperformedin38%(102/271) ofpatientsintheTTEgroup.InpatientswhoreceivedTEE,theetiologywasidentifiedin43%(239/556), andinterventionwasperformedin28%(68/239).IntheTEEgroup,ahigherincidenceregardingthe etiologyofCAwasobserved,particularlyforthosewithaorticdissection.However,theoutcomeofthose withaorticdissectionintheTEEgroupwaspoor.

Conclusion: WhileTEEcouldidentifymorecausesofCAthanTTE,sonographiccardiac activitywasreportedmuchmoreintheTTEgroup.TheimpactofTTEandTEEonthereturnof spontaneouscirculationandfurthersurvivalwasstillinconclusiveinthecurrentdataset.[WestJEmerg Med.2024;25(2)166–174.]

Keywords: cardiacarrest;resuscitation;transthoracicechocardiography; transesophagealechocardiography.

INTRODUCTION

Ultrasound(US)isconsideredavaluablediagnostictool whenthereisaclinicalsuspicionforaspecificreversible causeinpatientswithcardiacarrest(CA).1 TheuseofUS duringresuscitationhasbecomemorecommonbecauseofits non-invasiveandreadilyaccessiblecharacteristics.2,3

Transthoracicechocardiography(TTE)hasbeen introducedinresuscitativescenariosinrecentdecades.2,4,5 However,previousstudieshaveshownthatTTElengthensa singlepauseformorethan17seconds,6,7 possiblydelaying chestcompressions.Also,devicessuchasmechanicalchest compressionsystemsordefibrillationpadswouldinterfere withimageacquisition.Bycontrast,transesophageal echocardiography(TEE)couldovercomesuchlimitations, allowingreal-timevisualizationoftheheartwithout interruptingchestcompressions.8 However,thedisadvantages ofTEEincludehighcost,theneedforadvancedoperator skills,andthepotentialforiatrogenictraumaduetoits invasivenature.WhetherthediagnosticperformanceofTTE orTEEissuperiorduringresuscitationisunclear.We conductedasystematicreviewofintra-arrestTTEandTEE ontargetconditionsincludingpericardialeffusion,cardiac tamponade,aorticdissection,pulmonaryembolism(PE), myocardialinfarction(MI),hypovolemia,leftventricular (LV)dysfunction,andsonographiccardiacactivity.

METHODS

Weperformedasystematicreviewfollowingthelatest statementofthePreferredReportingItemsforSystematic ReviewsandMeta-Analysis(PRISMA2022).Thisreview protocolwassubmittedtotheInternationalProspective RegisterofSystematicReviews(PROSPERO)onMarch17, 2022(registrationnumber:CRD42022310670).

DataSourcesandSearchStrategy

Twoindependentinvestigatorssearchedliterature publisheduptoApril30,2023,inPubMed,Embase,and GoogleScholarwithoutlanguageorstudy-typerestriction. Eligibletrialswereidentifiedwiththefollowingkeywords: “echocardiography,CA,resuscitationorrescue.”

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Transthoracicechocardiography(TTE) delayschestcompressions;transesophageal echocardiography(TEE)offers real-timevisualizationwithout interruptingcompressions.

Whatwastheresearchquestion?

IsthediagnosticperformanceofTTE superiortoTEE?

Whatwasthemajor findingofthestudy?

Theetiologyofcardiacarrest(CA)was identi fi edin13%ofpatientsthroughTTEand in43%ofpatientsthroughTEE.

Howdoesthisimprovepopulationhealth?

WhileTEEcouldidentifymorecausesofCA sonographiccardiacactivity,itwasreported muchmoreintheTTEgroup.Theimpact ofTTEandTEEonfurthersurvival areinconclusive.

StudySelection

Twoauthors(YHandWL)independentlyexamined referencesusingtitlesandabstracts.Fulltextsofrelevant studieswereretrieved.Thestudyselectioncriteriawere framedusingthePICOST(Population,Intervention, Comparator,Outcome,StudyDesign,Timeframe)format asdescribedin Table1

DataExtractionandQualityAssessment

Thetwoauthors(YHandWL)extracteddatafrom eligiblestudiesincludingauthors,publicationyear,study design,casenumbers,gender,age,applicationofTTEor

InclusioncriteriaExclusioncriteria

PopulationAdultnon-traumaticpatientswithout-of-hospitalandin-hospital cardiacarrest

Traumaticarrestandpediatricpopulation

InterventionIntra-arrestTTE/TEEPost-arrestTTE/TEE

ComparativeStandardresuscitationaccordingtoALSguidelines

OutcomeIdenti ficationofthetargetconditions

StudytypeObservationalstudies(prospectiveandretrospective)and interventionalstudies(randomizedandnon-randomized)

TimeNolimitationsonthepublicationperiod

Casereportsandcaseseries,animalstudies, reviewarticles,guidelines,andeditorials

TEE,transesophagealechocardiography; TTE,transthoracicechocardiography; ALS,AdvancedLifeSupport.

Table1. Eligibilitycriteriaforstudyselection.

Hoetal. PerformanceofIntra-arrestEchocardiography

TEE,sonographers,sonographicdiagnoses,reference standard,andrateofreturnofspontaneouscirculation (ROSC)orsurvival.Qualityassessmentwasperformedusing theQualityAssessmentofDiagnosticAccuracyStudies

Version2(QUADAS-2)criteria,whichisanadequatetool fordiagnostictestaccuracyinsystematicreviews.9

Thetwoauthorsindependentlyevaluatedeachincluded study;anyinconsistencyordisagreementwasresolvedupon detaileddiscussion.

OutcomeMeasures

Theprimaryoutcomewastheincidenceproportionsof targetconditionsincludingpericardialeffusion,cardiac tamponade,aorticdissection,PE,MI,hypovolemia, LVdysfunction,andsonographiccardiacactivityby intra-arrestTTE/TEE.

DataSynthesis,StatisticalAnalysis,and SensitivityAnalysis

Duetomarkedheterogeneityamongtheincludedstudies, conductingarobustmeta-analysiswasnotfeasible.Thus,we usedanarrativesynthesistopresentthestudyresults.We calculatedthepooledincidenceproportionsoftarget conditionsofTTEandTEEandpresentthemas proportionand95%confidenceintervals(CI)using ComprehensiveMeta-AnalysisV4.0software(BiostatInc, Englewood,NJ).

RESULTS

LiteratureSearchandStudySelectionResults

Ourliteraturesearchidentifiedatotalof333resultsfrom PubMed,Embase,andGoogleScholar.Afterduplicates wereremoved,wescreened308titlesforinclusionwith34 excludedfornotmeetingthestudycriteria.Wealsoexcluded 31meta-analysesorsystematicreviewsand61casereports, caseseries,andanimalstudies.Ofthe182remainingstudies thatunderwentathoroughfull-textretrievalandreview,we selected27for finalreview(Figure).2,3,10–34

SummaryofStudies

ElevenstudiesassessedTTE,2,3,10–20 13assessed TEE,10,13,21–33 andthreeincludedboth(Supplementary Table1).10,13,34 Ofthetotal2,701patientsincluded,2,145 receivedTTE,and556receivedTEE.Onepatientwas excludedduetoatraumaticruptureofthethoracicaorta.21 Ninestudiesincludedpatientswithout-of-hospitalCA (OHCA),sevenincludedpatientswithin-hospitalCA,and another11assessedamix.Echocardiographywasperformed byemergencyphysiciansin17studies.

RiskofBiasandConcernsofApplicability

Ahighriskofbiasorapplicabilityconcernsinatleastone domainwaspresentin20studies(74%)(Table2).Theriskof biaswasunclearin10studies(38%)duetoalackof

informationregardingpatientselection.Theriskwashighin 15studies(58%)duetoaconveniencesample;onlyonestudy ratedlowriskenrolledaconsecutivesample.14 Intwostudies inwhichTTE,TEE,orbloodsamplinganalysiswas performedatthediscretionofphysicians19 theriskofbias wasratedhighbecauseofconcernfortheindextest.Therisk ofbiaswasunclearin23studies(88%)relatedtoalackof standardizedreferenceandinformationregardingthe flow andtiming.Lowrisksofbiasrelatedtoreferencestandards and flowandtimingwereratedintheotherfourstudiesin whichallimageswerereviewed(ie,uniformconfirmatory testing)andinter-raterreliabilitywasassessed.26,31,32,34

Theriskofbiasforapplicabilitywashighforpatient selectioninninestudies(35%)becauseoftheenrollmentof patientswithhemodynamicinstabilityorpulselesselectrical activity.2,11–14,22,26–28 Forapplicabilitytotheindextest,four studieswereevaluatedwithahighriskduetolargevariations indiagnosticassessment.10,13,19,34 Fourstudiesinwhich referencestandardswereprovidedwereratedwithalowrisk ofapplicabilityinreferencestandards.26,31,32,34 However,the remainingwereunclear.

PerformanceofTTEandTEEAmongthe TargetConditions

TransthoracicechocardiographyandTEEfoundpositive findingsin51%(1,101/2,145)and47%(264/556)ofpatients, respectively.Themostcommon findingwasthepresenceof sonographiccardiacactivityin855patients(830intheTTE groupand25intheTEEgroup).TheetiologyofCAwas identifiedin13%(271/2,145)ofpatientswithTTEand43% (239/556)ofpatientswithTEE(Table3, Supplementary Table2).Ahighincidenceproportionwasobservedinthe targetconditionintheTEEgroup,particularlyinthosewith aorticdissection.However,theoutcomeofpatientswith aorticdissectionwaspoor.

Thesummaryofdetailedsonographic findingsislistedin SupplementaryTable3.

InterventionAccordingtoSonographicFindings

Excludingpatientswithsonographiccardiacactivity,the effectonmanagementwasobservedin38%(102/271)of patientsreceivingTTEand28%(68/239)ofthosereceiving TEE.Themostcommoninterventionwaspericardial effusiondrainage,whichwasperformedin51%(58/113)of patientsintheTTEgroup,and51%(18/35)inthe TEEgroup.Surgerywasperformedononepatient(1/10, 10%)withsuspectedaorticdissectionintheTTEgroup17 and two(2/54,4%)intheTEEgroup.2 Thrombolysis/ embolectomywasperformedinsevenpatients(7/34,21%) and28patients(28/70,40%)withsuspectedPEinthe TTE3,16 andTEE10,24,25,27,30,31 groups,respectively. Coronaryangiographyorbypasswasperformedonone patient(1/4,25%)withsuspectedMIbyTTE17 and11 patients(11/28,39%)byTEE.21,24,25 Administrationof fluid

wasreportedin13patients(13/26,50%)withhypovolemia byTTE,2,12 andfourpatients(4/6,67%)byTEE.13,24,25 Inotropictherapywasadministeredin26%(22/84)of patientswithLVdysfunctionbyTTE2 and11%(5/46) byTEE.25,27

DISCUSSION

Intra-arrestTTEwasperformedin2,145patientsin14 studies,andTEEwasusedin556patientsin16studies.Both TTEandTEEfoundpositive findingsinnearlyone-halfof thepatients.TheetiologyofCAwasidentifiedin13%of patientswithTTEand43%ofpatientswithTEE.Prompt therapywasadministeredin38%ofpatientswithTTEpositive findingsand28%ofthosewithTEE-positive findings.IntheTEEgroup,ahigherincidenceproportion wasobservedinidentifyingtheetiologyofCA,particularly forthosewithaorticdissection.However,ahighdegreeof heterogeneityinreferencestandardsandsmall-samplesize

precludedfurthermeta-analysisforthediagnostic performanceofintra-arrestTTEandTEE.

Themajorweaknessoftheincludedstudiesisthatthe referencestandardsareinconsistent.Applyingauniform standardtoalltargetconditionsisnoteasy,andimagereview maybeaneffectivesolution.26,31,32,34 Theautopsycanbe regardedasthegoldstandardandwasperformedinthree studies,21,22,25 butthereferencestandardsarediversein patientswithROSC.Usingspecificmanagementasreference standardstojudgetargetconditionsisnotfeasible.For example,patientswithsuspectedPEdonotreceive pericardiocentesis.Also,evenfailureinpericardiocentesis doesnotindicatetheabsenceofpericardialeffusion.By contrast,pericardialeffusiondrainageisrarelyperformedin asmallamountofeffusion.True-positivecaseswereoften reported,butverificationbiasexisted.Informationwas limitedinthetrue-negative,false-positive,orfalse-negative cases.Therefore,thissystematicreviewpresentedthe

Figure. ThePreferredReportingItemsforSystematicReviewsandMeta-Analyses(PRISMA)diagram.

Hoetal. PerformanceofIntra-arrestEchocardiography

Table2. Thequalityassessmentofdiagnosticaccuracystudies(QUADAS)-2riskofbiasassessmentoftheincludedstudies.

RiskofbiasConcernsregardingapplicability Study Patient selection Index test Reference standard Flowand timing Patient selection Index test Reference standard

Transthoracicechocardiography

Varrialeetal(1997)HLUULHU

Kürkciyanetal(2000)LHLLLHL

Tayaletal(2003)ULUUHLU Breitkreutzetal(2010)HLUUHLU Chardolietal(2012)HLUUHLU Shillcuttetal(2012)HLUUHHU

Flatoetal(2015)LLUUHLU Gasparietal(2016)HLUULLU Zenginetal(2016)ULUULLU Chuaetal(2017)ULUULLU Lienetal(2018)HLUULLU Balderstonetal(2021)HLUULLU Heikkilä1etal(2023)UHUULHU Lienetal(2023)HLUULLU

Transesophagealechocardiography

Varrialeetal(1997)HLUULHU vanderWouwetal (1997) ULUULLU

Comessetal(2000)ULUUHLU Kürkciyanetal(2000)LHLLLHL Miyakeetal(2004)HLUUHLU Linetal(2006)ULUULLU Mentsoudisetal(2006)ULUULLU Shillcuttetal(2012)HLUUHHU Hilberathetal(2014)HLLLHLL Burrageetal(2015)ULUUHLU Arntfieldetal(2016)ULUUULU

Teranetal(2019)HLUULLU Jungetal(2020)ULUULLU Kimetal(2020)HLLLLLL Jungetal(2022)HLLLLLL Poppe(2023)HLUULLU H,highriskofbias; L,lowriskofbias; U,unclearriskofbias.

incidenceoftargetconditionsandcouldnotfurtherexplore whetherTTEorTEEwasbetterduringresuscitation.

Oneofthemostimportantindicatorsofthelikelihoodof ROSCorsurvivalisthepresenceofsonographiccardiac activity.35 Sonographiccardiacactivitywasmuchmore frequentlydetectedbyTTEthanbyTEE(830vs25)inour review.Interestingly,regionalwallmotionabnormality suggestiveofMIandLVdysfunctionwasdetectedinthe

presenceofsonographiccardiacactivity.Evenaddingthe numbersoftheseconditions,thetotalnumberwasstillhigher inpatientswithTTE.Whetheritoccurredduetotakinga longertimetosetupTEE(possiblyresultinginresuscitation timebias36)wasuncertain.Furtherstudiesareneededto determinewhetherTEEcanbettercharacterizeintra-arrest myocardialmovementorcardiacactivitydetected byTEEisunder-reported.TheimpactofTTEandTEEon

PerformanceofIntra-arrestEchocardiography Hoetal.

Findings

TransthoracicechocardiographyN; incidenceproportion[95%CI]

TransesophagealechocardiographyN; incidenceproportion[95%CI]

Pericardialeffusion113;0.068[0.046;0.100]35;0.117[0.056;0.226]

Cardiactamponade30;0.059[0.041;0.083]25;0.095[0.036;0.228]

Aorticdissection10;0.023[0.008;0.064]54;0.119[0.074;0.186]

Pulmonaryembolism34;0.053[0.021;0.126]70;0.220[0.116;0.378]

Myocardialinfarction4;0.022[0.002;0.192]28;0.291[0.131;0.528]

Hypovolemia26;0.044[0.013;0.142]6;0.147[0.067;0.291]

LVdysfunction84;0.181[0.086;0.343]46;0.535[0.170;0.866]

Sonographiccardiacactivity830;0.488[0.374;0.604]25;0.243[0.138;0.390]

LV,leftventricular; CI,confidenceinterval.

ROSCandfurthersurvivalwasstillinconclusiveinthe currentdataset.

Themostcommoninterventionwaspericardialeffusion drainageduringresuscitation.Theoutcomesvaried dependingontheetiologyofeffusions.Onepatientwith cardiactamponadeandLVfreewallrupturesecondaryto transmuralMIbyTTEdiedafteranexploratory thoracotomy.14 Returnofspontaneouscirculationwasnot achievedinonepatientwithtamponadeandaorticdissection byTTE.17 Onepatienthadcardiactamponadethatwasnot visibleonTTEowingtopooracousticwindowsbutwas evidentonTEEandsurvivedaftersurgicaltreatment.21 One patientwithtamponadeandrightventricularrupture survivedtodischargeafterreceivinganemergentwall repair.25 Earlyterminationofresuscitationwasconducted insixpatientswithtamponadeduetomyocardialrupture andonepatientwithtamponadeduetoaorticdissection byTEE.21,31

TheincidenceofaorticdissectionwashigherbyTEE, whichwasrelatedtodirectvisualizationoftheaorticroot anddescendingaortabythelong-axisandshort-axisviews. However,thedatashouldbecautiouslyinterpretedbecause Jungetalreported19patients 30 andKimetalreported10 patients,31 whichcouldskewtheresults.Moreover,the outcomeswerepoorinthatonlyonepatientwithTTE20 and twowithTEE23 hadROSC.

Pulmonaryembolismisthemostreported findingbyTEE, whichillustratesthethrombidirectly13,22,24,25,27,30 or obstructiontocolor flowinthepulmonaryartery.10 Transthoracicechocardiographyusestheindirectsignof rightventriculardilatation,indicativeofPE.2,3,14–16 However,false-positiveandfalse-negativecaseswere reported,andnotallthepatientsreceivedthrombolysisor thrombolectomy.VandeWouwetalreportedonehada TEEdiagnosisofPEbutnoemboluswasfoundatautopsy.21 Comessetalreportedbilateralperipheralpulmonaryemboli

atautopsybutnotseenbyTEE.22 Jungetalreportedone patientwithinitialnegativeTEE findingshadthrombiinthe mainpulmonaryarteryatthe finalreview,andonepatient withsalinebubblesinthepulmonaryarterywas misinterpretedasPE.30

Gasparietalreportedoneofthe15patientswith suspectedPEreceivingthrombolysissurvivedhospital discharge.3 Chuaetalreportedoneoffourpatients suggestiveofmassivePEbyTTEreceivingthrombolysis survivedtodischarge.16 AlthoughtheROSCratesofPEby TTEorTEEwerestilllower,theywerebetterthanthoseof aorticdissection.

Thesonographic findingsuggestiveofMIonintra-arrest TTEandTEEisregionalwallmotionabnormalityinthe presenceofsonographiccardiacactivity.VandeWouwetal reportedtwopatientshadMIatautopsythatcouldnotbe demonstratedwithTEEowingtolackofspontaneous rhythm.21 Lienetalreportedextensiveanteriorwallakinesia oftheleftventriclethatwasidentifiedinonepatientwith pulselesselectricalactivity.17

AlowLVend-diastolicvolumeisacharacteristic finding indicativeofhypovolemiabyTTE2 andTEE.13,24 Fluid resuscitationwasreportedintwostudiesofTTE,2,12 and threestudiesofTEE.13,24,25 LactatedRingersolution infusioninOHCAincreasedthelikelihoodofprehospital ROSC37;however,theinformationregardingthedetailsof fluidwaslackingintheincludedstudies.

ReducedLVfunctionpresentedasacommon findingby intra-arrestTTEandTEE.Insteadofprecisemeasurement, reducedLVfunctionisestimatedbyvisualassessment (eyeballing)viaECHOofanejectionfractionlessthan 45–55%.13,14,18 Also,theventricularfunctioncouldbe assessedonlyduringintervalsofspontaneouscardiac contraction.22 InotropictherapywasadministeredinonefourthofpatientswithLVdysfunctionbyTTE2 andonetenthofthosebyTEE.25,27

Table3. Pooledresultsoftarget findingsontransthoracicechocardiographyandtransesophagealechocardiographyinpatientswith cardiacarrest.

Hoetal. PerformanceofIntra-arrestEchocardiography

Intra-arrestTTEisaconvenientimagingmodality,butit ischallengedbythetechnicaldifficultyinobtainingadequate cardiacwindowsduringthepausesandpotentialdelaysin chestcompressions.7 Transesophagealechocardiography hasbeenrecognizedasanalternativewithoutinterferingwith ongoingresuscitationefforts.Itprovidesreal-timefeedback onthelocationofchestcompressionsandthequalityof cardiopulmonaryresuscitation(CPR).38 However,TEEhas disadvantagessuchashighcost,highlevelofoperatorskill trainingrequired,andpotentiallyiatrogenictrauma includingoropharyngealesophagealandgastriclacerations, andperforation.39 Threestudiesreportednocomplications ordelaysinresuscitationprocedures.22,32,33 Thetransgastric viewwasexcludedfromtheTEEprotocoltoavoidpotential complicationsortolesseninterferencewiththechest compressionprocedure.30,31

LIMITATIONS

Therewerelimitationsinthisreview.First,theselected studieswerehighlyheterogeneous,small-sizedsamples,with alackofreferencegroupsandstandardizedconfirmation tests.Mostofthepatientswerecollectedfromaconvenience samplefromasingleinstitution.Beyondtheselectionbias, resuscitationfacilitiesandinterventionsforspecificdiseases maydifferdependingontheinstitution’scapability,limiting thegeneralizability.Futurestudiesassessingthediagnostic accuracyofUSinpatientswithCAshouldavoid methodological flaws;arandomizedcontrolledtrial comparingTTEwithTEEwouldbeasolution.Second,the detailedtrainingbackgroundofsonographerswasunclearin someoftheincludedstudies.10,13,19,21,23–25,34 The2022 guidelinessuggestUScanbeperformedbyexperienced personnelwithoutinterruptingCPR.1 Third,thetimingof theintroductionofTTEandTEEwasnotclearinthestudies; soresuscitationtimebiascouldnotbeestimated.39 Lienetal usedUSwithin10minutesofAdvancedLifeSupport(ALS), andGasparietalintroducedUSafter fiveminutesof ALS.3,17 Ontheotherhand,JungetalandKimetal introducedTEEafter10minutesofALS.30,31 Lienetal reportedthatTTEwasperformedinpatientswithlonger resuscitationtime.20

Lastly,theimpactofTTEandTEEonROSCandfurther survivalwasnotthoroughlydiscussed.TheetiologyofCA wasidentifiedin13%ofpatientswithTTEand43%of patientswithTEE.Amongthem,approximately20–25%of patientswitheachtargetconditionachievedROSC exceptthosewithaorticdissection(10%).However,the resuscitationdataassociatedwithROSCsuchaswitnessed arrest,earlyALS,andearlydefibrillationwerenot presented.Also,patientswithearlyROSCbeforeUSwas attemptedwereexcludedfromsomestudies.3,30 Future researchwouldfocusonevaluatingthevaluesofTTE andTEEonROSC,hospitalsurvival,orlong-term neurologicaloutcomes.

CONCLUSION

Transesophagealechocardiographycouldidentifymore causesofcardiacarrestthantransthoracic echocardiography.However,sonographiccardiacactivity, indicativeofbetterratesofreturnofspontaneouscirculation, wasreportedmuchmoreintheTTEgroup.Theimpactof TTEandTEEonROSCandfurthersurvivalwas inconclusiveinthecurrentdata.Ahighdegreeof heterogeneityinpatientselectionandalackof referencestandardsprecludedfurthermeta-analysis forthediagnosticperformanceofintra-arrest

TTEandTEE.

AddressforCorrespondence:Wan-ChingLien,MD,PhD,National TaiwanUniversityHospitalandNationalTaiwanUniversityCollege ofMedicine,DepartmentofEmergencyMedicine,No.7,ChungShanSouthRd.,Taipei100,Taiwan.

Email: wanchinglien@ntu.edu.tw

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Thisstudyreceivedgrantfundingfrom theNationalScienceandTechnologyCouncil,Taiwan(NSTC1122410-H-002-171).Therearenootherconflictsofinterestorsources offundingtodeclare.

Copyright:©2024Hoetal.Thisisanopenaccessarticledistributed inaccordancewiththetermsoftheCreativeCommonsAttribution (CCBY4.0)License.See: http://creativecommons.org/licenses/by/ 4.0/

REFERENCES

1.WyckoffMH,GreifR,MorleyPT,etal.2022InternationalConsensuson CardiopulmonaryResuscitationandEmergencyCardiovascularCare SciencewithTreatmentRecommendations:summaryfromtheBasic LifeSupport;AdvancedLifeSupport;PediatricLifeSupport;Neonatal LifeSupport;Education,Implementation,andTeams;andFirstAidTask Forces. Circulation.2022;146(25):e483–557.

2.BreitkreutzR,PriceS,SteigerHV,etal.Focusedechocardiographic evaluationinlifesupportandperi-resuscitationofemergencypatients:a prospectivetrial. Resuscitation.2010;81(11):1527–33.

3.GaspariR,WeekesA,AdhikariS,etal.Emergencydepartmentpoint-ofcareultrasoundinout-of-hospitalandin-EDcardiacarrest. Resuscitation.2016;109:33–9.

4.HernandezC,ShulerK,HannanH,etal.C.A.U.S.E.:cardiacarrest ultrasoundexam–abetterapproachtomanagingpatientsin primarynon-arrhythmogeniccardiacarrest. Resuscitation 2008;76(2):198–206.

5.ClattenburgEJ,WroePC,GardnerK,etal.Implementationofthe cardiacarrestsonographicassessment(CASA)protocolforpatients withcardiacarrestisassociatedwithshorterCPRpulsechecks. Resuscitation.2018;131:69–73.

6.HuisIn ’tVeldMAAllisonMG,BostickDS,etal.Ultrasounduseduring cardiopulmonaryresuscitationisassociatedwithdelaysinchest compressions. Resuscitation.2017;119:95–8.

7.ClattenburgEJ,WroeP,BrownS,etal.Point-of-careultrasoundusein patientswithcardiacarrestisassociatedprolongedcardiopulmonary resuscitationpauses:aprospectivecohortstudy. Resuscitation Jan2018;122:65–8.

8.FairJ3rd,MallinMP,AdlerA,etal.Transesophagealechocardiography duringcardiopulmonaryresuscitationisassociatedwithshorter compressionpausescomparedwithtransthoracicechocardiography. AnnEmergMed.2019;73(6):610–6.

9.WhitingPF,RutjesAW,WestwoodME,etal.QUADAS-2:arevisedtool forthequalityassessmentofdiagnosticaccuracystudies. AnnIntern Med.2011;155(8):529–36.

10.VarrialePandMaldonadoJM.Echocardiographicobservations duringinhospitalcardiopulmonaryresuscitation. CritCareMed 1997;25(10):1717–20.

11.TayalVSandKlineJA.Emergencyechocardiographytodetect pericardialeffusioninpatientsinPEAandnear-PEAstates. Resuscitation.2003;59(3):315–8.

12.ChardoliM,HeidariF,RabieeH,etal.Echocardiographyintegrated ACLSprotocolversusconventionalcardiopulmonaryresuscitationin patientswithpulselesselectricalactivitycardiacarrest. ChinJ

Traumatol.2012;15(5):284–7.

13.ShillcuttSK,MarkinNW,MontzingoCR,etal.Useofrapid “rescue” perioperativeechocardiographytoimproveoutcomesafter hemodynamicinstabilityinnoncardiacsurgicalpatients. JCardiothorac VascAnesth.2012;26(3):362–70.

14.FlatoUA,PaivaEF,CarballoMT,etal.Echocardiographyfor prognosticationduringtheresuscitationofintensivecareunit patientswithnon-shockablerhythmcardiacarrest. Resuscitation 2015;92:1–6.

15.ZenginS,YavuzE,AlB,etal.Benefitsofcardiacsonographyperformed byanon-expertsonographerinpatientswithnon-traumatic cardiopulmonaryarrest. Resuscitation.2016;102:105–9.

16.ChuaMT,ChanGW,KuanWS.Reversiblecausesincardiovascular collapseattheemergencydepartmentusingultrasonography (REVIVE-US). AnnAcadMedSingap.2017;46(8):310–6.

17.LienWC,HsuSH,ChongKM,etal.US-CABprotocolfor ultrasonographicevaluationduringcardiopulmonaryresuscitation: validationandpotentialimpact. Resuscitation.2018;127:125–31.

18.BalderstonJR,YouAX,EvansDP,etal.Feasibilityoffocusedcardiac ultrasoundduringcardiacarrestintheemergencydepartment. CardiovascUltrasound.2021;19(1):19.

19.HeikkiläE,JousiM,NurmiJ.Differentialdiagnosisandcause-specific treatmentduringout-of-hospitalcardiacarrest:aretrospective descriptivestudy. ScandJTraumaResuscEmergMed 2023;31(1):19.

20.LienWC,ChongKM,ChangCH,etal.Impactofultrasonographyon chestcompressionfractionandsurvivalinpatientswithout-of-hospital cardiacarrest. WestJEmergMed.2023;24(2):322–30.

21.vanderWouwPA,KosterRW,DelemarreBJ,etal.Diagnosticaccuracy oftransesophagealechocardiographyduringcardiopulmonary resuscitation. JAmCollCardiol.1997;30(3):780–3.

22.ComessKA,DeRookFA,RussellML,etal.Theincidenceofpulmonary embolisminunexplainedsuddencardiacarrestwithpulselesselectrical activity. AmJMed.2000;109(5):351–6.

23.MiyakeM,IzumiC,TakahashiS,etal.Efficacyoftransesophageal echocardiographyinpatientswithcardiacarrestorshock. JCardiol Nov2004;44(5):189–94.

24.LinT,ChenY,LuC,etal.Useoftransoesophagealechocardiography duringcardiacarrestinpatientsundergoingelectivenon-cardiac surgery. BrJAnaesth.2006;96(2):167–70.

25.MemtsoudisSG,RosenbergerP,LofflerM,etal.Theusefulnessof transesophagealechocardiographyduringintraoperativecardiacarrest innoncardiacsurgery. AnesthAnalg.2006;102(6):1653–7.

26.HilberathJN,BurragePS,ShernanSK,etal.Rescuetransoesophageal echocardiographyforrefractoryhaemodynamicinstabilityduring transvenousleadextraction. EurHeartJCardiovascImaging 2014;15(8):926–32.

27.BurragePS,ShernanSK,TsenLC,etal.Emergenttransesophageal echocardiographyinhemodynamicallyunstableobstetricpatients. IntJObstetAnesth.2015;24(2):131–6.

28.ArntfieldR,PaceJ,HewakM,etal.Focusedtransesophageal echocardiographybyemergencyphysiciansisfeasibleandclinically influential:observationalresultsfromanovelultrasoundprogram. JEmergMed.2016;50(2):286–94.

29.TeranF,DeanAJ,CentenoC,etal.Evaluationofout-of-hospitalcardiac arrestusingtransesophagealechocardiographyintheemergency department. Resuscitation.2019;137:140–7.

30.JungWJ,ChaKC,KimYW,etal.Intra-arresttransoesophageal echocardiographic findingsandresuscitationoutcomes. Resuscitation 2020;154:31–7.

31.KimYW,JungWJ,ChaKC,etal.Diagnosisofaorticdissectionby transesophagealechocardiographyduringcardiopulmonary resuscitation. AmJEmergMed.2021;39:92–5.

32.JungWJ,ChaK,RohYI,etal.Right-to-leftshuntsoccurduring cardiopulmonaryresuscitation:echocardiographicobservations. CritCareMed.2022;50(10):1486–93.

33.PoppeM,MagnetIAM,ClodiC,etal.Resuscitativetransoesophageal echocardiographyperformedbyemergencyphysiciansinthe emergencydepartment:insightsfroma1-yearperiod. EurHeartJAcute CardiovascCare.2023;12(2):124–8.

34.KürkciyanI,MeronG,SterzF,etal.Pulmonaryembolismasacause ofcardiacarrest:presentationandoutcome. ArchInternMed. 2000;160(10):1529–35.

35.BlythL,AtkinsonP,GaddK,LangE.Bedsidefocused echocardiographyaspredictorofsurvivalincardiacarrestpatients:a systematicreview. AcadEmergMed.2012;19(10):1119–26.

36.AndersenLW,GrossestreuerAV,DonninoMW. “Resuscitationtime bias” auniquechallengeforobservationalcardiacarrestresearch. Resuscitation.2018;125:79–82.

Hoetal. PerformanceofIntra-arrestEchocardiography

37.HagiharaA,HasegawaM,AbeT,etal.Prehospitallactated Ringer’ssolutiontreatmentandsurvivalinout-of-hospitalcardiacarrest: aprospectivecohortanalysis. PLoSMed.2013;10(2):e1001394.

38.TeranF,PratsMI,NelsonBP,etal.Focusedtransesophageal echocardiographyduringcardiacarrestresuscitation:

JACCreviewtopicoftheweek. JAmCollCardiol 2020;76(6):745–54.

39.HilberathJN,OakesDA,ShernanSK,etal.Safetyof transesophagealechocardiography. JAmSocEchocardiogr 2010;23(11):1115–27;quiz1220-1.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

StaffingPatternsofNon-ACGMEFellowshipswith4-Year ResidencyPrograms:ANationalSurvey

DavidA.Haidar,MD*

LauraR.Hopson,MD*

RyanV.Tucker,MD†

RobD.Huang,MD*

JessicaKoehler,MD*

NikTheyyunni,MD*

NicoleKlekowski,MD*

ChristopherM.Fung,MD,MS*

SectionEditor:BenjaminSchnapp,MD

*UniversityofMichigan,DepartmentofEmergencyMedicine,AnnArbor,Michigan † UniversityofColorado,DepartmentofEmergencyMedicine,Aurora,Colorado

Submissionhistory:SubmittedSeptember13,2023;RevisionreceivedNovember10,2023;AcceptedDecember7,2023

ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18454

Introduction: Emergencymedicine(EM)isoneoffewspecialtieswithvariabletraininglengths.Hiringa three-yeargraduatetocontinuefellowshiptraininginadepartmentthatsupportsafour-yearresidency programcanleadtoconflictsaroundresidentsupervision.Wesoughttounderstandhiringandclinical supervision,orstaffing,patternsofnon-AccreditationCouncilforGraduateMedicalEducation(ACGME) fellowshipshostedatinstitutionssupportingfour-yearresidencyprograms.

Methods: Weperformedaweb-based,cross-sectionalsurveyofnon-ACGMEfellowshipdirectors(FD) hostedatinstitutionssupportingfour-yearEMresidencyprograms.Wecalculateddescriptivestatistics. Ourprimaryoutcomewastheproportionofprogramswithfour-yearEMresidenciesthathirenonACGMEfellowsgraduatingfromthree-yearEMresidencies.

Results: Of119eligibleFDs,88(74%)completedthesurvey.SeventyFDs(80%)indicatedthattheyhire graduatesofthree-yearresidencies.Fifty-six(80%)indicatedthatthree-yeargraduatessupervise residents.MostFDs(74%)indicatednoadditionalrequirementsexisttosuperviseresidentsoutsideof beinghiredasfaculty.TheFDsciteddepartmentpolicy,concernsaboutqualityandlengthoftraining, andresidentcomplaintsasreasonsfornothiringthree-yeargraduates.Amajority(10/18,56%)noted thatnothiringfellowsfromthree-yearprogramsnegativelyimpactsrecruitmentandgivesthemaccessto asmallerapplicantpool.

Conclusion: Mostnon-ACGMEfellowshipsatinstitutionswithfour-yearEMprogramsrecruitthree-year graduatesandallowthemtosuperviseresidents.Thissurveyprovidesprogramsinformationonhow comparablefellowshipsrecruitandstafftheirdepartments,whichmayinformpoliciesthat fittheneedsof theirlearners,thefellowship,andthedepartment.[WestJEmergMed.2024;25(2)175–180.]

INTRODUCTION

Emergencymedicine(EM)isoneoffewspecialtiesinthe UnitedStateswithvariabletraininglengths.1,2 Most residenciesimplementathree-yearmodel,whileonly20% implementafour-yearmodel.3–5 Thereislittledatato supporteithertraininglength.3–7 Somearguethatfour-year

graduateshavemoretimetogainconfidence,develop proceduralskills,developacademicinterests,andgain experiencesupervisinglearners.Advocatesofthree-year programsarguethatanextrayearasfacultywould providethesesameexperiences.1,8,9 Thesepersonal biasesmayimpactrecruitmentandhiringof

three-yeargraduatesatinstitutionssupportingfour-year residencyprograms.6,7

Whenaninstitutionhostingafour-yearresidencyhiresa three-yeargraduateintofellowshiptraining,thiscanleadto conflictsaroundclinicalsupervision,orstaffing,ofresidents relatedtoperceptionsofseniorityandqualityoftraining.1,9 Therearecurrentlynobestpracticesorguidelinestoinform programsonhowtoaddressthissituation.Thesituationis furthercomplicatedasnon-AccreditationCouncilfor MedicalEducation(ACGME)fellowshipsfrequentlylack uniformrulesthatgovernrecruitment,program requirements,andclinicalresponsibilities.10 Nostudies currentlyevaluatetheprevalenceoftheseissuesorexamine variabilityinrecruitment,hiring,andclinicalresponsibilities oftraineesatnon-ACGMEfellowships.Inthisstudy,we soughttounderstandthehiringandstaffingpatternsofnonACGMEfellowshipshostedatinstitutionswithfour-year EMresidencyprograms.

METHODS

StudyDesignandParticipants

Thiswasacross-sectionalsurveyoffellowshipdirectors (FD)ofnon-ACGMEfellowshipshostedatinstitutions supportingafour-yearEMresidencyprogram.We conductedthesurveybetweenJanuary–April2023.This studywasdeemedexemptbyourinstitutionalreviewboard (HUM00221519).InNovember2022,wegeneratedalistof 54four-yearEMresidencyprogramsfromtheEmergency MedicineResidents’ Association(EMRA)Matchrosterand ElectronicResidencyApplicationServicedirectory.11–13 We identifiednon-ACGMEfellowshipsofferedusingeach program ’swebpage,theSocietyforAcademicEmergency MedicineFellowshipDirectory,andtheSocietyforClinical UltrasoundFellowshipsdirectory.14,15

SurveyDevelopmentandDistribution

WedevelopedthesurveybasedonPanacek’sgeneral surveyprinciples,literaturereview,andexpertopinionto providecontentvalidityevidence.6,16–18 Allauthorshave experiencedevelopingsurveystudies,andthegroup (includingfourcurrentorformerFDs)iterativelypilotedand revisedthesurveyforoptimalphrasing,surveylength, functionality,andappropriatemixofsuggestedand open-endedresponses,whichprovidedcontentand responseprocessvalidityevidence.18 WeusedQualtrics (QualtricsXM,Provo,UT),aweb-basedsurveyplatform,to distributethesurveyviaemailwithapersonalizedlinkfor eachFDtocollectandanalyzethedata.Wesentweekly reminderstoFDs’ institutionalemails,withanoptionto declineparticipation,foreightweeks.Wethensent personalizedweeklyreminderemailsforanadditional fourweeks.Wecollectedindividualresponsestothe surveyanonymously.

OutcomesandDataAnalysis

WeaskedFDstoreporttheirfellowshiptype,yearsin currentrole,anddemographicdatasuchasnumberof clinicalsites,programenvironment(academic,county, community,etc),andgeographiclocation.Ourprimary outcomewastheproportionofprogramsaffiliatedwithfouryearEMresidenciesthathirenon-ACGMEfellows graduatingfromthree-yearEMresidencies.Wealsoasked clarifyingquestionstobetterunderstandtheirstaffingmodel, andrecruiting,hiring,andclinicaloversightpolicies.The surveyincludedspaceforcommentssothattheFDscould providecontexttotheiranswers,butwedidnotanalyzethese forthemes.Thefullsurveyisavailablein AppendixA1.We analyzedthedatausingExcel365(MicrosoftCorporation, Redmond,WA)togeneratedescriptivestatisticsand analysis.Weassessedtheassociationbetweencategorical variablesusingtheFisherexacttest.Wedidnotcalculatean apriorisample-sizeestimateasweattemptedtocapturea 100%responserate.

RESULTS

Of54four-yearEMresidenciesintheUS,32institutions offeredatleastonenon-ACGMEfellowshipwithatotalof 128fellowshipsidentified(median3.5;range1–10).We received88responsesafterexcludingnineopt-outsandone blankresponse(88/119)foraresponserateof73.9%. ProgramandFDcharacteristicsarelistedinthe Table. Freetextresponsesareincludedin AppendixA2

Ofthe88responses,70FDs(80%)reportedhiring graduatesofthree-yearEMprogramsfortheirrespective fellowships.Fifty-sixFDs(80%)whoacceptthree-year graduatesindicatedthattheirfellowscansuperviseEM residents.Wefoundvariationinwhofellowscouldsupervise. Themostcommonpolicy(40%)wasthatfellowscan superviseEMpostgraduate-year(PGY)-3residentsand below.MostFDs(74%)indicatedthattheyhadno additionalrequirementstosuperviseresidentsoutsideof beinghiredonasfaculty.Fullsurveyresultsappearin the Figure

Programswithmultipleclinicalsitesaremorelikelytohire three-yeargraduates.Tenof23programs(57%)withone clinicalstaffingsitehiredthree-yeargraduatescomparedto 88%(57/65)ofsiteswithtwoormoreclinicalsites (P < 0.001).TheFDsreportedtheimplementationofvarious strategiestomitigatepotentialconflicts.Oneprogram hostsajointfellowshipcurriculumfortheirfellows,which incorporatesinstructiononbedsideteaching,giving feedback,andteachingvariousskills.OtherFDsreported thattheirprogramspreventedtheirfellowsfromstaffingin highacuityareasordelayworkingwithresidents.

Twenty-sevenFDs(50%)citeddepartmentpolicyasthe reasonfortheirhiringandstaffingpolicies.Selected commentsfromotherFDsincludedconcernsaboutquality andlengthoftrainingandresidentcomplaints.Others

Central(IL,IN,IA,KS,MI,MN,MO,NE,OH,WI)13(15%)

Northeast(CT,DC,DE,MA,MD,ME,NH,NJ,NY,PA,RI,VT)45(51%)

Southern(AL,AR,FL,GA,KY,LA,MS,NC,OK,PR,SC,TN,TX,VA,WV)0(0%)

Western(AZ,CA,CO,NM,NV,OR,UT,WA)30(34%)

’

*Respondentscouldselectmorethanonetypeofclinicalsite.

reportedtheirclinicalenvironmentwasnotconduciveto separatingfellowsfromresidents.SevenFDsreported wantingtoavoidPGY-4fellowsstaffingPGY-4residents. OneFDindicatedthat “becausewearea4-yearprogram,we

wanttoacknowledgetoourresidentsthat4yearsiswhatwe thinkisrequiredforgraduation.”

Amongtheprogramsnothiringfellowsfromthree-year programs,56%(10/18)ofFDsnotedthatthispolicy

Table. Demographicdetailsofthefellowshipsrepresentedinoursurveyoffellowshipdirectorsofnon-ACGMEfellowshipprograms.

EM,emergencymedicine; PGY,postgraduateyear; FPPE,focusedprofessionalpracticeevaluation.

negativelyimpactedtheirfellowshiprecruitmentandgave themaccesstoasmallerpoolofapplicants.

DISCUSSION

Toourknowledgethisstudyisthe firsttodescribestaffing patternsofnon-ACGMEfellowshipshostedatinstitutions withfour-yearEMresidencies.MostoftheFDswesurveyed hirethree-yeargraduatesasfellows,andmostprograms permitthree-yeargraduatestostaffresidentswithno additionalrequirementsbeyondbeinghired.Wealso identifiedpotentialnegativeimpactsonfellowshipsasthey restricttheirapplicantpool.OneFDindicatedthattheir fellowshipwasmostlygoingunfilledduetotheirrecruitment policy.Anotherindicatedthatthe financialsacrificeofa four-vsthree-yearresidencymayunintentionallyfavor recruitmentofthosewithout financialneedorburden, especiallysincethedebtloadofEMapplicantsisreportedly higherthanforothermedicalspecialties.1

Someprogramsoffertheirfellowsalternativeclinical sites – suchasVeteransAffairshospitals,freestandingEDs, orurgentcares.Bystaffingmultiplelocations,non-ACGME fellowscanworkwithoutaresidentpresence.This flexibility allowsprogramstohirethree-yeargraduatesandpermits fellowstointerfacewithresidentsacademicallywithout havingtosupervisethemclinically.Thisallowsforatraining

environmentconducivetotheneedsofalllearners’ growth anddevelopment.

TheFDscitedclinicalconcernsanddepartmentpolicyas themainreasonsfortheirstaffingandhiringpolicies.There isalackofobjectivedatathatfour-yeargraduates outperformthree-yeargraduatesclinicallyoronthe qualifyingwrittenboardexam,suggestingthatthismaybe rootedinbias.1,6,7 Intheabsenceofrobustdatatosupport theclinicalcapabilitiesoftraineesfromeitherthree-orfouryearprograms,theprinciplesofcompetency-basedmedical education(CBME)mayoffersolutions.19 Theprinciplesof CBMErequiredemonstrationofcompetencyanddecouple attainmentofcompetencyfromtime-in-training.19 Theuse ofCBMEtodeterminereadinessforunsupervisedpractice throughaprocessknownas “promotioninplace” hasbeen pilotedbysomeresidencyprogramsandmaybeauseful modeltoreplicateindeterminingfellowreadinessfor staffing,regardlessofPGYstatus.19,20 Ifweremovethefocus fromtime-boundedtrainingandfocusondemonstratedskill acquisition,programsmaydesignprocessestoonboard three-yeargraduatesbyfocusingondevelopingandassessing appropriateskillsforsupervisionoftrainees.

Futurestudiescouldexplorewhosetsdepartmental policiesregardingfellowstaffing,evaluatefellowand residentperceptionsofstaffingpolicies,andcompare

Figure. Flowdiagramdetailingthesurveyresponsehierarchyoffellowshipdirectorsofnon-AccreditationCouncilforGraduateMedical Educationfellowshipsregardingclinicalsupervisionpatternsfor3-yearemergencymedicine(EM)residencygraduatesatinstitutionswitha 4-yearEMprogram.

careeroutcomesoffellowsworkinginvarious staffingenvironments.

LIMITATIONS

Wemaynothavecapturedallnon-ACGMEfellowships atfour-yearinstitutions.Wedidnotidentifyfellowship directoriesbesidesultrasound,whichmayhaveledto samplingbias.Weattemptedtomitigatethisbysearching specificprogramwebsitesforlistedfellowships.TheFDs whodidnotparticipateinourstudymayrepresentaunique populationwithdifferenthiringandstaffingpatterns.Wedid notidentifynon-ACGMEfellowshipshostedatfour-year EMprogramsinthesouthernUS,nordidwereceive responsesfromprimarilycommunityEMprograms,which couldalsohavebiasedourresults.Wedidnotsurvey ACGME-accreditedfellowships,asfellowsvaryintheway they “maintaintheirprimaryBoardskills.”21 SomeACGME fellowships(eg,criticalcare,emergencymedicalservices)do notrequireminimumclinicalhoursintheemergency department,whichleadstoaqualitativelydifferent experiencefromnon-ACGMEfellowships,wherefellowsare appointedasclinicalfaculty.2,21,22

CONCLUSION

Ourresultsindicatethatmostnon-ACGMEfellowships hostedatinstitutionswithfour-yearEMprogramsrecruit graduatesofthree-yearprogramsandallowthemto superviseresidents.Thissurveydataprovidesprogram informationonhowcomparablefellowshipprogramsrecruit andstafftheirdepartments,whichmayinformpoliciesthat fittheneedsoftheirlearners.

AddressforCorrespondence:DavidA.Haidar,MD,Universityof Michigan,DepartmentofEmergencyMedicine,1500E.Medical CenterDr.,B1-380,AnnArbor,MI48109.Email: dahaidar@med. umich.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Haidaretal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.RossTM,WolfeRE,MuranoT,etal.Three-vs.four-yearemergency medicinetrainingprograms. JEmergMed. 2019;57(5):e161–5.

2.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationin

emergencymedicine.2023.Availableat: https://www.acgme.org/ specialties/emergency-medicine/program-requirements-and-faqsand-applications/.AccessedJanuary6,2023.

3.LotfipourS,LuuR,HaydenSR,etal.Becominganemergencymedicine resident:apracticalguideformedicalstudents. JEmergMed. 2008;35(3):339–44.

4.NelsonLS,CalderonY,AnkelFK,etal.AmericanBoardofEmergency Medicinereportonresidencyandfellowshiptraininginformation (2021–2022). AnnEmergMed. 2022;80(1):74–83.e8.

5.AccreditationCouncilforGraduateMedicalEducation(ACGME).Listof programsbyspecialty.Availableat: https://apps.acgme.org/ads/Public/ Reports/Report/1.AccessedApril29,2023.

6.HopsonL,ReganL,GisondiMA,etal.Programdirectoropiniononthe ideallengthofresidencytraininginemergencymedicine. AcadEmerg Med. 2016;23(7):823–7.

7.NikollaDA,ZocchiMS,PinesJM,etal.Four-andthree-yearemergency medicineresidencygraduatesperformsimilarlyintheir firstyearof practicecomparedtoexperiencedphysicians. AmJEmergMed. 2023;69:100–7.

8.HaydenSandPanacekE.Proceduralcompetencyinemergency medicine:thecurrentrangeofresidentexperience. AcadEmergMed. 1999;6(7):728–35.

9.WeichenthalL.Advantagesofafour-yearresidency. CalJEmergMed. 2004;5(1):18–9.

10.AccreditationCouncilforGraduateMedicalEducation(ACGME). Commonprogramrequirements(fellowship).2022.Availableat: https:// www.acgme.org/globalassets/pfassets/programrequirements/ cprfellowship_2022v3.pdf.AccessedJanuary6,2023.

11.EmergencyMedicineResidents’ Association(EMRA).Matchlist.2023. Availableat: https://webapps.acep.org/utils/spa/match#/search/list AccessedJanuary6,2023.

12.AssociationofAmericanMedicalColleges(AAMC).Electronic residencyapplicationservice(ERAS)directory.2023.Availableat: https://systems.aamc.org/eras/erasstats/par/display.cfm? NAV_ROW=PAR&SPEC_CD=110.AccessedJanuary6,2023.

13.AssociationofAmericanMedicalColleges(AAMC).Residencyexplorer tool.Availableat: https://www.residencyexplorer.org/Home/Dashboard AccessedJanuary6,2023.

14.SocietyforAcademicEmergencyMedicine(SAEM).Fellowship directory.Availableat: https://member.saem.org/SAEMIMIS/ SAEM_Directories/Fellowship_Directory/SAEM_Directories/P/ FellowshipList.aspx.AccessedJanuary6,2023.

15.SocietyofClinicalUltrasoundFellowships(SCUF).Programlist. Availableat: https://www.eusfellowships.com/programs AccessedJanuary6,2023.

16.PanacekEA.Survey-basedresearch:generalprinciples. AirMedJ. 2008;27(1):14–6.

17.AlerhandS,Situ-LacasseE,RamdinC,etal.Nationalsurveyofpoint-ofcareultrasoundscholarlytracksinemergencymedicineresidency programs. WestJEmergMed. 2021;22(5):1095–101.

18.HillJ,OgleK,GottliebM,etal.Educator’sblueprint:ahow-toguidefor collectingvalidityevidenceinsurvey-basedresearch. AEMEducTrain. 2022;6(6):e10835.

19.RyanMS,LomisKD,DeiorioNM,etal.Competency-basedmedical educationinanorm-referencedworld:arootcauseanalysisof challengestothecompetency-basedparadigminmedicalschool.

AcadMed. 2023;98(11):1251–60.

20.GoldhamerMEJ,Martinez-LageM,Black-SchafferWS,etal. Reimaginingtheclinicalcompetencycommitteetoenhanceeducation andprepareforcompetency-basedtime-variableadvancement.

JGenInternMed. 2022;37(9):2280–90.

21.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducation inemergencymedicalservices.2021.Availableat: AccessedApril29,2023.

22.AccreditationCouncilforGraduateMedicalEducation(ACGME). ACGMEprogramrequirementsforgraduatemedicaleducationincritical caremedicine.2022.Availableat: https://www.acgme.org/specialties/ internal-medicine/program-requirements-and-faqs-and-applications/ AccessedApril29,2023.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

ChangesinResidencyApplicantCancellationPatternswith VirtualInterviews:ASingle-siteAnalysis

MeryllBouldin,MD*

CarlyEastin,MD*†

RachaelFreeze-Ramsey,MD*

AmandaYoung,MD*

MeredithvonDohlen,MD*

LaurenEvans,MD*

TravisEastin,MD,MS*

SarahGreenberger,MD*

*UniversityofArkansasforMedicalSciences,DepartmentofEmergencyMedicine, LittleRock,Arkansas

† UniversityofArkansasforMedicalSciences,DepartmentofPediatrics, SectionofEmergencyMedicine,Toxicology,andPharmacology, LittleRock,Arkansas

SectionEditors:AbraFant,MD,andWilliamEdwardSoares,MD

Submissionhistory:SubmittedOctober1,2023;RevisionreceivedFebruary7,2024;AcceptedFebruary21,2024

ElectronicallypublishedMarch14,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18487

Background: ResidencyprogramstransitionedtoprimarilyvirtualinterviewsduetotheCOVID-19 pandemic.Thisshiftraisedquestionsregardingexpectationsandpatternsofapplicantcancellation timeliness.Thepurposeofthisstudywastoexaminechangesinapplicantcancellationsafter transitioningtovirtualinterviews.

Methods: Thiswasaretrospectivecohortstudyofinterviewdatafromathree-yearemergencymedicine residencyatatertiary-careacademicmedicalcenter.UsingarchiveddatafromInterviewBroker,we examinedschedulingpatternsbetweenonein-person(2019–2020)andtwovirtualinterviewcohorts (2020–2021and2021–2022).Ouroutcomesweretheoverallcancellationratesrelativetointerviewslots aswellastheproportionofcancellationsthatoccurredwithin7or14daysoftheinterviewdate.

Results: Therewere453interviewslotsand568applicantsinvited.Overall,applicantscanceled17.1% ofscheduledinterviews.Comparedwithin-personinterviews,applicantscanceledsignificantlyfewer virtualinterviews(inperson:40/128(31.3%),virtualyear1:22/178(12.4%),virtualyear2:15/143 (10.5%), P = 0.001).Conversely,applicantscanceledsignificantlymorevirtualinterviewswithinboththe 14-daythreshold(inperson:8/40(20%),virtualyear1:12/22(55.5%),virtualyear2:12/15(80%), P < 0.001)andthe7-daythreshold(inperson:0/40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004).

Conclusion: Whilelimited,atoursite,changingtoavirtualinterviewformatcorrelatedwithfewer cancellationsoverall.Theproportionofcancellationswithin14dayswasmuchhigherduring virtualinterviewseasons,withmostcancellationsoccurringduringthattimeframe.Additional studiesareneededtodeterminetheeffectsofcancellationpatternsonemergencymedicinerecruitment. [WestJEmergMed.2024;25(2)181–185.]

INTRODUCTION

Historically,residencyapplicantstraveledtoUSprograms forin-personinterviews.In2020,theCOVID-19pandemicled theCoalitionforPhysician Accountability(COPA)to recommendthatresidencyprogramsconductonlyvirtual

interviews.1 Proponentsofvirtualinterviewscitedcostand safetyaspotentialupsides,andapplicantshavereportedoverall satisfactionwithvirtualinterviewsandmoreadvantagesthan barriers.2–4 However,programshaveexpressedcontinued doubtsaboutsomeaspectsofvirtualrecruitment.2

Evenbeforethepandemic,therewerenoestablishedrules acrossspecialtiesregardinganacceptabletimeframefor interviewcancellations.Foremergencymedicine,the EmergencyMedicineResidentAssociation(EMRA) recommendedatleasttwoweeks’ noticeintheir2019 “EMRAandCORDStudentAdvisingGuide.”5 In2020,the firstyearofvirtualinterviews,emailcommunicationonthe CouncilofResidencyDirectorsinEmergencyMedicine (CORD)listservsuggestedthatprogramdirectors’ acceptablecancellationthresholdsrangedfrom48hoursto 10dayspriortotheinterviewdate.6 Ultimately,CORD statedthatsevendayswasrecommendedforapplicantsina 2020blogpostaboutinterviewingduringthepandemic, whileotherpublicationsstillrecommendedtwoweeks.7,8 Currently,the2023CORDApplicationProcess ImprovementCommitteeandthe2022–2023National ResidentMatchingProgram(NRMP)agreementhave advisedapplicantstocancelnolaterthan1–2weeksbefore theirinterviewdates.9,10

Virtualinterviewsmaybeheretostay,asevidencedby recentCOPAandAssociationofAmericanMedicalColleges (AAMC)statements,aswellasthe2023-24CORD guidelines.11–13 Understandingpatternsofvirtualinterview cancellationbehaviormayhelpprogramdirectors, applicants,andtheiradvisorsprepareforasuccessfulMatch. Tocharacterizetheeffectsofvirtualrecruitmentoninterview cancellations,wecomparedin-personinterviewcancellation patternstothoseofvirtualrecruitmentcyclesatour academicemergencymedicine(EM)residency.

METHODS

Thiswasaretrospectivecohortstudyatathree-yearEM residencysponsoredbyatertiary-care,academicmedical centerinanurbansettinginthesouth-centralUnitedStates. Thisresidencyisanestablishedprogram(foundedin1984) withaclasssizeof10residentsperyear,whichincreasedto12 residentsforthe2022Match.TheUniversityofArkansasfor MedicalSciencesInstitutionalReviewBoard(IRB) approvedthisstudyinexemptstatus.

Ourprogrambeganusingtheonlineinterviewscheduling softwareInterviewBroker(TheTenthNerve,LLC,Lewes, DE;www.interviewbroker.com)inFall2019toinvite applicantstointerview.InFall2020,interviewstransitioned frominpersontovirtualandadditionalslotswereadded, withCORDcontinuingtorecommendvirtualinterviewsfor EMresidenciesinsubsequentcycles.Similartoin-person interviews,applicantsforvirtualinterviewsareinvitedina 1:1applicanttoslotratioandgiven48hourstorespond beforeanotherapplicantisinvited.

UsingarchiveddatafromInterviewBroker,weexamined schedulingpatternsbetweenthein-personinterviewcohort (2019–2020season)andtwovirtualinterviewcohorts (VirtualYear1:2020–2021andVirtualYear2:2021–2022). Unfortunately,cancellationdatapriortotheinitiationof

InterviewBrokeratoursitewasnotavailable.Asingle investigatorabstracteddatafromInterviewBrokerin aggregateformbyacademicyearusingoverallcountsof relevantvariables,includingnumberofinterviewslots,days, invitations,interviewsscheduled/unscheduled(ie,no applicantresponsereceived)/declined,cancellations,andthe timingofthosecancellationsrelativetotheinterviewdate. Wedefinedaninterviewcancellationasaninterviewthatwas scheduled,canceled,andneverrescheduled;interviewsthat wererescheduledwereconsideredcompleted.Demographic variableswerenotavailableasInterviewBrokeronlyrecords thestudent’snameandAAMCID;accessingadditional informationwouldhaverequiredqueryingtheElectronic ResidencyApplicationService,whichwasnotcoveredinour exemptIRBagreement.

Ouroutcomesweretheoverallproportionofinterview cancellationsrelativetointerviewslots,aswellasthe proportionofinterviewcancellationsthatoccurredwithin 14daysoftheinterviewdateandwithinsevendaysofthe interviewdate.Descriptivestatisticswereperformed.We performedcomparisonsusingchi-squaredortheFisherexact testassomeobservationswereuncommon.Allcomparisons weretwo-sidedwith ɑ = 0.05.Analyseswereperformedusing SPSSStatisticsforMacintoshVersion28.0(IBM Corporation,Armonk,NY).

RESULTS

Overthreeyears,therewere453interviewslotsand568 applicantsinvited.Mostoftheinterviewslotswerevirtual (71.7%).Overall,theprogramsentout1.25interview applicationsperslotandapplicantscanceled17.1%of scheduledinterviews(Table1).Wefoundasignificant decreaseintheproportionofoverallcancellationsrelativeto filledinterviewslots,with40/128(31.3%),22/178(12.4%), and15/143(10.5%)cancellationsforin-person,virtualyear 1,andvirtualyear2,respectively(P < 0.001).When analyzedfurtherandadjustingformultiplecomparisons,the decreasewassignificantwhencomparinginpersonvs.either virtualyear,butnotwhencomparingthetwovirtualyears.

Whilefewerinterviewswerecanceled,theproportionof virtualinterviewcancellationsthatoccurredwithin14days oftheinterviewdatewassignificantlyhigher(inperson:8/40 (20%),virtualyear1:12/22(55.5%),virtualyear2:12/15 (80%), P < 0.001).Similarly,morevirtualinterviewswere canceledwithinsevendaysoftheinterviewdate(inperson:0/ 40(0%),virtualyear1:3/22(13.6%),virtualyear2:4/15 (26.7%), P = 0.004),althoughthesenumberswerelow overall.Inboththe14and7daycancellationanalyses,these dataindicatedayear-over-yearincrease,meaninginboth14 and7daycomparisonswesawasignificantincreasein cancellationsbetweeninpersonandvirtualyear1,andagain sawasignificantincreasebetweenvirtualyear1andvirtual year2.See Figures1 and 2 forgraphicalbreakdownofthe

InterviewsandCancellations

InterviewgroupInpersonVirtualyear1Virtualyear2

Numberofinterviewdays151616

Numberofinterviewslots128180145

Numberofapplicantsinvited195206167

Numberofinvitationsperinterviewslot1.521.141.15

Totalinterviewslots filled128178143

Numberofunscheduledinvitations(ie,noapplicantresponsereceived)1413

Numberwhodeclinedwithoutscheduling1356

Overallcancellations(%ofscheduled)40(31.3%)22(12.4%)15(10.5%)

Numberwhocanceled < 7days(%ofcanceled)0(0%)3(13.6%)4(26.7%)

Numberwhocanceled7–14days(%ofcanceled)8(20%)9(40.9%)8(53.3%)

Numberwhocanceled >14days(%ofcanceled)32(80%)10(45.5%)3(20.0%)

Overalldeclined,unscheduled,orcanceled(%oftotalinvited)67(34.4%)28(13.6%)24(14.3%)

overalldistributionofinvitedapplicantsandinterview cancellationrates.

DISCUSSION

Comparedwithin-personinterviews,applicantstoour programwerelesslikelytocanceltheirvirtualinterview.Of thosewhodidcancel,severalvirtualapplicantscanceled withinsevendays,andmostcancellationsoccurredwithin 14daysoftheinterviewdate.Forin-personinterviews, applicantsweretraditionallyinstructedtocancelassoonas possibleandatleasttwoweekspriortotheinterviewdate.5 Asdiscussedpreviously,recommendationsforEMvirtual interviewcancellationshaverangedfrom48hourstotwo weeks,withtheNRMPcurrentlyrecommendingatleast1–2 weeksinadvance.10 Ourresultssuggestthatshort-notice

cancellations(ie,lessthantwoweeks)bystudentsmaybe morecommoninthevirtualera.

Wearenotawareofliteratureregardingthespecifictiming ofvirtualinterviewcancellations,butour findingoffewer overallcancellationsisconsistentwithLewkowitzetal’ s findingsthatmaternal-fetalmedicinefellowshipvirtual interviewshadalowerrateofcancellationscomparedwith in-personinterviews(39.1%vs72.3%).14 Thiscouldstemfrom thereducedtimeandcostrequiredtointerviewvirtually.15,16

Unfortunately,fewerinterviewcancellationsoverallcould contributetointerviewhoardingandaninequitable distributionofinterviews.TheAAMCandsomespecialties haveexpressedconcernsabouthigherqualityapplicants receivinginvitationsforandschedulingexcessivelyhigh numbersofinterviewsandleavinglowertierstudentswith

Table1. Breakdownofin-personandvirtualinterviewcohorts;totalcountsprovidedunlessotherwisespecified.

Figure1. Overalldistributionofinvitedapplicants.

feweroptions.15,17 WhilethishasnotbeenstudiedinEM specifically,theEmergencyMedicineConsensusStatement onthe2020–2021ResidencyApplicationprocesssuggested aninterviewlimitof17interviewsandencouragedapplicants nottointerviewattheirless-preferredprogramsloweron theirlistto “maketheseslotsavailabletootherstudents,” indicatingapotentialconcernfortheeffectsofhoardingsuch as “peersnotmatchingand/orprogramsnot filling.”18

Short-noticeinterviewcancellationsposeafewother challengesforresidencyprograms.Previously, fillinganinpersoninterviewslotrequired findingareplacementwho couldstillarrangetraveltotheinterviewlocation,whichisno longerrelevantforvirtualinterviews.Nonetheless,the NRMPrequiresthatprogramsprovidenolessthan48hours forapplicantstorespondtointerviewinvitations.10 If applicantsarecancelingonlyafewdaysbeforeaninterview, fillingtheopenspotmaybeachallengesinceprograms cannotinvitemorethanoneapplicantatatimeperspot. Short-noticecancellationscanalsobeproblematicas interviewersmayhavetoreviewcandidates’ applications wellinadvanceoftheinterviewdate.Withshort-notice cancellations,thiscouldmeanlosttimeforinterviewerswho hadalreadyreviewedthoseapplicationsorinadequatetime toreviewthereplacements.

Conversely,programdirectorswanttoavoidinterviewing applicantswhoarenotinterestedintheirprogram,anda cancellation evenonshortnotice providesan opportunitytointerviewanapplicantwithgreaterinterestin theprogram.Inourcase,wehadonlyfouropeninterview spotsoverthe firsttwovirtualyears(twounfilledperyear), indicatingthatwe filledmostcanceledspots.Therefore, whilenoofficialopinionexists,programdirectorsmaynot mindshort-noticecancellationaslongastheinterview scheduleisfull.Infact,theymaypreferfortheapplicantnot tofeelpressuredtointerviewataprograminwhichtheyare uninterestedonlybecausetheyareconcernedabout canceling,withshortnoticebeingviewedasunprofessional. Asvirtualinterviewsappeartobeheretostay,understanding cancellationpatternswillbeimportantforprograms, especiallyinbalancingthetimingcancellationswithnew

invitationssoprogramscanideallymaintainafull interviewschedule.

LIMITATIONS

Thisstudywaslimitedtoonespecialtyatasingle institution,thereforethegeneralizabiltyofthese fi ndingsto otherinstitutionsorspecialtiesisunclear,especiallygiven thesmallsamplesizeandlimitedpre-postperiod.The changinglandscapeofEMresidencyrecruitmentmayalso affectthegeneralizabilityofthese fi ndings.Unfortunately, wehadonlyoneyearofin-personinterviewdataaswedid notkeeptheserecordspriortotheuseofInterviewBroker, whichcouldhaveintroducedbias.Wealsohadanincrease inresidentcomplementduringvirtualyear2,whichmay haveconfoundedtheresults.Unfortunately,wewere unabletoincludedemographicdata,whichmighthave helpedtoidentifyadditionalcancellationpatterns.Lastly, examiningtrendsinthosewhorescheduleinterviewswas notperformedinthisstudyandmaybeofvalueinfuture investigations,assomedownsidesdiscussedwithshortnoticecancellations(eg, fi llingemptyslots;havingtimeto reviewapplications)wouldstilloccurinapplicantswhoare reschedulingwithshortnotice.

CONCLUSION

Comparedwithin-personinterviewcycles,applicantsto ourresidencyprogramweresignificantlylesslikelytocancel virtualinterviews.However,themajorityofvirtual cancellationsthatdidoccurwerewithin14daysofthe interviewdateandnearlyone-fifthoccurredinunderseven days.Additionalstudies,ideallymultisitethatinclude applicantdemographicdata,areneededtodeterminehow cancellationpatternsaffectEMrecruitmentandmatch outcomesinthevirtualera.

AddressforCorrespondence:MeryllBouldin,MD,Universityof ArkansasforMedicalSciences,DepartmentofEmergency Medicine,4301WMarkhamSt.,Slot584,LittleRock,AR72205. Email: mebouldin@uams.edu

Figure2. Interviewcancellationsbylengthoftimefrominterview. d,day.

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Bouldinetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2021MovingAcrossInstitutionsforPost GraduateTraining.Finalreportandrecommendationsformedical educationinstitutionsofLCME-accredited,U.S.osteopathic,andnonU.S.medicalschoolapplicants.2020.Availableat: https://www.aamc. org/system/files/2020-05/covid19_Final_Recommendations_ 05112020.pdf.AccessedMay22,2022.

2.DavisMG,HaasMRC,GottliebM,etal.Zoominginversus flyingout: virtualresidencyinterviewsintheeraofCOVID-19. AEMEducTrain. 2020;4(4):443–6.

3.DomingoA,RdesinskiRE,StensonA,etal.Virtualresidencyinterviews: applicantperceptionsregardingvirtualintervieweffectiveness, advantages,andbarriers. JGradMedEduc. 2022;14(2):224–8.

4.Li-SauerwineS,WeygandtPL,SmylieL,etal.Themorethingschange themoretheystaythesame:factorsinfluencingemergency medicineresidencyselectioninthevirtualera. AEMEducTrain. 2023;7(6):e10921.

5.EmergencyMedicineResidentsAssociationJarouZ,HillmanEA, KellogA,etal.(Eds.). EMRAandCORDStudentAdvisingGuide: AnEvidence-basedApproachtoMatchinginEM. 2019.Availableat: https://www.emra.org/books/msadvisingguide/msag AccessedJanuary19,2024.

6.EmergencyMedicineProgramDirectors.Re:interviewcancellations. CORDListservprivateemailchain.LastaccessedJanuary19,2024.

7.GorgensS.TheEMinterviewseason:pandemicedition.2020. Availableat: https://cordemblog.com/2020/09/03/the-em-interviewseason-pandemic-edition/.AccessedDecember20,2023.

8.HopsonLR,EdensMA,GoodrichM,etal.Calmingtroubledwaters:a narrativereviewofchallengesandpotentialsolutionsintheresidency interviewofferprocess. WestJEmergMed. 2020;22(1):1–6.

9.CouncilofResidencyDirectorsinEmergencyMedicine.Application ProcessImprovementCommittee(Apic):BestPracticesforthe 2023–2024ResidencyApplicationandInterviewSeason.2023.

Availableat: https://www.cordem.org/siteassets/files/match/ apic-2023-2024-application-interview-season-best-practices.pdf

AccessedJuly26,2023.

10.NationalResidentMatchingProgram.MatchParticipationAgreement forPrograms.2023.Availableat: https://www.nrmp.org/wp-content/ uploads/2022/09/2023-MPA-Main-Match-Program-FINAL-3.pdf

AccessedJune28,2023.

11.CouncilofResidencyDirectors.2023CORDstatementonresidency interviews.2023.Availableat: https://www.cordem.org/siteassets/files/ board/adv.–position-statements/2023-cord-statement-on-residencyinterviews.pdf.AccessedJuly26,2023.

12.AAMC.InterviewsinGME:Wheredowegofromhere?2023. Availableat: https://www.aamc.org/about-us/mission-areas/ medical-education/interviews-gme-where-do-we-go-here

AccessedJuly23,2023.

13.TheCoalitionforPhysicianAccountability’sWorkGrouponMedical StudentsintheClassof2022MovingAcrossInstitutionsforInterviews forPostgraduateTraining.Recommendationson2021–22residency seasoninterviewingformedicaleducationinstitutionsconsidering applicantsfromLCME-accredited,U.S.osteopathic,andnon-U.S. medicalschools.Availableat: https://physicianaccountability.org/ wp-content/uploads/2021/08/Virtual-Rec_COVID-Only_Final.pdf AccessedJune22,2022.

14.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.

15.BoydCJ,AnanthasekarS,VernonR,etal.Interviewhoarding: disparitiesintheintegratedplasticsurgeryapplicationcycleinthe COVID-19pandemic. AnnPlastSurg. 2021;87(1):1–2.

16.GordonAM,ConwayCA,ShethBK,etal.Howdidcoronavirus-19 impacttheexpensesformedicalstudentsapplyingtoanorthopaedic surgeryresidencyin2020to2021? ClinOrthopRelatRes. 2022;480(3):443–51.

17.WhelanA.Openletteronresidencyinterviews.2020.Availableat: https://www.aamc.org/media/50291/download?utm_source=sfmc& utm_medium=Email&utm_campaign=ERAS&utm_content=Interviews AccessedJanuary19,2024.

18.FarcyD,JungJ,AintablianH,etal.Consensusstatementonthe 2020–2021residencyapplicationprocessforUSmedicalstudents planningcareersinemergencymedicineinthemainresidency match.2020. https://www.emra.org/be-involved/be-an-advocate/ working-for-you/residency-application-process AccessedJanuary22,2024.

Geography:ANationalSurvey

AlineBaghdassarian,MD,MPH*†

JessicaA.Bailey,MD‡

DeryaCaglar,MD§∥

MichelleEckerle,MD,MPH¶#

AndreaFang,MD**

KatherineMcVety,MD††‡‡

ThuyNgo,DO,MEd§§

JerriA.Rose,MD∥∥¶¶

CindyGanisRoskind,MD##

MelissaM.Tavarez,MD,MS***

FrancesTurcotteBenedict,MD,MPH†††‡‡‡

JoshuaNagler,MD,MHPEd§§§∥∥∥

Authorscontinuedatendofpaper

*InovaL.J.MurphyChildren'sHospital,DepartmentofPediatrics, FallsChurch,Virginia

† UniversityofVirginia,SchoolofEducation,Charlottesville,Virginia ‡ OregonHealth&ScienceUniversity,DepartmentofPediatricsand EmergencyMedicine,Portland,Oregon

§ UniversityofWashington,DepartmentofPediatrics,Seattle,Washington

∥ SeattleChildren’sHospital,DepartmentofPediatrics, Seattle,Washington

¶ UniversityofCincinnatiCollegeofMedicine,DepartmentofPediatrics, Cincinnati,Ohio

# CincinnatiChildren’sHospital,DepartmentofPediatrics,Cincinnati,Ohio Affi liationscontinuedatendofpaper

SectionEditors:KendraParekh,MD,andChrisMerritt,MD

Submissionhistory:SubmittedNovember15,2023;RevisionreceivedFebruary10,2024;AcceptedFebruary21,2024

ElectronicallypublishedMarch14,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18581

Introduction: Virtualinterviews(VI)arenowapermanentpartofpediatricemergencymedicine(PEM) recruitment,especiallygiventhecostandequityadvantages.Yetinabilitytovisitprogramsinpersoncan impactdecision-making,leadingapplicantstoapplytomoreprograms.Moreover,thecostadvantagesof VImayencourageapplicantstoapplytoprogramsfartherawaythantheymightotherwisehavebeen willingorabletotravel.Thiscouldcreateunnecessarystrainonprograms.Weconductedthisstudyto determinewhetherPEMfellowshipapplicantswouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedtoin-personinterviews(2018and2019).

Methods: WeconductedananonymousnationalsurveyofallPEMfellowscomparingtwocohorts: currentfellowswhointerviewedinperson(appliedin2018/2019)andfellowswhounderwentVIsin2020/ 2021(currentfellowsandthoserecentlymatchedin2021).ThestudytookplaceinMarch–April2022. Questionsfocusedongeographicconsiderationsduringinterviewsandthematch.Weuseddescriptive statistics,chi-squareand t-testsforanalysis.

Results: Overallresponseratewas42%(231/550);32%(n = 74)interviewedinpersonand68% (n = 157)virtually.Fellowsappliedtoamedianof4/6geographicregions(interquartilerange2,5).Most appliedforfellowshipbothinthesameregionasresidency(216,93%)andoutside(192,83%).Onlythe PacificregionsawastatisticallysignificantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02). Therewasnostatisticaldifferenceinthenumberofprogramsappliedtoduringin-personvsVI (meandifference(95%confidenceinterval0.72, 2.8 – 4.2).Amajoritymatchedintheirpreferred statebothduringVI(60.4%)andin-personinterviews(65.7%).Thedifferencewasnotstatistically significant(P = 0.45).

Conclusion: WhilemorePEMfellowshipapplicantsappliedoutsidethegeographicareawheretheir residencywasandtothePacificregion,therewasnooverallincreaseinthenumberofprogramsor geographicareasPEMapplicantsappliedtoduringVIascomparedtoin-personinterviewseasons.As thiswasthe firsttwoyearsofVI,ongoingdatacollectionwillfurtheridentifytrendsandtheimpactofVI. [WestJEmergMed.2024;25(2)186–190.]

INTRODUCTION

Since2020,virtualinterviews(VI)havebeenpreferredfor traineerecruitment.1 Withthebenefitsoflowercostand greaterequity,itislikelytoremainapermanentpartof recruitment,despiteageneralpreferenceforface-to-face interviews.2–5 TheVIprocessandassociatedperceptions havebeendescribedintheliterature.2,3,6–9 Theinabilityto visitaprograminpersoncanimpactdecision-makingduring ranking,4,10–14 andanincreasednumberofapplications couldcreateunduestrainonprograms.15–17

Geographiclocation,senseof “fit,” andprogram leadershipweredescribedasmajorcontributorsto applicants’ rankpreference.18 Anationalcohortofpediatric emergencymedicineprogramdirectors(PEMPD),inajoint statement,raisedconcernthatVIcouldleadapplicantsto applytomoreprogramsandtoprogramsfartherawaythan theymaybewillingorabletotravel.10 Weconductedthis studytodeterminewhetherPEMfellowshipapplicants wouldapplytoalargernumberofprogramsandindifferent geographicpatternswithVI(2020and2021)ascomparedto in-personinterviews(2018and2019).

METHODS

DesignandParticipants

Thiswasananonymous,self-administered,crosssectional,web-basedsurveyofPEMfellowsinthe UnitedStates.Participationwasvoluntary,andnoincentive wasprovidedforcompletion.Thestudywasexempted bytheinstitutionalreviewboardatYaleUniversity, withinformedconsentimpliedbycompletionofthesurvey byparticipants.

SurveyDevelopment

Thesurveyquestionnairewasdevelopedthroughiterative feedbackandamodifiedDelphiprocesstodetermine itemimportance.ThirteenPEMPDswithexpertisein performanceandevaluationparticipatedinmultiplerounds ofrevisionsandediting.Pilottestingwasconductedwithtwo pediatrichospitalmedicinefellowswhohadappliedtothe matchduringVIsandtwopediatricchiefresidentswhowere alsointerviewingforfellowshipsusingVI,atthelead institution.Revisionsweremadebasedonpilotfeedback (surveyprovidedin SupplementaryAppendix1).Thesurvey includedmultiple-choicequestionsaboutlocationof residency,statesappliedtoandinterviewedforfellowship, preferredlocationforfellowship,statesvisitedinpersonfor thepurposeofthematch,andstatematchedin.Italsoasked fellowstoindicatestatesofresidenceofimmediatefamily (parents,siblings,orpartners)andaboutcompellingreasons (otherthanfamily)thatmayhaveledfellowstofavorastate orregion(freetext).Geographicregionsweredefinedas Northeast,Southeast,Midwest,Southwest,Rocky Mountain,andPacificregions.19

Whatdowealreadyknowaboutthisissue?

Virtualinterviewsareapermanentpartof recruitment.Theyoffercostandequity advantageswhileposingchallengestoboth applicantsandprograms.

Whatwastheresearchquestion?

DidPEMfellowshipapplicantsapplytoa largernumberofprogramsandindifferent geographicpatternswithVIascomparedto in-personinterviews?

Whatwasthemajor findingofthestudy?

VIdidnothaveasigni fi cantimpactonthe numberofprogramsorgeographicareas applicantsappliedto.

SurveyDistribution

ThesurveywasreviewedandapprovedbytheAmerican AcademyofPediatrics(AAP)SectiononEmergency Medicine(SOEM)PDsurveysubcommitteepriorto distributiononQualtrics(Qualtrics,Provo,UT)toallPEM PDs,viatheAAPSoEMPDCommitteelistserv.ThePDs forwardedthesurveylinktotheircurrentandincoming fellows(thoserecentlymatchedtostartinJuly2022).Each PDcompletedaseparatequestionnaireindicatingthetotal numberofcurrentandrecentlymatchedfellowstowhom theyforwardedthesurvey.

Analysis

Participantsweredividedintotwogroups:VI(2020or 2021)andinperson(2018or2019).Weperformed descriptivestatisticsincludingfrequencies,percentages, meanswithstandarddeviations,andmedianswith interquartilerange(IQR).Chi-squaretestscompared categoricalvariablesandt-tests,continuousvariableswith 95%confidenceintervals(CI).Weconsideredatwo-tailed alphaof <0.05tobestatisticallysignificant.Weconducted analysesinIBMSPSSStatisticsversion28(IBM Corporation,Armonk,NY).

RESULTS

ThePDsreportedthattheyforwardedthesurveyto406 currentfellowsand144incomingfellows.Theresponserate forcurrentfellowswas35%(143/406)andforincoming fellows,61%(88/144).Overall,theresponseratewas42% (231/550).Ofthetotalrespondents,62%(143/231)were currentfellowsand38%(88/231)incoming.Twofellows (1%)didnotcompleteresidencyintheUS,and12(5%) appliedtoPEMfellowshipmorethanonce.

Baghdassarianetal. VIandthePEMMatchGeography

AllincomingfellowshadundergoneVI,whereas 48%ofthecurrentfellowshadundergoneVI(69/143). Overall,32%ofrespondents(74/231)interviewedinperson and68%(157/213)virtually.Therewasnostatistical differenceinthenumberofprogramsappliedtoduringinpersonvsVI(meandifference(95%CI):.72[ 2.8,4.2]) (Appendix2Table).

Datadescribingthegeographictrainingandlocation preferenceofparticipants arepresentedinthetablein appendix2.Fellowsappliedtoamedianoffourofthe sixgeographicregions(IQR2,5).Mostparticipantsapplied forfellowshipinthesamegeographicregionastheirresidency (216,93%)andoutsidetheirresidencyregionaswell (192,83%).OnlythePacificregionsawastatistically significantincreaseinapplicantsduringVI(59.9%vs43.2%, P = 0.02)(Table1).

Lessthanhalfofrespondentshadimmediatefamily memberslivinginthesamestateasresidency(N = 111,48%), fellowship(N = 90,39%),ortheirpreferredmatch state(N = 95,41%).Compellingreasonstoapplyto anareaincludedfamiliaritywithlocation(N = 128,55%); similarlocationtoresidency(N = 65,28%);anda desiretotraininanewarea(N = 53,23%).Partner’ s employmentwasanimportantfactorfor89(38%),salary

andcostoflivingfor76(33%),andschoolforchildren for20(9%).

DISCUSSION

OurresultsshowthatVImayallowsomecandidatesto exploreandconsiderregionstheymaynothaveotherwise duetologisticalor financialconstraints,withoutincreasing thenumberofprograms,regionsorstatestheyapplyto. Theseresultsareconsistentwiththe2021NRMPsurvey where52%reportednoimpactoftheVIonthenumberof programsappliedto.5 Residencyprogramshavereportedan increaseinmatchedinternalcandidatesduringVI.11,12,20,21 InPEM,apre-pandemicstudyofPDsshowedthat29%of fellowscompletedresidencyatthesameinstitution.22 While wedidnothavedataattheinstitutionallevel,therewasno significantincreaseinfellowsmatchingwithinthestateof theirresidencyprogramwithVI.ThissuggeststhatVIwere notasignificantdetrimenttoapplicantsrankingprograms andgeographicareas,despitetheabsenceofopportunitiesto meetinpersonandvisitprograms.Thisalsoallowsprograms tohaveaccesstoalargerandpotentiallymorediversepool ofcandidates.9

Proximitytofamilywasnotasignificantconsiderationfor mostapplicantsandwasnotimpactedbyVI.Residency

In-person interviews(N = 74) Virtualinterviews (N = 157)

Statisticalsignificance (P valueor95%CI)

Appliedtoregionforfellowship,N(%)

Northeast59(79.7)123(78.3)0.81

Southeast41(55.4)102(65)0.16

Midwest50(67.6)111(70.7)0.63

Southwest38(51.4)86(54.8)0.63

RockyMountains31(41.9)73(46.5)0.51

Pacific32(43.2)94(59.9)0.02

Appliedtosamegeographicregionas residency,N(%)

Appliedoutsidegeographicregionas residency,N(%)

71(98.6)145(94.8).278

56(77.8)136(88.9)0.03

Numberofregionsappliedto,mean(SD)3.4(1.8)3.8(1.8)Meandifference(95%CI):.36( .15,.89)

Numberofstatesappliedto,mean(SD)9(7.3)9.7(6.8)Meandifference(95%CI):.73( 1.2,2.7)

Numberofprogramsappliedto,mean(SD)13.3(12.8)14(12.5)Meandifference(95%CI):.72( 2.8,4.2)

Numberofprogramsinterviewedat, mean(SD)

Matchedinpreferredstate,N(%)46(65.7)84(60.4)0.46

Matchedinsamestateasresidency,N(%)31(42%)59(38%)0.58

Preferredtomatchinstatewithimmediate familypresent,N(%)

Wenttovisitstate/program,N(%)9(14)23(17)0.61 CI,confidenceinterval.

3.1( 1.7,1.1)

Table1. Influenceofvirtualinterviewsonapplicantbehaviorandoutcomes.

applicantsreportedgeography,qualityoflife,casevariety, curriculum,institutionalreputation,expertiseinareasof interest,andprogramsizeaskeyfactors.23 Applicantsto PEMhighlightedfamiliaritywiththeregionorwantingto exploreanewareaasfactorsforexploringprogramsin differentregions.

LIMITATIONS

Limitationsofthisstudyincludethesmallerresponserate ofthecurrentfellowsascomparedtotheincomingfellows. Thislowresponseratelimitedthesamplesizeofthein-person cohort,impactingthestatisticalsignificanceofourresults. Thisdifferentialresponsefromtheincomingfellowsmay havebeenduetodesirabilitybiaswherethiscohortof applicantsmayhavetendedtostatethattheymatchedin theirpreferredstate.Tominimizethis,wedesignedourstudy tobefullyanonymousandself-administered,andthe questionswerewordedtoretainobjectivityoftheanswers. Respondentsmayalsohaveexperiencedrecallbiasregarding thestatesandprogramstowhichtheyapplied.Thisbias couldpotentiallyhavecontributedtothelowerresponserate amongthecurrentfellowswhohadinterviewedin2018/2019, 3–4yearspriortothesurveydate,comparedtothemore recentapplicantswhohadamorerecentrecollectionofthe questionsaskedinthesurvey.

Anotherlimitationisthatwedidn’texplicitlyaskthetotal numberoffellowsineachclasscohort;however,sincethe PEMfellowshipclasssizeintheUSdoesn’tvarysignificantly fromyeartoyear(byvirtueoftheapprovedfellowship positionsavailable),thedenominatorisexpectedtobe relativelyconstant.

Thisstudywasnotdesignedtolookattheratesof applicationstoindividualprogramsnorassessthepostmatchopinionsofprogramsandfellowsregardingthe resultsofthematch.Thisinformationwouldprovidea deeperinsightintotheimpactoftherecruitmentprocess; however,itisalsopronetobiasasfellowsonlyexperience trainingatasingleinstitution.Wealsodidnottakeinto considerationtheconcentrationofPEMprogramsby regionortheavailablefellowshipslotsperprogramor region.However,theobjectiveofthisstudywastolookat thedifferencesbeforeandduringVIs,andtherewasnota signi fi cantchangeinavailablefellowshipslotsorprograms duringtheseyears.Asthenumberofpediatricfellowship applicantsrises,furtherinvestigationintotheimpactofVIs isnecessarytogainadeeperunderstandingofits implicationsandtooptimizethisprocessbothfor applicantsandprograms. 24

CONCLUSION

WhilemorePEMfellowshipapplicantsappliedoutside thegeographicareawheretheirresidencywasandtothe Pacificregion,therewasnooverallincreaseinthenumberof

programsorgeographicareasthatPEMapplicantsapplied toduringVIduringthe firsttwoyearsofitsinstitution,as comparedtoin-personinterviewseasons.Ongoing monitoringoftheinterviewandmatchseasonswillhelp identifyfuturetrendsandimpactofVIs.

AUTHORSANDAFFILIATIONSCONTINUED

MelissaL.Langhan,MD,MHS¶¶¶

** StanfordUniversitySchoolofMedicine,Departmentof PediatricEmergencyMedicine,PaloAlto,California

††Children’sHospitalofMichigan,DepartmentofPediatrics, Detroit,Michigan

‡‡CentralMichiganUniversity,SchoolofMedicine, DepartmentofPediatrics,Detroit,Michigan

§§JohnsHopkinsUniversity,SchoolofMedicine,Department ofPediatrics,Baltimore,Maryland

∥∥RainbowBabies&Children’sHospital,Departmentof Pediatrics,Cleveland,Ohio

¶¶CaseWesternReserveUniversity,SchoolofMedicine, DepartmentofPediatrics,Cleveland,Ohio

##ColumbiaUniversityIrvingMedicalCenter,Pediatricsin EmergencyMedicine,NewYork,NewYork

*** UniversityofPittsburgh,SchoolofMedicine,Departmentof Pediatrics,Pittsburgh,Pennsylvania

†††UniversityofMissouriofKansasCitySchoolofMedicine, DepartmentofPediatrics,KansasCity,Missouri

‡‡‡UniversityofKansasMedicalCenter,KansasCity, Missouri

§§§BostonChildren’sHospital,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts

∥∥∥HarvardMedicalSchool,DepartmentofPediatricsand EmergencyMedicine,Boston,Massachusetts

¶¶¶YaleUniversitySchoolofMedicine,Departmentof PediatricsandEmergencyMedicine,NewHaven, Connecticut

AddressforCorrespondence:AlineBaghdassarian,MD,MPH, InovaFairfaxL.JMurphyChildren’sHospital,3300GallowsRd., FallsChurch,VA22042-3300.Email: aline.baghdassarian@ inova.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Baghdassarianetal.Thisisanopenaccess articledistributedinaccordancewiththetermsoftheCreative CommonsAttribution(CCBY4.0)License.See: http:// creativecommons.org/licenses/by/4.0/

REFERENCES

1.AssociationofPediatricProgramDirectors,CouncilofPediatric Subspecialties,AssociationofMedicalSchoolPediatricDepartment Chairsetal.APPD/CoPS/AMSPDC/NextGenPediatricianslettertoour pediatricscommunityaboutthefellowshiprecruitmentprocess.2021. Availableat: https://downloads.aap.org/AAP/PDF/2021_05_19_-_ Pediatric_Fellowship_Recruitment_Recommendations_Final.pdf AccessedJuly15,2022.

2.VanDerLaanL,GeorgeR,NesiamaJA,etal.Virtualinterviewingfor pediatricemergencymedicinefellowship-anationalsurvey. Pediatr EmergCare. 2022;38(4):e1207–12.

3.GuptaS,GrierArthurL,ChandlerN,etal.Isthechanginglandscapeof fellowshiprecruitmentduringCOVID-19heretostay? JPediatrSurg. 2022;57(10):445–50.

4.DasAJ,AugustinRC,CorbelliJA,etal.Residencyandfellowship programleaders’ perceptionsofvirtualrecruitmentandinterviewing. JGradMedEduc. 2022;14(6):710–3.

5.NationalResidentMatchingProgram.2021Applicantandprogram directorsurvey findings:impactofthevirtualexperienceonthetransition toresidency.2021.Availableat: https://www.nrmp.org/wp-content/ uploads/2021/08/Research-Brief-Virtual-Experience-2021-FINAL.pdf AccessedJuly15,2022.

6.WolffMandBurrowsH.Planningforvirtualinterviews:residency recruitmentduringapandemic. AcadPediatr. 2021;21(1):24–31.

7.BernsteinSA,GuA,ChretienKC,etal.Graduatemedicaleducation virtualinterviewsandrecruitmentintheeraofCOVID-19. JGradMedEduc. 2020;12(5):557–60.

8.McCainC,KempB,BaierMB,etal.Aframeworkforthevirtualmedical interviewprocess:considerationsfortheapplicantandtheinterviewer. OchsnerJournal. 2022;22(1):61–70.

9.PetersenTL,KingJC,FussellJJ,etal.Benefitsandlimitationsofvirtual recruitment:perspectivesfromsubspecialitydirectors. Pediatrics. 2022;150(4):e2022056735.

10.AllisterL,BaghdassarianA,CaglarD,etal.Pediatricemergency medicinefellowshipdirectors’ 2021collectivestatementonvirtual interviewsandsecondlooks. PediatrEmergCare. 2021;37(11):585–7.

11.EderleA,ShahriariS,WhisonantC,etal.TheimpactofCOVID-19on thedermatologymatch:anincreaseinthenumberofstudentsmatching athomeprograms. DermatolOnlineJ. 2021;27(9).

12.FaletskyA,ZitkovskyH,GuoL.TheimpactofCOVID-19on plasticsurgeryhomeprogrammatchrates. AnnPlastSurg. 2022;88(1):4–6.

13.MulcahyCF,TerhaarSJ,BoulosS,etal.Didmoreotolaryngology residencyapplicantsmatchattheirhomeinstitutionsin2021? InvestigatingtheimpactoftheCOVID-19pandemic. AnnOtolRhinol Laryngol. 2022;131(12):1375–80.

14.QuinnA,MannE,RaikinJ,etal.PD24-06Theeffectofthe COVID-19pandemiconurologymatchbylocation. JUrol. 2021;206(Suppl3):e427–8.

15.InclanPM,WoiczikMR,CummingsJ,etal.Virtualpediatricorthopaedic fellowshipinterviewsduringthepandemic:Whatdidtheapplicantsand programsthink? JPediatrOrthop. 2022;42(7):e806–10.

16.LewkowitzAK,RamseyPS,BurrellD,etal.Effectofvirtualinterviewing onapplicantapproachtoandperspectiveofthematernal-fetal medicinesubspecialtyfellowshipmatch. AmJObstetGynecolMFM. 2021;3(3):100326.

17.ReamMAandThompson-StoneR.Virtualresidencyinterview experience:thechildneurologyresidencyprogramperspective. PediatrNeurol. 2022;126:3–8.

18.DiGiustoM,LupaMC,CorridoreM,etal.TheimpactoftheCOVID-19 pandemiconthe2020pediatricanesthesiologyfellowship applicationcycle:asurveyofapplicants. PaediatrAnaesth. 2021;31(9):968–76.

19.STUDYGUIDEUSII.2cGeographyThemes.Availableat: https://www.solpass.org/7ss/standards/StudyUSII.2c.htm AccessedFebruary4,2024.

20.WhisonantCT,ShahriariSR,HarrisonJ,etal.Evaluatingtheintegrated plasticsurgeryresidencymatchduringthenovelcoronaviruspandemic. Cureus. 2021;13(8):e16988.

21.CotnerCE,MercadanteSF,SheaJA.Assessingtheimpactofthe COVID-19pandemicongeographicresidencyplacementrelativeto medicalschoollocation. JGradMedEduc. 2022;14(1)108–11.

22.BradleyT,ClingenpeelJM,PoirierM.Internalapplicantstopediatric emergencymedicinefellowshipsandcurrentuseofthenationalresident matchingprogrammatchasurveyoffellowshipdirectors. PediatrEmerg Care. 2015;31(7):487–92.

23.LoveJN,HowellJM,HegartyCB,etal.Factorsthatinfluence medicalstudentselectionofanemergencymedicineresidency program:Implicationsfortrainingprograms. AcadEmergMed. 2012;19(4):455–60.

24.TheAmericanBoardofPediatrics.Yearlygrowthinpediatricfellowsby subspecialtybydemographicsandprogramcharacteristics.Available at: https://www.abp.org/content/yearly-growth-pediatric-fellowssubspecialty-demographics-and-program-characteristics AccessedJanuary26,2023.

ORIGINAL RESEARCH

AnalysisofAnonymousStudentNarrativesAboutExperiences withEmergencyMedicineResidencyPrograms

MollyEstes,MD*

JacobGarcia,MD†

RonnieRen,MD,MSEd‡§

MarkOlaf,DO∥

ShannonMoffett,MD¶

MichaelGaluska,MD#

XiaoChiZhang,MD,MS**

*LomaLindaUniversity,DepartmentofEmergencyMedicine,LomaLinda,California

† MountSinaiMorningside/West,DepartmentofEmergencyMedicine,NewYork,NewYork

‡ UniversityofMassachusetts,DepartmentofEmergencyMedicine,Boston,Massachusetts

§ UniversityofFloridaHealthShandsHospital,DepartmentofEmergencyMedicine, Gainesville,Florida

∥ GeisingerCommonwealthSchoolofMedicine,DepartmentofEmergencyMedicine, Scranton,Pennsylvania

¶ RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey

# ConemaughMemorialMedicalCenter,DepartmentofEmergencyMedicine, Johnstown,Pennsylvania

**ThomasJeffersonUniversity,DepartmentofEmergencyMedicine, Philadelphia,Pennsylvania

SectionEditors:JeffreyDruck,MD,andMichaelGottlieb,MD

Submissionhistory:SubmittedMarch15,2023;RevisionreceivedOctober18,2023;AcceptedNovember21,2023

ElectronicallypublishedFebruary5,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17973

Background: Academicemergencymedicine(EM)communitieshaveviewedanonymousonline communities(AOC)suchasRedditorspecialty-specific “applicantspreadsheets” aspooradvisingresources. Despitethis,robustEMAOCsexist,withlargeuserbasesandheavyreadership.Insightsaboutapplicants’ authenticexperiencescanbecriticalforapplicantsandprogramleadershipdecision-making.Todate,there arenoEMstudiestoqualitativelyassessEMAOCnarrativesduringtheapplicationcycle.Ourgoalwasto performaqualitativeanalysisofstudents’ EMprogramexperiencesthroughapubliclyavailableAOC.

Methods: Thiswasaqualitativeanalysisofapubliclyavailable,time-stamped,user-lockedAOC dataset: “Official2020–2021EmergencyMedicineApplicantSpreadsheet.” Weextractedandthendeidentifiedalldatafromselectedsub-sheetsentitled “VirtualInterviewImpressions” and “Rotation Impressions.” Fourinvestigatorsusedconstantcomparativemethodtoanalyzethedatainductively,and theysubsequentlymettogeneratecommonthemesdiscussedbystudents.Preliminarythematic analysiswasconductedonarandomsampleof 37/183(20%)independentnarrativestocreatetheinitial codebook.Thiswasusedandupdatediterativelytoanalyzetheentirenarrativesetconsistingof841 discretestatements.Finally,twouniquecodeswerecreatedtodistinguishwhethertheidentifiedsubthemes,orprogramattributes,werelikely “modifiable” or “non-modifiable.”

Results: Weidentifiedsixmajorthemes:livingandworkingconditions;interpersonalrelationships;learning experiences,postgraduatereadiness,andonline/virtualsupplements.Commonsub-themesincludedpatient population(13%);residentpersonality(7%);programleadershippersonality(7%);relationshipwithfaculty/ leadership(6%);geography(4%);practicesetting(4%);programreputation(4%),andpostgraduateyear-3 experiences(4%).Modifiablesub-themesoutnumberednon-modifiablesub-themes,60.7%to39.3%.

Conclusion: Inthisanalysisofselectedmedicalstudents’ narrativesinanAOC,themajorityofidentified themesrepresentedtopicsthatmayserveasexternalfeedbackforEMresidencyprogramsandtheir clerkships.SelectiveuseofAOCsmaysetaprecedentforfutureprogramassessmentsbyapplicants andinformprogramleadershipofimportantprogrammaticelementsintheeyesofapplicants.It elucidatesimportantthemesintheirinteractionsorlearningexperienceswithprogramsandcreates opportunitiesforlearner-centricprogramimprovement.[WestJEmergMed.2024;25(2)191–196.]

INTRODUCTION

Theacademiccommunityhastraditionallyviewed anonymousonlinecommunities(AOC)aspoorresourcesfor advising,recommendingthatstudentsbewaryofthemor avoidthemaltogether.1 Commonthemesaddressedagainst theseforumsincludelackofcommenterprofessionalism,2–4 informationinaccuracy,5 andbreachofethicsviamalicious postingoffalsified,incomplete,orprivilegedinformation.2,3 PriorstudieshavealsoshownthatanonymousAOC commentersmaynotnecessarilyreflecttheentireapplicant population.6 Withinemergencymedicine(EM),students reportinformationfromAOCssuchasRedditandStudent DoctorNet(SDN)asthe “leasttrustworthy” comparedto otheradvisingresources.7

Despitethis,mostspecialties,includingEM,haverobust AOCsformedicalstudents,boastinglargeuserbasesand robustdiscussionthreadswithheavyreadership.8–11 Within theseanonymousforums,studentsdiscussdiversetopics abouttheapplicationprocess,specialty-specificquestions, andstudentexperiencesapplyingto,rotatingat,or interviewingwithspecificprograms.12,13 Forexample, studiesidentify “program-specificinformation” asa commonthemeinotolaryngology-andradiology-applicant AOCs;however,their findingswerelimitedincharacterizing specifictopicsdiscussed.12,13 Thereisalsoaconsolidated, annuallyrenewed,anduser-generatedGoogleSpreadsheet circulatingwithinEMforumswithastatedgoalto “provideacentrallocationforapplicantstoresearch differentresidencyprograms,viewinformationabout otherapplicantsandwheretheyareapplying, andshareinformationaboutaway,interviews,and generaladvice.”14

ForEM,thediscussionofauthentic,program-specific experiences,suchasthestudent’sinterviewdayexperience andinteractionwithresidents,havehistoricallybeenranked asthetoptwofactorsinimpactingtheirrankorder,making thisinformationhighlyvaluabletobothapplicantsand programstakeholders.15 Ourprimarygoalwasto characterizewhatprospectiveEMapplicantsdiscussinan AOCforumregardingtheirexperienceswithspecific programs.Oursecondarygoalwastoidentifypotentially usefulinformationforprogramimprovement anddevelopment.

METHODS

Thiswasaqualitative,retrospectivereviewofapublicly availableAOCforEMrotationsin2020–2021.Itwas submittedforinstitutionalreviewboardreviewthrough ThomasJeffersonUniversityanddeterminedtonotmeetthe definitionofhumansubjectsresearch.Weanalyzedextracted datafromanonline,time-stamped,anduser-lockedGoogle Sheetentitled “Official2020–2021EmergencyMedicine ApplicantSpreadsheet,” whoselinkcanbefoundwithin popularAOCssuchasReddit,SDN,andDiscord.14 “The

Spreadsheet” allowsanonymousindividualstopost requestedinformationregardingspecificEMprograms.The spreadsheetcontainsmultiplesub-pages,or “sheets,” to addressdifferenttypesofinformationanapplicantmight seek.Thisincludessheetslistingprogram-specificfactssuch as “ProgramBenefits” and “ProgramInformation”;sheets describingstudentexperienceswithaprogramlike “RotationExperience,”“VirtualIV(Interview) Impressions,” or “NameandShame”;andsheetsaddressing miscellaneousapplicationtopicssuchas “Rejection/Wait List” or “DroppedInterviews” tohelpapplicants coordinatelogistics.

Withpermissionfromthepageadministrator,confirmed tobeacurrentEMresident,wecreatedareplicaofthe spreadsheetonSeptember12,2021,forthepurposeofthis study.Uponreviewofallavailablesheetswithinthe spreadsheet,thesub-pagesentitled “VirtualIV” and “RotationImpressions” werepurposefullysampledvia groupconsensusforanalysisastheywerefelttomostlikely includestudents’ directimpressionsofprograms.Incontrast, weexcludedsub-pagessuchas “NameandShame” from qualitativeanalysisduetohighlikelihoodofcontaining causticandcontroversialnarratives.Asthepurposeofthis studywastoinvestigate “what” isbeingsaid,not “who” is discussingthemorto “whom” itisaddressed,one investigatortransferredallcommentsfromtheselectedpages intoasingledatasetwhileremovingpotentiallyidentifying userorprograminformation.

Weperformedqualitativeanalysisprimarilyusingthe constantcomparativemethod,16 whereexcerptsofrawdata areorganizedintogroupsaccordingtoattributesandthose groupsarefurtherstructuredtoformulateanewtheory.The selectedsub-pagesyielded183individualnarratives discussingstudents’ impressionofthede-identified programs.Arandomnumbergeneratorwasusedtoselect 20%ofindividualnarrativesasaconveniencesamplefor investigatorstoinductivelycreateaworkingcodebook,de novo.Allduplicateswereidentifiedandremoved,untilthe excerptswereallunique.Thedatasetwasindependently analyzedbytheinvestigators[ME,JG,RR,XCZ]toidentify thematiccontentwithineachnarrativeforinductivecoding. Individuallyidentifiedthemeswerecomparedamong investigatorstogeneratecommonthemes.Thesethemes wereorganizedintomajor “themes” and “sub-themes” to createtheinitialcodebook.Thiswasusedbyasingle investigator(JG)tocodetheentiredataset.Additionalsubthemesidentifiedduringthisprocesswereupdatedintothe codebookunderexistingmajorthemes.Uponcompletion, allthemesandtheirassociatedexcerptswerereviewedbythe remaininginvestigatorstoensurecodingconsensus.The entiredatasetwasrevieweduntilnoadditionalthemeswere identified(see Appendix1).

Uponreviewingthe finalcodebook,wecreatedtwo additionalthematiccategories:1)potentially “modifiable”

programattributesand2)lesslikelyor “non-modifiable” programattributes.Drawinguponourcollective experiences,wedefined “modifiable” subthemesasattributes mostlikelyunderthedirectcontroloftheeducation leadershipand “non-modifiable” subthemesasattributes thatareeithertrulynon-modifiableorwouldrequire significantinputfromoutsidestakeholderstochange.This distinctionwasmadewiththeunderstandingthatdifferent programshavedifferentabilitiestomodifycertainattributes.

RESULTS

From183comments,841discretestatementswere identifiedandcoded.Weidentifiedsixthemes:working conditions;interpersonalrelationships;learningexperience; livingconditions;postgraduatereadiness;andonline/virtual

supplement,asshownin Figure and Table1.Thetoptwo encodedthemes workingconditionsandinterpersonal relationships comprised572(68%)ofthetotalcoded statements(324[38.5%]and248[29.5%],respectively).

Sub-themesidentifiedwithineachtheme(see Appendix1 forafulllistingofsub-themeswiththeircorresponding numberofcodedstatements)werethensubdividedto representmodifiableandnon-modifiableclerkship/program aspects(Tables2 and 3).Modifiablesub-themes outnumberednon-modifiablesub-themes(60.7%vs39.3%). Thesub-themeshousedwithinthethemeofinterpersonal relationshipsrepresentedthelargestsinglecategoryof modifiableattributeswith248(29.5%)statements. Commentsonworkingconditionsandlearningexperience werethesecondandthirdlargestcategories,with109(13%) and118(14%)comments,respectively.Themajorityofnonmodifiablesub-themeswerefoundwithinthethemeof workingconditionswith215(25.6%)individualcomments, whichrepresented65%ofallnon-modifiablecomments.The secondlargestnon-modifiablesub-themewaswithinthe themeoflivingconditions,includingcommentsonthelocal geography,costofliving,ornearbyamenities.

DISCUSSION

Anonymousonlinecommunitieshavebeenhistorically viewedbyclerkshipandresidencyprogramleadershipas unreliableforumsforstudentdiscussionthatfoster confabulationoffactsandoperatesonrumorsandhearsay,a communicationtoolofthedisgruntled,andnotaresourceto betakenseriously.1–3,5–7 Thisisthe firststudytodescribe,in detail,thenarrativecontentofstudentsdiscussingtheir programimpressionsonanAOC.Our findingssuggestthat manyofthediscusseditemsarecommonconsiderationsfora studentseekingto findtheidealnextstageoftraining.

ThemeRepresentativecomment

Workingconditions

“ academicinstitutionmeansalotofconsultssometimes,some12-hourshifts(butmixof8sand12s), 50%ofshiftsasaninternareovernight nodebriefingprocessaftercodes/traumas ”

Interpersonal relationships “Everyprogrammentionsfamilyfeelbuthavingrotatedhereitwastrulytightknit.Facultyandresidenthang outsoftenincludingbeachtrips.”

Learningexperience “Mostattendingstoleratestudents,andtherestarereallyproactiveaboutteachingandgettingthestudents involved Whenitdoessettleabit,residentsareenthusiasticaboutyoureducationforthemostpart.You’ll gettodoalmostanyprocedureyouwantbecausetheresidentshavealreadydonethemathousandtimes before.”

Livingconditions “Arearightoutsideof[location]canbeaprooracon.Probably[would]havetodealwithalotoftrafficand highcostofliving.”

Postgraduate readiness “Reallyoldprogram,soalumnialloverthecountrytohelpwithjobplacement(lastclass1/2community, 1/4fellowship,1/4academic).With4differenthospitals,varietyoftrainingisverygoodandwillbeprepared foranytypeofjobcomingout.”

Online/virtual supplement “BestEDtour,literallytookaGoProfromtheEDdepartmenttotheactualEDsoyoucouldactually seetheED.”

Figure. Thematiccategoriesofcodedstatements,includingthe numberofindividualstatementsandpercentageoftotalstatements.

Table1. Identi fiedthemeswithrepresentativecommentscodedtoeach.

ThemesSub-themes#Totalcodedcomments

WorkingconditionsPerks(fundingfortravel/activities,food,lounge,parking,etc)37109(13.0%)

DEI(includesLGBTQ+)27

Relationshipwithotherspecialties23

Wellness20

Scutwork2

InterpersonalrelationshipsResidents76248(29.5%)

Otherleadership/facultypersonality76

PDpersonality56

Responsivenesstoupwardfeedback16

Opportunityforupwardfeedback14

Generic8

Objectiveexperience2

LearningexperienceProcedures25118(14.0%)

Didactics/conference20

On-shiftteaching17

Autonomy16

POCUS12

Pediatrictraining12

EMS/prehospitaltraining5

Scholarlytracks5

Research5

Personalpatientload1

Online/virtualsupplementVirtualinterviewday2835(4.2%)

Virtualtour4

Virtualrotation2

Website1

MentorsinEMhavehistoricallyencouragedprospective EMapplicantstoinquireaboutinterpersonalinteractions andresidentworkingconditionswithinaspecificprogram. Ouranalysisrevealsthatstudentsarealsoseekingmore informationandcommentingonmanyofthesamefactors wehavebeenadvisingthemtoseekout.17

Moreover,analysisofthesub-themesrevealsauniquetrend towardpotentiallymodifiableprogramattributesthat,if addressed,couldbemutuallybeneficialforprogramsand applicants.Topicssuchasperceivedresidentwellness, diversity,equity,andinclusion,opportunitiesforupward feedback,andeffectivenessofon-shiftteachingareallunder thecontrolofaprogramtopotentiallyimprove.Manyofthese topicsareofrisingimportancetostudents.18,19 Theavailability ofthisinformationraisesaveryinterestingquestionfor programsandrecruitment:ifprogramswereawareofthese discussedtopicsandthestudentcommentsrelativetoeach

topic,wouldaprogrambelikelytochangeinternalelement(s) tomakeitselfmoreappealingtostudents?20

InlightoftherecentNationalResidentMatching Program(NRMP)resultsfrom2022andthecontinued downtrendofapplicationsin2023,21,22 manyEMprograms mustcontendwithasmallerapplicantpool,whichreduces thelikelihoodof fillingprograms,andoverallprogram competitivenessforapplicantrecruitment.Whilewecannot predictfuturetrends,ourspecialtyhasfaceddeclining studentapplicationsfortwoyearsinarowwitharising numberofresidencyprogramsandpositionsoverthelast severalyears.Astraditionalmatchingpatternsbeginto falter,residencyleadershipshouldconsideraddressing criticalelementsfromAOCs,insteadofignoringthemas traditiondictates.

Aninteresting finalobservationfromourstudyisthe relativelyscarcenumberofcommentsfromstudentson

Table2. Modifiablemajorthemesandsub-themesdeterminedbyauthorconsensusbyaclerkshiporresidencyprogram;percentagesareof thetotalnumberofcomments,N = 841.

DEI,diversity,equity,inclusion; PD,programdirector; POCUS,point-of-careultrasound; EMS,emergencymedicalservices.

ThemesSub-themes#Totalcodedcomments

WorkingconditionsPatientpopulation(underserved,volume,trauma,pathologyetc)66215(25.6%)

Practicesetting(community,academic,county,Lvl1,HCA,etc)66

Programreputation/prestige/age21

Workhours17

Ancillaryhealthcarestaff15

EHR12

Salary9

Metrics6

Moonlighting3

LivingconditionsGeography5370(8.3%)

Amenities11

Costofliving6

PostgraduatereadinessFellowships1746(5.5%)

Jobs13

PGY4experience(lengthoftraining)12

PGY3experience(lengthoftraining)4

HCA,HospitalCorporationofAmerica; EHR,electronichealthrecord; PGY,postgraduateyear.

virtualoronlinecomponentsofaprogram.Ourdataset reflectedthe firstapplicationcycleduringtheCOVID-19 pandemicwithradicalparadigmshiftsinaway-rotation restrictionsandstudentsexploringvirtualinterview processesforthe firsttime.Despitetheseunprecedented large-scalechanges,only4.2%ofthetotalcommentsfocused onthe “virtual” aspectofprogramrecruitment.Thisisin starkcontrasttothesignificantamountoftimespentby institutionsandnationalorganizationsonvirtualrotations, virtualtours,ongoingvirtualinterviews,virtualresidency fairs,andvirtualhangoutsforstudentstosocializewith residents.Thedataremainsunclearbasedonthis informationfromasingleyeartoexplainthislackof commentary.Itmayperhapsbeduetolackofstudent participationinvirtualrotations,giventhiswastheir first yearbeingavailableasarotationoption,orperhapsvirtual rotationswerejustsimplynotseenasappealing,thus demandinglessdiscussiontimeonAOCs.Furtheranalysisof subsequentyearswouldbeneededtofullyanalyzethe effectivenessofvirtualoptionsforstudentapplicants.

LIMITATIONS

Potentiallimitationsrestlargelyondata fidelity.2,3,5 Prior studieshavealsoshownthatAOCcommentersmaynot necessarilyreflecttheentireapplicantpopulation.6 Thereis limited-to-nodemographicinformationprovidedonthe analyzedAOC.Additionally,the183narrativesanalyzed fromthespreadsheetarerelativelylowcomparedtothe

numberofapplicantsrankingEMastheirpreferredspecialty orthe273EMprogramsinexistenceatthetimeof2021 NRMPMatch.21 Thismayhaveputouranalysisatriskof notreachingthematicsaturation.Nevertheless,basedonour collectiveexperiencesasEMresidencyapplicantsandasEM applicationadvisers,wedidnot findanyidentifiedsubthemesparticularlysurprisingorcontroversial.Although onlyasingleAOCwasanalyzedinthisstudy,webelieveitto befairlyrepresentativeofcommonlyrecurringstudent opinionsandobservations.Forthepurposeofthisstudy,we specificallyselectedtwosub-pageswiththehighestdensityof meaningfulcommentaryforanalysis;thereisthepotential thatcommentsfromotherpagesmayrevealfurtherthemes orsub-themes.

CONCLUSION

Ourqualitativeanalysisofasingleanonymousonline communityrevealedsixmajorthemesdiscussedamong studentswithregardtoEMresidencyprograms: workingconditions;interpersonalrelationships;learning experience;livingconditions;postgraduatereadiness;and online/virtualsupplement.Mostofthesub-themestothese categoriesrepresentedaspectsofclerkshipsandresidency programsthatarepotentiallymodifiablebythe programleadership.These findingssuggestthatAOC narrativescoverseveraltopicsthatmayserveasuseful externalfeedbackforEMresidencyprogramsorclerkships. Iterativereviewofprogram-speci ficAOCnarratives

Table3. Non-modifiablemajorthemesandsub-themesdeterminedbyauthorconsensusbyaclerkshiporresidencyprogram;percentages areofthetotalnumberofcomments,N = 841.

Estesetal. StudentNarrativesRegardingEMResidency

couldserveasadditionaldataindeterminingwhethera program ’sinternalimprovementeffortsarenoticed bystudents.Additionalstudiesmayhelpcharacterize thelevelofinterestbykeyprogramstakeholdersto considerandmakechangesbasedonfeedbackfrom AOCsources.

AddressforCorrespondence:MollyEstes,MD,LomaLinda University,DepartmentofEmergencyMedicine,11234AndersonSt, A-890,LomaLinda,CA92354.Email: mollykestes@gmail.com

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.This researchwasfundedinpartbyanEMRABetheChangeGrant awardedin2021.

Copyright:©2024Estesetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.JainSHandMaxsonER.Risksofonlineforumsforpremedicaland medicalstudents. AcadMed. 2011;86(2):152.

2.GusehJS2nd,BrendelRW,BrendelDH.Medicalprofessionalisminthe ageofonlinesocialnetworking. JMedEthics. 2009;35(9):584–6.

3.ThompsonLA,DawsonK,FerdigR,etal.Theintersectionofonline socialnetworkingwithmedicalprofessionalism. JGenInternMed. 2008;23(7):954–7.

4.YapKandTiangYL.Recommendationsforhealthcareeducatorson e-professionalismandstudentbehavioronsocialnetworkingsites. Medicolegalandbioethics. 2014;4:25–36.

5.FacultyofMedicineMDUndergraduateProgram,TheUniversityof BritishColumbia.NAQMythsDebunked-Part1.2014.Availableat: https://mdprogram.med.ubc.ca/2014/12/15/ naq-myths-debunked-part-1/.AccessedJune5,2021.

6.SuraK,WilsonLD,GrillsIS.Comparisonofself-reported dataonStudentDoctorNetworktoobjectivedataofthe NationalResidentMatchingProgram. JAmCollRadiol. 2017;14(12):1594–7.

7.CarleD,ChristensenR,JarouZ,etal.Studentuseandperceived reliabilityofemergencymedicineadvisingsources. WestJEmergMed. 2018;19(4.1):S21–2.

8.CuratoM,HusainA,ShahK,etal.Theirresponsibleuseofsocialmedia amongmedicalstudents. AEMEducTrain. 2019;4(2):168–70.

9.DornerSC,SwansonMB,ZuckerSM.Theresponsibleuseofsocial mediaamongmedicalstudentsandresidents. AEMEducTrain. 2019;4(2):171–3.

10.FabrisC.Anonymousfeedback, fine.Insults?Notontheseplatforms. 2015.Availableat: https://www.chronicle.com/article/ anonymous-feedback-fine-insults-not-on-these-platforms/ AccessedJune5,2021.

11.LeeTandRosenC.Socialmediaandothersourcesofinformationused duringtheemergencymedicineresidencyapplicationprocess. WestJ EmergMed.2017;18(5.1):S27.

12.KozinED,SethiRK,LehmannA,etal.Analysisofanonlinematch discussionboard:improvingtheotolaryngology-headand necksurgerymatch. OtolaryngolHeadNeckSurg. 2015;152(3):458–64.

13.YiPH,NovinS,VanderT,etal.Howdoescurrentgenerationofmedical studentsviewtheradiologyMatch?AnanalysisoftheAuntMinnieand StudentDoctorNetworkonlineforums. AcadRadiol. 2018;25(6):699–707.

14.Official2020-2021EmergencyMedicineApplicantSpreadsheet.2021. Availableat: https://docs.google.com/spreadsheets/d/ 1knDL01qZfYB6HATiyL32gBPmuhsiW-n52a2baISVR8M/ edit#gid=286889120.AccessedJune1,2021.

15.LoveJN,HowellJM,HegartyCB,etal.Factorsthatinfluencemedical studentselectionofanemergencymedicineresidencyprogram: implicationsfortrainingprograms. AcadEmergMed. 2012;19(4):455–60.

16.Hewitt-TaylorJ.Useofconstantcomparativeanalysisinqualitative research. NursStand. 2001;15(42):39–42.

17.TrainorA,JurvisA,MessinaM.Chapter4:FindingYourFit:Learningthe LandscapeofEM.InJarouZ(Ed.) EMRAandCORDStudent AdvisingGuide. Irving,Texas:EmergencyMedicineResidency Association;2019.

18.HammoudMM,StandifordTC,CarmodyJB.The2020-2021residency applicationcycle:lessonslearnedandlingeringproblems. JAMA. 2021;325(22):2249–50.

19.KatsufrakisPJ,UhlerTA,JonesLD.Theresidencyapplicationprocess: pursuingimprovedoutcomesthroughbetterunderstandingofthe issues. AcadMed. 2016;91(11):1483–7.

20.BuckleyRJJ,BrownC,StringerS,etal.Onblast:aframeworkfor monitoringandrespondingtoonlinecommentsaboutyourgraduate medicaleducationprogram. JGradMedEduc. 2020;12(6):651–4.

21.TheMatch.MatchData&ReportArchives,NationalResidentMatching Program.Availableat: https://www.nrmp.org/match-data-analytics/ archives/.AccessedAugust3,2021.

22.CouncilofResidencyDirectorsinEmergencyMedicine.2022.Cord BODStatementonApplicantNumbersforEMResidencyPositions. Availableat: https://www.cordem.org/globalassets/files/ position-statements/cord-bod-response-on-applicant-numbers-for-emresidency-positions.pdf.AccessedOctober14,2022.

RapidCycleDeliberatePracticeTrainingforSimulated CardiopulmonaryResuscitationinResidentEducation

JaronD.Raper,MD*

CharlesA.Khoury,MD*

AndersonMarshall,MD†

RobertSmola,MD†

ZacharyPacheco,MD*

JasonMorris,MD†

GuihuaZhai,PhD‡

StephanieBerger,MD§

RyanKraemer,MD†

AndrewD.Bloom,MD*

*UniversityofAlabamaatBirmingham,DepartmentofEmergencyMedicine, Birmingham,Alabama

† UniversityofAlabamaatBirmingham,DepartmentofInternalMedicine, Birmingham,Alabama

‡ UniversityofAlabamaatBirmingham,CenterforClinicalandTranslationalScience, Birmingham,Alabama

§ UniversityofAlabamaatBirmingham,DepartmentofPediatrics,Birmingham,Alabama

SectionEditors:DavidThompson,MD,andDanMayer,MD

Submissionhistory:SubmittedMarch3,2023;RevisionreceivedNovember21,2023;AcceptedNovember27,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17923

Background: Simulation-basedmedicaleducationhasbeenusedinmedicaltrainingfordecades. Rapidcycledeliberatepractice(RCDP)isanovelsimulationstrategythatusesiterativepracticeand feedbacktoachieveskillmastery.Todate,therehasbeenminimalevaluationofRCDPvsstandard immersivesimulation(IS)fortheteachingofcardiopulmonaryresuscitationtograduatemedical education(GME)learners.OurprimaryobjectivewastocomparethetimetoperformanceofAdvanced CardiacLifeSupport(ACLS)actionsbetweentraineeswhocompletedRCDPvsIS.

Methods: Thisstudywasaprospective,randomized,controlledcurriculumevaluation.Atotalof55 postgraduateyear-1internalmedicineandemergencymedicineresidentsparticipatedinthestudy. ResidentswererandomizedtoinstructionbyRCDP(28)orIS(27).Stressandabilitywereself-assessed beforeandaftertrainingusingananonymoussurveythatincorporated five-pointLikert-typequestions. WemeasuredandcomparedtimestoinitiatecriticalACLSactionsbetweenthetwogroupsduringa subsequentIS.

Results: PriorlearnerexperiencebetweenRCDPandISgroupswassimilar.Timestocompletionofthe firstpulsecheck,chestcompressioninitiation,backboardplacement,padplacement,initialrhythm analysis, firstdefibrillation,epinephrineadministration,andantiarrhythmicadministrationweresimilar betweenRCDPandISgroups.However,RCDPgroupstooklesstimetocompletethepulsecheck betweencompressioncycles(6.2vs14.2seconds, P = 0.01).Followingtraining,learnersintheRCDP andISgroupsscoredtheirabilitytoleadandtheirlevelsofanticipatedstresssimilarly(3.43vs3.30, (P = 0.77),2.43vs.2.41, P = 0.98,respectively).However,RCDPgroupsratedtheirabilitytoparticipate inresuscitationmorehighly(4.50vs3.96, P = 0.01).TheRCDPgroupsalsoreportedtheirrealizedstress ofparticipatingintheeventaslowerthanthatoftheISgroups(2.36vs2.85, P = 0.01).

Conclusion: Rapidcycledeliberatepracticelearnersdemonstratedashorterpulsecheckduration, reportedlowerstresslevelsassociatedwiththeirexperience,andratedtheirabilitytoparticipateinACLS caremorehighlythantheirIS-trainedpeers.OurresultssupportfurtherinvestigationofRCDPinother simulationsettings.[WestJEmergMed.2024;25(2)197–204.]

INTRODUCTION

Despiteadvancesinresuscitationscienceandtraining, cardiacarrestremainsthethirdleadingcauseofdeathinthe UnitedStates.1 MillionsofcliniciansreceiveBasicLife Support(BLS)andAdvancedCardiacLifeSupport(ACLS) training,yetpatients’ survivalratesvaryconsiderably.2,3,4 Immediaterecognitionofcardiacarrest,highquality cardiopulmonaryresuscitation(CPR),andtimely defibrillationarethemainstaysofcare.5,6 Effectiveeducation iscrucialtoexecutetheseprinciples,improveteam performance,andenhanceoutcomes.7

Simulation-basedmedicaleducation(SBME)iswell establishedinmedicaltrainingandgraduatemedical education(GME).InSBME,learnersgainexperiencefroma realisticclinicalscenariowithoutthepossibilityofcausing harmtoapatient.8 Learnersaregiventhefreedomtodevelop skillsthroughpracticeandgainvaluablefeedbackvia debriefing.Asaresult,SBMEhasbeenassociatedwith improvedskilldevelopmentandpatientoutcomes.9,10,11

Rapidcycledeliberatepractice(RCDP)isaninnovative simulationstrategythatusesiterativepracticeandfeedback toachieveskillmastery.DevelopedfromAndersEricsson’ s workondeliberativepractice,RCDPallowsforadvanced learningthroughrepetitionandskillrefinement.12 Itwas originallydescribedbyHuntin2014andimplementedin pediatricresuscitationtraining.13 InRCDP,learnersbegina simulatedscenario,butincontrasttotheclassicalpostsimulationdebrief,thecaseisfrequentlypausedbythe instructor.Eachbreakservesasanopportunityforcorrective instruction,coaching,feedback,andsubsequent supervisedrepitition.13

Overthelastdecadetherehasbeenanincreasedfocuson RCDPtraininginresuscitation,withmoststudiesfocusedon pediatrictrainees.13,14 Whencomparedtothestandard immersivesimulation(IS)approach,RCDPhas demonstratedshortertimetoinitialchestcompressionand defibrillationinpediatricmedicinetrainees,improvedchest compressionfractioninadultmedicaltrainees,andbetter skillretention.13,14,15,16 Evenmorerecently,wehaveseen RCDPimplementedintoproceduraltrainingwhereithas alsodemonstratedpositivelearneroutcomes.Groupstrained inRCDPdemonstratedbetterpreparednessforintubation andpost-procedurecareinpediatricairwaymanagement.17 Similarly,RCDP-basedtraininghasbeensuggestedforthe donninganddoffingofpersonalprotectiveequipment,and ourobstetriccolleagueshaveprovenitsutilityforforcepsbaseddeliveries.18,19

InstructionbasedinRCDPhasstrongevidencetosupport itsuseinareasofmedicaleducationthatarealgorithmicin nature,and/orrequireahighdegreeofproceduralskill.The AmericanHeartAssociation(AHA)recognizedthisas recentlyas2020,recommendingthatdeliberatepracticebe incorporatedintoBLSandACLStraining,simultaneously

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Rapidcycledeliberatepractice(RCDP)isa simulationstrategythatusesiterative practiceandcoachingtoachieveskillmastery andiseffectiveinproceduralinstruction.

Whatwastheresearchquestion?

IsRCDPorimmersivesimulation(IS)more effectiveintrainingresidentstoperform cardiopulmonaryresuscitation(CPR)?

Whatwasthemajor findingofthestudy?

RCDPshortenspulsechecks,andlearners reportedlessstressandgreatercon fi dence performingCPR.

Howdoesthisimprovepopulationhealth?

ResuscitationinstructionbasedinRCDP showspromiseasatooltoenhanceresidents ’ masteryoflifesavingCPRskills.

identifyingitasaneducationalstrategywarrantingfurther research.7 Despitethiscalltoaction,therehasbeenapaucity ofliteratureevaluatingRCDPinACLStrainingforthecare ofadultpatients,regardlessoflearnertype.20 Wesoughtto addressthisknowledgegapthroughtheevaluationofRCDP forACLSasitisappliedtopostgraduateyear(PGY)-1 residentsinGME.Wedidthisthroughacomparisonoftime tocompletionofcriticalACLSactionsbetweenRCDPand ISgroups(ourprimaryobjective).Asasecondaryobjective, wecomparedresidentperceptionsbetweenRCDP-and IS-trainedgroups.

METHODS

StudyDesign

InJuly2022,weconductedaprospective,randomized, controlledstudyapprovedbytheinstitutionalreviewboard.

SettingandParticipants

Thestudywasconductedinanaccreditedsimulation centerthatispartofalargeacademicteachinghospitaland involved43internalmedicine(IM)and12emergency medicine(EM)PGY-1residentswhohadobtainedACLS certificationinthetwoweeksprecedingthisstudy.Noother coachingorinstructionregardingthecareofapulseless patientwasprovidedpriortostudyimplementation.All55 residentsparticipatedvoluntarilyinthestudy.Faculty facilitatorsofallsimulationsessionswereIMandEMfaculty whowereboardcertifiedintheirrespective fields.Each

facilitatorunderwentformalIS-andRCDP-facilitator trainingpriortoinvolvementinthestudy.Facilitatorswere notblindedtothestudyobjectives.

Protocol

Intheweekprior,residentswereprovidedwitha descriptionofthestudyandanelectroniccopyofthe informedconsentdocumenttoallowforadetailedand privatereview.Eachofthe55participantsthenprovided writteninformedconsentonthedateoftheirscheduled simulationevent.Ourstudyused fiveteamsforeach instructionalintervention.Eachteamwascomprisedof five orsixmemberswhowererandomlyassignedtoeitherRCDP orIS,foratotalof55participants(28inRCDPgroups,27in ISgroups).Whiletherewasa fixedandlimitednumberof availableparticipants(IMandEMinterns),weperformeda post-hocpoweranalysistoestablishabasisforfuturework. Withan α = 0.05,thisstudyhad29%powertodetectalarge effectsize(d = 1)forprimaryoutcomesand71%powerto detectamediumeffectsize(w = 0.3)forsecondaryoutcomes (G*Power3.1.9.7).Weusedanonlinerandomization generator(https://www.randomizer.org/)todivide participantsinto10teams,with fiveteamsforeach instructionalmethod.

Duetoschedulingdifferences,IMandEMparticipants wereseparatedandcompletedtheirrespectiveexperienceson differentdays.TheIMfacultyfacilitatedallIMresident sessions.Tominimizeconfoundingrelatedtothe effectivenessoftheindividualfacilitator,thetwoIMfaculty facilitatorsledboththeRCDPandtheISsessionsfortheIM residents.The12EMparticipantscompletedtheirexperience thefollowingweekintwoteamsofsix,oneofwhichwas assignedtoRCDPandtheothertoIS.TheEMfaculty facilitatedbothEMresidentsessions.Allfacultyfacilitators weretrainedinimplementationofRCDPandIS.This trainingwasprovidedbycertifiedhealthcaresimulation instructorsinourinternationallyaccreditedinstitutional simulationcenter.Nofacultyfacilitatorswereinvolvedinthe extractionofperformancedata.

Thesametwoembeddedsimulationparticipants(ESP) functionedasnursesforallsessions.TheESPsinallsessions wereregisterednursesandcertifiedhealthcaresimulation educatorsemployedbyourinstitutionalsimulationcenter. TheESPswereinstructedtoassistonlywithcaretaskswhen directlyaskedforspecifictaskassistance(eg,locatingcare items)butdidnottriggerinitiationofindividualtask completionorprovideguidanceontaskperformance.

Followinginformedconsent,learnerswereaskedto completeapre-simulationsurveytoestablishbaseline learnercharacteristics.Thesurveyqueriedeachparticipant’ s priorlevelofexperienceaswellasself-perceivedabilityto leadandparticipateinthecareofapulselesspatient.The surveysalsoassessedthelearner’santicipatedandprior experiencedstressassociatedwithcodeleadershipand

participation.Eachmeasurewasassessedusinga five-point Likert-typescale.

ImmersiveSimulationProtocol

AllISteamswereprovidedwiththesamescriptedprebrief,whichdescribedthebasictenetsofsimulationand informedparticipantsthattheywouldbecaringfora pulselesspatient.Teamswerenotinstructedregardingthe assignmentofclinicalrolesbutwereallowedtoself-assignas theydeemedappropriate.TheISteamswerethenactivated byanESPfunctioningintheroleofanursewhobroughtthe participantstothecareareaandaskedparticipantsto evaluateanunresponsivepatient.

Onceoutsidethepatient’sroom,participantsassumed careforthepatientwithoutfurthercoachingorintervention. TheISparticipantswerepermittedtonavigatethepatient’ s casewithoutinterruption,whilephysicianfacilitators observedtheiractionsfromasimulationcontrolroomwith audioandvisualsurveillanceofthesimulationarea.TheIS learnerswereallowedtonavigatetheircasewithout interruptionuntilthefourthpulsecheckoruntil30minutes hadelapsed,atwhichtimefacilitatorsinitiatedreturnof spontaneouscirculationandthecasewasterminated.Given thenatureoftheISeducationalsessions,learnersdidnot havetheopportunitytorotateroles.Learnersthenreturned tothebriefingroom,andphysicianfacilitatorsdebriefed basedonobservedperformanceaccordingtoastandardized debriefingguideanduntiltotalcasetimereached45minutes. Theguideemphasizedcoachingregardingresuscitationand time-sensitiveinterventionsthatmatchedtheprimary outcomemeasures(eg,timetoidentificationofpulselessness, timetoinitiationofchestcompressions,etc).

RCDPSimulationProtocol

AllRCDPgroupsweregivenastandardpre-briefthat describedthebasictenetsofsimulation.Groupswerethen providedwithanintroductiontothesimulationmodality assignedtothem.Teamswerenotinstructedregardingthe assignmentofclinicalrolesbutwereallowedtoself-assignas theydeemedappropriate.TheRCDPteamswereactivated byanESPwhobroughttheparticipantstothecareareaand askedthemtoevaluateanunresponsivepatientwhile physicianfacilitatorsobservedatthebedside.TheRCDP groupsrotatedroles,allowingthemtheopportunitytodirect theresuscitationandreceivefeedback.

Inadditiontotheirstandardizedtraining,allfacilitators wereprovidedwithanRCDPcoachingguide,whichwas focusedonthesameresuscitationandtime-sensitive interventionsastheimmersivecasedebriefingguide. Facilitatorsprovidedreal-timecoachingandfeedbackbased ontheRCDPcoachingguide.Caseswerethenrestarted, rewound,orresumedaccordingtofacilitatordiscretion. Totallearnersimulationanddebriefingtimewas45minutes foreachRCDPcase.

ProtocolOverlap

Uponcompletionofdebriefingandclosureoftheir respectivecases,allparticipantsreturnedtothebriefing space.MaintainingseparationofinitialRCDPvsISgroups, asubsequentISsessionwascompletedbyallparticipants duringwhichaudioandvisualrecordingswereobtained.

DataabstractionoftimestocompletionofcriticalACLS actionswasobtainedfromthissession.Participantswere activatedasecondtimebytheESPtocareforanadditional, unresponsivepatient.Learnerswereallowedtoroleassign andcompletethecasewithoutinterventionfromtheESP orfacilitator.

PrimaryOutcomeMeasures

Timetocompletionofcriticalactionswasusedasa surrogateforproficiencyintheperformanceofanACLSbasedresuscitation.Thesecriticalactionsweredefinedby researchteamconsensusafterreviewingACLSprotocols. Timezerowasdeterminedbaseduponlearnerentryintothe carearea,andtimestocompletionofresuscitativetimebasedinterventionswereextractedthroughvideoreviewby theprimaryinvestigator.Tomitigatebiasfromfaculty workingwiththeirownresidents,dataabstractionfrom videorecordingswasperformedbytheprimaryinvestigator, whowasnotinvolvedinsimulationsessionfacilitation.The primaryinvestigatorwasblindedtoRCDPvsISgroup assignmentatthetimeofdataabstraction.Timesfromroom entryto firstpulsecheck, firstchestcompression,backboard placement,defibrillatorpadattachment,initialrhythm analysis,initialdefibrillation,initialepinephrine administration,andantiarrhythmicadministrationwere recorded.Thedurationofpausebetweencompressioncycles wasalsoobtainedforeachsession.

SecondaryOutcomeMeasures

Learnerswerequeriedusingpre-andpost-experience surveys,whichweredistributedinpaperformatimmediately beforeandafterthesimulationsessions.Wedevelopedthe surveysbasedonKirkpatrick’stheoryofeducational trainingandevaluation,focusingprimarilyonlevel1and2 analyses.21 Allsurveyitemsuseda1–5Likert-typescaleto quantifyallqualitativequestions,andsurveyresponserates forallsurveyswere100%.Priortotheeducational intervention,learnerswereaskedtoratetheirself-perceived abilitytoparticipateinandabilitytoleadacode(1notatall capable,to5extremelycapable).Theywerealsoaskedto ratetheiranticipatedstressassociatedwithparticipationand leadershipofacode(1notatallstressful,to5extremely stressful).Finally,theywerequeriedregardingthenumberof simulatedcodestheyhadparticipatedinorled,aswellasthe numberofactualcodestheyhadparticipatedinorled.

Followingtheeducationintervention,learnerswereasked toagainratetheirself-perceivedabilitytoparticipateinand leadacode.Theywerealsoaskedtoratetheoverall

effectivenessoftheirexperience(1notatalleffective, 5extremelyeffective).Finally,learnerswereaskedtoratethe stressleveltheyperceivedtobeassociatedwithparticipating andleadingtheirsimulatedexperience(1notatallstressful, to5extremelystressful).

StatisticalAnalysis

First,wecomparedpriorsimulatedandgenuineCPR experiencesasleaderandasparticipantforRCDPandIS groups,usingtheCochran-Mantel-Haenszeltest,giventhe ordinalnatureoftheLikert-typescale.Wedefinedsimulated experiencesasthoseinvolvingCPRtrainingthatdidnot involvethecareofapatient.Genuineexperienceswere definedasthoseinvolvingtheCPR-basedresuscitationofa codingpatient.Wethencomparedthetime-based differencesbetweenRCDPandISgroupsusingaStudent t -testoraWilcoxontestwhentherewassubstantialdeviation fromnormality.Oursamplesizeforallprimaryoutcome measureswas10teams.Wecomparedmeantimedifferences betweenthetwogroupsfor firstpulsecheck, firstchest compression,pauseduration,backboardplacement, defibrillatorpadplacement, firstrhythmanalysis, first defibrillation, firstepinephrineadministration,and amiodaroneadministration.

Oursamplesizeforallsecondaryoutcomemeasureswas 55individuals.Wealsocomparedpre-andpost-training surveydatabetweenthetwogroupsusingtheCochranMantel-HaenszeltestgiventheordinalnatureoftheLikerttypescale.Thelearner’sexperienceascodeleaderand participantandoveralleffectivenessofexperiencewerealso includedinthepost-trainingsurvey.Abilitytolead,abilityto participate,anticipatedstressleading,andanticipatedstress participatingwereincludedinbothsurveys.Finally,we comparedstressleadingandstressparticipatinginpre-and post-trainingforbothgroupsusingageneralizedStuartMaxwelltesttoevaluatetheimprovementaftertraining.22 Weusedanalphalevelof0.05forallstatisticaltests.A Benjamini-Hochbergfalsediscoveryrateadjustmentwas appliedformultiplecomparisons.Allprogramswerewritten inSAS9.4.(SASInstituteInc,Cary,NC).

RESULTS

PriorLearnerExperience

Priorlearnerexperiencewassimilarbetweenthegroups anddidnotappeartobeasignificantconfounder(Table1). Thenumbersofexperiencesarereportedasmedians withminimumandmaximumvaluesduetolackof normaldistribution.

PrimaryOutcome:Time-basedDifferences

Althoughthereweretrendstowardshortermeantimesto completionofcriticalactionsforRCDPvsISgroups,we observedonlyonecategorywithastatisticallysignificant

GroupMedianMinMax P-value

SimulationIS10170.34

leaderRCDP206

GenuineIS00400.81

leaderRCDP003

SimulationIS30500.46

participantRCDP3.5015

GenuineIS20750.67

participantRCDP2025

IS,immersivesimulation; RCDP,rapidcycledeliberatepractice. *Genuinereferstoexperiencesinactualpatientcarescenarios.

difference:CPRmeanpausedurationinsecondswas6.20vs 14.20seconds(P = 0.01)inRCDPvsISgroups(Table2).

SecondaryOutcome:AbilityandStress

Forstresslevelsandself-reportedability,learners providedratingsona five-pointLikert-typescale.Wepresent themeanvaluesin Tables3 and 4.Priortotraining,RCDP andISlearnersratedtheiranticipatedstressofleadingand participatinginCPRsimilarly(4.36vs4.00(P = 0.44);3.18 vs3.00(P = 0.08),respectively).TheRCDPandISlearners alsoratedtheirpre-trainingabilitytoleadaswellas participateintheeventsimilarly(2.50vs2.37(P = 0.75);3.61 vs3.52(P = 0.59)(Table3).Therewasnosignificant differenceintheanticipatedstresslevelsoffutureevents followingtraining,whetherconsideringtheroleofleader (P = 0.93)orparticipant(P = 0.98)(Table4).Similarly,there wasnosignificantdifferenceinexperiencedstressasaleader betweenRCDPandISlearners(P = 0.93)andtheoverall effectivenessoftheexperiencewasratedsimilarlybetween groups(P = 0.09).However,RCDPlearnersreportedlower levelsofexperiencedstressasaparticipant(P = 0.01)

Table3. Pre-simulationmeanLikert-typeratings.

GroupMedianMinMax P-value

AbilitytoleadIS2140.75 RCDP313

AbilitytoparticipateIS3350.59 RCDP425

Stressanticipatedas leader IS4350.44 RCDP435

Stressanticipatedas participant IS4240.08 RCDP325

IS,immersivesimulation; RCDP,rapidcycledeliberatepractice.

(Tables3, 4).Whenwecomparedpre-andpost-training responsesregardinganticipatedstress,theanticipatedstress offutureresuscitationexperiencesdroppedsignificantlyfor bothleaderandparticipantcategoriesfollowingtraining, regardlessofinstructionalmethod.

DISCUSSION

LearnersreceivingRCDPinstructionshoweda significantlyshortenedpauseduration,reducedstress,and improvedself-perceivedCPRskillscomparedtoIS.The RCDPinstructionalsoshortenedvarioustime-basedACLS metrics,althoughstatisticalsignificancewasnotreacheddue tothesmallsamplesize.Areducedpausedurationcarries notableclinicalsignificance.Pausedurationisanimportant metricofhigh-qualityCPRandisassociatedwithimproved patientoutcomes.23,24 Reducedpausedurationhasa significantimpactonterminatingarrhythmiasandincreasing returnofspontaneouscirculation,whileincreasedpause durationisassociatedwithadecreaseinsurvival.26

AlthoughthereisapaucityofliteraturecomparingRCDP toISinthecareofanadultpatient,whatlittledatathatdoes existdemonstratesimprovementsinchestcompression fractioninRCDPvsISgroups.20 Manyofthesepriorstudies

±2.95)0.27

Backboardplacement40.40(±31.33)193.40(±183.36)0.25

Padplacement66.40(±12.56)74.80(±20.75)0.46

Firstrhythmanalysis73.60(±13.50)111.20(±37.63)0.25

Firstdefibrillation93.00(±17.46)150.60(±63.49)0.25

Firstepinephrine131.60(±28.75)158.20(±55.21)0.41

Pauseduration6.20(±2.07)14.20(±6.53)0.01

Antiarryhthmic376.60(±94.25)438.80(±99.19)0.41

IS,immersivesimulation; RCDP,rapidcycledeliberatepractice.Timeisinseconds.

Table1. Cardiopulmonaryresuscitationexperienceprior tosimulation.

Table2. Rapidcycledeliberatepracticevsimmersivesimulationtimeinseconds.

GroupMedianMinMax P-value

AbilitytoleadIS3350.77

RCDP334 AbilitytoparticipateIS4350.01

RCDP4.545 Stressanticipated

RCDP324 Stressanticipated

Stressexperienced

Stressexperienced

RCDP214

RCDP325

RCDP213 OveralleffectivenessIS4350.09

RCDP545

IS,immersivesimulation; RCDP

weredoneinpediatrics,buttheresultsshouldhaveclinically similarinterpretationsasthosecompletedinadults.13–16 HuntetalconductedtheonlypriorstudyexaminingtimebasedmetricsasasurrogateforproficiencyandfoundRCDP tobesuperiorforinstructionofBLSinterventionsinjunior medicalstudents.27 Ourresultsaddtothisworkthroughthe furtherexaminationoftime-basedmetricsandlearner perceptions.Althoughlimited,theseresultslendfurther credibilitytotheargumentthatRCDPmaybesuperiortoIS forACLStraining.

WhileRCDP-trainedlearnersinourstudyexhibited trendstowardotherfavorableACLSmetrics,therewereno otherstatisticallysignificantdifferences.Priorworkhas demonstratedimprovementintimetodefibrillation,initial chestcompression,andbackboardplacementwithRCDP traininginpediatricresuscitations.13,15,20 Ourworkdoesnot independentlysupportthese findings;however,ourtrends areinlinewithexistingliterature.

Timeto firstdefibrillationsuggestedfavorabilityinthe RCDPgroup(93vs150seconds[sec]),althoughdifferences didnotreachstatisticalsignificance.Thisdistinctionis important,however,astheRCDPgroupwasableperform thisactionwithintheAHA’ s “GetwiththeGuidelines” recommendationof firstdefibrillationinlessthantwo minutes.Similarly,timeto firstepinephrineadministrationin RCDPvsIS(131vs158sec),suggestsreducedtimeinthe RCDPgroupwithoutreachingstatisticalsignificance.Both groupsperformedwithinthe five-minutemetricoutlinefrom “GetwiththeGuidelines” recommendations.Asboth groupsperformedwellwiththisaction,obtainingstatistical significancemayprovedifficult.Itisunclearwhyother

metricssuchaspadplacementoradministration antiarrhythmicshowednosignificantchangebetween groups.Theseactionsaredependentonavarietyoffactorsin ateamfocusedonCPR,andasLemkeetalsuggest,theymay bedifficulttomeasureeffectively.15

Aspreviouslynoted,ourstudywasunderpowered,which playedaroleintheabsenceofstatisticallysignificant differencesformanyofouroutcomemeasures.TheLikertscalemeasureswerebetterpowered,astheyrepresented55 individualsurveyresponsesasopposedtothe10totalteams dividedintwoforeachinstructionalmethod.For comparison,Huntetalstudiedtheperformanceof81 individualpediatricresidentswhoparticipatedinthepostinterventionassessmentandfoundthatRCDPimproved learnerconfidence,buttherewasnocontrolgroupfor comparisonorpowercalculation.13 DeCastroetalused five teamsfortheirRCDPgroupandfourteamsfortheircontrol group,withan80%powertodetecta20%differenceinthe primaryoutcome.Theauthorsfoundahigherchest compressionfractionandshortertimestorhythm identification/defibrillationintheRCDPgroup.However, duetodatalosstheywereunabletoachievetheplanned power. 20 Lemkeetalstudiedthegreatestnumberoflearners, with102participantsin21teamsfortheircontrolcohort,and 108participantsin20teamsfortheirRCDPcohortand foundthatRCDPgroupsdemonstratedshortertimesto defibrillation.Whilenoformalpowercalculationwas performed,Lemke’sworkappearstobethebestpowered thusfar.15 Futureworkshouldincludemorerobustpowering withlargersamplesizes,whichwilllikelyrequireinterinstitutionalcollaboration.

Anotherfactorcontributingtoourinabilitytodetect significantdifferencesinmanytime-basedmetricsmaybethe learnerlevelstudied.Byitsverynature,RCDPservesasa methodtodevelopperfectpractice.Providingthelearner withreal-timefeedbackandcoachingbuildsmicro-skill developmentandmastery,asopposedtoproficiencyalone. ThisstudyfocusedonPGY-1residentsfortworeasons.First, inanefforttoavoidconfoundingbyvariationsintraining, westudiedPGY-1levellearnersintheir firstmonthof residency.Second,weexcludedadvancedlearnersdueto concernsthattheirinvolvementwouldconfoundthe studyofthejuniorlearnerthroughadvancingthe performanceoftheentiregroup.Conversely,priorworkthat founddifferencesinsimilarcategoriesevaluatedlearners fromPGYlevels1–3orstudiedlargerlearnergroups.14,15,27 Therefore,trueskillmasterymaybemoreattainable throughtheinclusionofmoreadvancedlearners andmaycontributetomorestatisticallysignificant results.13,15 Conversely,theinclusionofmoreadvanced learnersmayinfluencetheentiregroup,leadingtoa moreuniformperformance.Thismaylimitor reduceobservabledifferencesbetween instructionalmethods.

Table4. Post-simulationmeanLikert-typeratings.

,rapidcycledeliberatepractice.

HuntetalalsonotesadoseresponsewithRCDP(ie, increasingexperienceandrepetitionfostersimproved performanceandskillmastery).13 Westudiedthelearners’ firstperformance,butwedidnotconductadditional simulatedexperiencesbeyondthis.Furtherrepetitionmay haveexpandeddifferencesinRCDPandISgroups.

Acommongoalofsimulationinmedicaleducationisto reducethestressandanxietyexperiencedbythelearner,and thisisespeciallytrueforhigh-stakesscenariossuchasthe careofapulselesspatient.However,thesimulation experiencecanbeindependentlystressfulforlearners,and priorworkhassuggestedthatRCDP-basedinstructionmay provideanoverallpreferredexperience.Thisiswell illustratedbytheworkofChanceyetal,whoselearners expressedapreferenceforthefrequentinterruptionsand improvedsenseofemotionalsecurityassociatedwithRCDP instruction.25 Chancey’slearnersalsoreportedincreased confidenceintheirownresuscitationskills.Ourresults supportthese findings,demonstratinganincreased confidenceinabilitytoparticipateintheRCDPgroups. Similarly,ourlearnersreportedlowerstresslevels experiencedduringtheirRCDP-basedsimulation.

LIMITATIONS

Duetothestudy’snature,blindingparticipantsand facilitatorswasnotpossible.Skillretentionwasnotassessed, andthesmallsamplesizelimitsgeneralizability. Additionally,whileallfacilitatorshadundergone standardizedtraininginbothinstructionalmethods, individualfacilitatorsmayhavebeenmoreeffectiveatone strategyvstheother.Allparticipatingresidentscompleteda standardACLScourseinthetwoweeksprecedingthestudy. Also,mostoftheresidentshadsignificantexperienceaspart ofresuscitationteams(Table1).Asaresult,theremayhave beenlessofadifferenceinperformancebetweenthetwo groups.OurstudyfoundRCDPwaswellreceivedbyour learners,butthedataislimitedbylearnerevaluationat KirkpatricklevelsIandII.Whilewebelievelearner perceptionsininstructionareimportantforengagement, futureinvestigationsshouldfocusonobjective impactsandclinicalperformancewith patient-orientedoutcomes.

Surveyswerenotbasedonanypriorsurveyinstrumentbut werecreated,reviewed,andeditedbytheresearchteam.The surveyswerenovelinstruments,andwedidnotobtain validityevidencepriortotheiruse.Recallbiaswas minimizedthroughtheimplementationofsurveys immediatelyfollowinginstructionandperformanceofthe learners.Wewereunabletoeliminatetheeffectsofsocial desirabilitybiasforourlearnersandsuspectthatlearners wouldtendtoreportimprovedperformanceregardlessof instructionalmethod.However,thepotentialforthisbias existedinbothRCDPandISgroups.Samplingandnonresponsebiaswerenotfactorssecondarytoour100%

responserate,butduetothenatureofour five-pointLikerttypequestionscale,thepotentialforneutralbiasexists.

Duetothefrequentinterruptionsassociatedwiththe RCDPmethod,RCDPparticipantswereabletorotate througheachroleontheresuscitationteam.However,IS groupsdidnothaveanopportunitytochangerolesasapart oftheirtraining,andthisintroducesaconfounderin comparingthelearnerexperienceaswellasproficiency betweentheseinstructionalmethods.

Finally,thisstudyfocusedontimetocompletionof criticalactionsbutdidnotassessthequalityofthoseactions, includingfactorssuchaschestcompressionfraction(CCF). However,CCFhasbeenpreviouslystudiedandfoundtobe superioringroupsundergoingRCDP-basedinstructionas comparedtostandardIS.13,20,27

CONCLUSION

Rapidcycledeliberatepracticewasfavoredbylearnersfor ACLS-basedCPRinstruction,improvingself-perceived skillsandreducingpauseduration.ThissuggestsRCDPisa validstrategytoteachresidentsACLS-basedCPRand supportsfurtherinvestigationofRCDPinothersettings.

AddressforCorrespondence:JaronD.Raper,MD,Universityof AlabamaatBirmingham,DepartmentofEmergencyMedicine,214 GeneralServicesBuilding,52119thStreetSouth,Birmingham,AL 35233.Email: jdraper@uabmc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Raperetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.TaniguchiD,BaernsteinA,NicholG.Cardiacarrest:apublichealth perspective. EmergMedClinNorthAm. 2012;30(1):1–12.

2.CouteRA,NathansonBH,MaderTJ,etal.Trendanalysisofdisabilityadjustedlifeyearsfollowingadultout-of-hospitalcardiacarrestinthe UnitedStates:astudyfromtheCARESSurveillanceGroup. Resuscitation. 2021;163:124–29.

3.PerkinsGDandCookeMW.Variabilityincardiacarrestsurvival:the NHSAmbulanceServiceQualityIndicators. EmergMedJ. 2012;29(1):3–5.

4.ChengA,NadkarniVM,ManciniMB,etal.AmericanHeartAssociation EducationScienceInvestigators;onbehalfoftheAmericanHeart AssociationEducationScienceandProgramsCommittee,Councilon Cardiopulmonary,CriticalCare,PerioperativeandResuscitation;

CouncilonCardiovascularandStrokeNursing;andCouncilonQuality ofCareandOutcomesResearch.Resuscitationeducationscience: educationalstrategiestoimproveoutcomesfromcardiacarrest:a scientificstatementfromtheAmericanHeartAssociation. Circulation. 2018;138(6):e82–122.

5.CommitteeontheTreatmentofCardiacArrest:CurrentStatusand FutureDirections;BoardonHealthSciencesPolicy;Instituteof Medicine.(2015).In:GrahamR,McCoyMA,SchultzAM(Eds.), StrategiestoImproveCardiacArrestSurvival:ATimetoAct (firstpage ofchapter-lastpage).Washington,DC:NationalAcademiesPress(US).

6.CumminsRO,ThiesW,ParaskosJ,etal.Encouragingearly defibrillation:theAmericanHeartAssociationandautomatedexternal defibrillators. AnnEmergMed. 1990;19(11):1245–8.

7.ChengA,MagidDJ,AuerbachM,etal.Part6:resuscitationeducation science:2020AmericanHeartAssociationguidelinesfor cardiopulmonaryresuscitationandemergencycardiovascularcare. Circulation. 2020;142(16_suppl_2):S551–79.

8.EdmondsonA.Psychologicalsafetyandlearningbehaviorinwork teams. AdmSciQ. 1999;44(2):350–83.

9.CookDA,BrydgesR,HamstraSJ,etal.Comparativeeffectivenessof technology-enhancedsimulationversusotherinstructionalmethods.A systematicreviewandmeta-analysis. SimulHealthc. 2012;7(5):308–20.

10.McGaghieWC,IssenbergSB,PetrusaER,etal.Effectofpracticeon standardizedlearningoutcomesinsimulation-basedmedicaleducation. MedEduc. 2006;40(8):792–7.

11.WayneDB,ButterJ,SiddallVJ,etal.Simulation-basedtrainingof internalmedicineresidentsinadvancedcardiaclifesupportprotocols:a randomizedtrial. TeachLearnMed. 2005;17(3):210–6.

12.EricssonKA.Deliberatepracticeandtheacquisitionandmaintenanceof expertperformanceinmedicineandrelateddomains. AcadMed. 2004;79(10Suppl):S70–81.

13.HuntEA,Duval-ArnouldJM,Nelson-McMillanKL,etal.Pediatric residentresuscitationskillsimproveafter “rapidcycledeliberate practice” training. Resuscitation. 2014;85(7):945–51.

14.LemkeDS,FielderEK,HsuDC,etal.Improvedteamperformance duringpediatricresuscitationsafterrapidcycledeliberatepractice comparedwithtraditionaldebriefing:apilotstudy. PediatrEmergCare. 2019;35(7):480–6.

15.LemkeDS,YoungAL,WonSK,etal.Rapid-cycledeliberatepractice improvestimetodefibrillationandreducesworkload:arandomized

controlledtrialofsimulation-basededucation. AEMEducTrain. 2021;5(4):e10702.

16.WonSK,DoughtyCB,YoungAL,etal.Rapidcycledeliberatepractice improvesretentionofpediatricresuscitationskillscomparedwith postsimulationdebriefing. SimulHealthc. 2022;17(1):e20–7.

17.GrossTI,AbrahamDG,KumarA,etal.Rapidcycledeliberatepractice (RCDP)asamethodtoimproveairwaymanagementskills – a randomizedcontrolledsimulationstudy. Cureus. 2019;11(9):e5546.

18.OliveiraHC,SouzaLC,LeiteTC,etal.Personalprotectiveequipmentin thecoronaviruspandemic:trainingwithrapidcycledeliberatepractice. RevBrasEnferm. 2020;73Suppl2(Suppl2):e20200303.

19.PascoalE,MuellerV,MoraisM.Rapidcycledeliberatepracticeversus traditionalsimulationtoimproveresidentperformanceinforceps deliveries. JObstetGynaecolCan. 2022;44(8):934–7.

20.deCastroLT,CoriolanoAM,BurckartK,etal.Rapid-cycledeliberate practiceversusafter-eventdebriefingclinicalsimulationin cardiopulmonaryresuscitation:aclusterrandomizedtrial. AdvSimul (Lond). 2022;7(1):43.

21.KirkpatrickJDandKirkpatrickWK.Associationfortalentdevelopment. In: Kirkpatrick’sFourLevelsofTrainingEvaluation(73-90).Alexandria, VA:ATDPress;2016.

22.StuartA.Atestforhomogeneityofthemarginaldistributionsina two-wayclassification. Biometrika. 1955;42(3–4):412–6.

23.CrowleyCP,SalciccioliJD,KimEY.TheassociationbetweenACLS guidelinedeviationsandoutcomesfromin-hospitalcardiacarrest. Resuscitation. 2020;153:65–70.

24.BhanjiF,DonoghueAJ,WolffMS,etal.Part14:Education:2015 AmericanHeartAssociationEducation:2015American HeartAssociationguidelinesupdateforcardiopulmonary resuscitationandemergencycardiovascularcare. Circulation. 2015;132(suppl2):S561–73.

25.ChanceyR,SampayoE,LemkeD,etal.Learners’ experiencesduring rapidcycledeliberatepracticesimulations:aqualitativeanalysis. Simul Healthc. 2019;14(1):18–28.

26.CheskesS,SchmickerRH,ChristensonJ,etal.Perishockpause:an independentpredictorofsurvivalfromout-of-hospitalshockablecardiac arrest. Circulation. 2011;124(1):58–66.

27.HuntEA,Duval-ArnouldJM,ChimeNO,etal.Integrationofin-hospital cardiacarrestcontextualcurriculumintoabasiclifesupportcourse:a randomized,controlledsimulationstudy. Resuscitation. 2017;114:127–32.

SimulationImprovesEmergencyMedicineResidents’ Clinical PerformanceofAortaPoint-of-CareUltrasound

BrandonM.Wubben,MD CoryWittrock,MD

SectionEditor:ShaneSummers,MD

UniversityofIowaHospitalsandClinics,DepartmentofEmergencyMedicine, IowaCity,Iowa

Submissionhistory:SubmittedAugust30,2023;RevisionreceivedDecember4,2023;AcceptedDecember5,2023 ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18449

Purpose: Usingpoint-of-careultrasound(POCUS)todiagnoseabdominalaorticaneurysm(AAA)isan essentialskillinemergencymedicine(EM).Whilesimulation-basedPOCUSeducationiscommonly used,thetranslationtoperformanceintheemergencydepartment(ED)isunknown.Weinvestigated whetheraddingcase-basedsimulationtoanEMresidencycurriculumwasassociatedwithchangesin thequantityandqualityofaortaPOCUSperformedbyresidentsintheED.

Methods: Acase-basedsimulationwasintroducedtoresidentdidacticsatouracademic,LevelItrauma center.Acaseofundifferentiatedabdominalpainwaspresented,whichrequiredexaminationofan ultrasoundphantomtodiagnoseanAAA,withahands-ondidactic.Wecomparedthequantity,quality, anddescriptiveanalysesofaortaPOCUSperformedintheEDduringthefourmonthsbeforeandafter thesimulation.

Results: Forparticipatingresidents(17/32),therewasan86%increaseintotalstudiesandan80% increaseinclinicalstudies.Onanopportunity-adjusted,per-residentbasis,therewasnosignificant differenceinmediantotalscansper100shifts(4.4[interquartilerange(IQR)0–15.8vs8.3[IQR]

3.3–23.6, P = 0.21)oraveragetotalqualityscores(3.2 ± 0.6vs3.2 ± 0.5, P = 0.92).Thetotalnumberof limitedorinadequatestudiesdecreased(43%vs19%, P = 0.02),andtheproportionofscanssubmitted byinternsincreased(7%vs54%, P =< .001).

Conclusion: Aftersimulationtraining,aortaPOCUSwasperformedmorefrequently,andEDinterns contributedahigherproportionofscans.Whiletherewasnoimprovementinquantityorqualityscoreson aper-residentbasis,thereweresignificantlyfewerincompleteorlimitedscans.[WestJEmergMed. 2024;25(2)205–208.]

Keywords: point-of-care;ultrasonography;simulation;abdominalaorticaneurysm;POCUS; emergencymedicine.

INTRODUCTION

Point-of-careultrasound(POCUS)oftheaortato diagnoseabdominalaorticaneurysm(AAA)isacore emergencyultrasoundapplicationandanessential componentofemergencymedicine(EM)residency education.1 Emergencydepartment(ED)POCUShas previouslybeenshowntohaveexcellentperformance characteristicsfortheevaluationofAAA.2 Simulationisa commonlyusededucationaltoolforresidentultrasound

education,butascopingreviewfoundthemajorityofprior studiesexaminedchangesinconfidence,knowledge,and skillsratherthanobjectiveclinicalperformance.3 However, simulationinadditiontoclinicaltraininghasbeenshownto beeffectiveattranslatingtoclinicalperformanceinother specialties,suchasforobstetricsandgynecologyresidents learningtransvaginalultrasound.4

Itiscriticalthatemergencyphysicianslearntoquickly identifyAAAatthebedside,asthisisatime-sensitiveand

potentiallydeadlydiagnosisrequiringagoaltimefrom presentationtoemergencysurgeryof <90minutes.5 We aimedtoinvestigatewhethertheadditionofcase-based ultrasoundsimulationtotheexistingEMresidency curriculumwasassociatedwithanincreaseinthequantity andqualityofaortaPOCUSperformedonEDpatients.

MATERIALSANDMETHODS

Design,Setting,andIntervention

Atouracademic,LevelItraumacenterwithathree-year EMresidencyprogramandadvancedpracticeprovider trainingprogram,EMresidentshaveexistingsimulationbasedlearningbuiltintotheirmonthlyacademicconference. However,theuseofPOCUSisnottypicallyincorporated intosimulationatourinstitution.InSpring2023,we introducedanewsimulatedcaserequiringtheuseofPOCUS fordiagnosisofAAAinthesettingofundifferentiated abdominalpain.Duringsimulation-basedlearning,residents aredividedintogroupsof5–8residentsofvaried postgraduateyear(PGY)andcyclebetweenthesimulation sessionandothereducationalactivities.Inadditiontoverbal promptsregardingcasehistory,physicalexam findings,and patientresponsestointerventions,residentswereaskedto useacart-basedultrasoundsystem(SonositePX,Fujifilm, Bothell,WA)toexamineanultrasoundphantomabdomen (41903–000,KyotoKagaku,Japan)withmultiple findings includinganinfrarenalAAAwithintermuralthrombus,free abdominal fluid,andnormalbowelsandrenalsystem.The caseconcludedwithahands-ondidacticledbyaPOCUS fellowship-trainedemergencyphysicianandincludedtimeat theendofthesimulationforhands-onscanningbyresidents.

UltrasoundStudyReviewandOutcomes

AllultrasoundstudiesperformedintheEDaresubmitted throughaqualityassuranceworkflowforreviewbyateamof POCUSfellowship-trainedfaculty.Allresidentswho participatereceivecreditforperformingthePOCUS.The submittingresidentwhoperformedthePOCUScompletesa worksheetdescribingthe findings,interpretation,andstudy limitations.Studiesaremarkedcompleteifresidentsindicate thataviewwasobtainedofthesuprarenalaorta,infrarenal aorta,andiliacbifurcationandincompleteifoneormoreof theseviewswasnotobtained.Imagesandworksheetsare thensentforreviewandsignaturetothefacultycaringforthe patientwiththeresident.Facultycaneitherplacethestudyin aneducationalarchive(iftheydidnotusethestudyfor medicaldecision-making)oraclinicalarchive(iftheyused thestudyformedicaldecision-makingandwishforthe imagestotransfertothepatient’shealthrecord).

Studiesarereviewedforqualityandassignedaquality scorefrom1(worst)to5(best)aswellasnotationofanyfalse positivesorfalsenegatives,withEMultrasoundfaculty servingasthegoldstandard.Qualityscores1and2are consideredinsufficientfordiagnosis,withscores

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue? Simulationhasincreasinglybeenusedto prepareEMresidentsforlesscommon conditions,suchasdiagnosingabdominal aorticaneurysmusingPOCUS.

Whatwastheresearchquestion?

Doescase-basedPOCUSsimulationaffect thequantityorqualityofaortaPOCUSthat residentsperformintheED?

Whatwasthemajor findingofthestudy?

AortaPOCUSincreased86%,andthe numberoflimitedorinadequatestudies decreased(43%vs19%,P = 0.02).

Howdoesthisimprovepopulationhealth?

AortaPOCUSsimulationtrainingmayhelp physicianswholessfrequentlyencounter aorticaneurysmtoidentifythis time-sensitivecondition.

of3–5consideredadequate.Weexaminedcumulative measuresofsensitivityandspecificitybeforeand afterthesimulation.

Analysis

WecomparedthemediannumberofaortaPOCUS studiesthatEMresidentsperformedintheEDinthefour monthspriortothesimulationsessiontothemediannumber ofexamsperformedinthefourmonthsfollowingthe simulationsession.Thesequantitieswerereportedasscans performedper100shiftsperresidentandcomparedusing Wilcoxonsigned-ranktest.Wecomparedmeanquality scoresoverthefourmonthspriortotheinterventiontothe qualityscoresoverthefourmonthsaftertheinterventionas describedabovewiththequantityofexamsusingapaired t -test.Proportionsoflimitedstudiesandtrainingyear distributionwerecomparedwithPearsonchi-square.The significancelevelofalltestswassetto0.05withBonferroni correctionappliedwhereappropriate.Analysiswas performedinSPSSforMacintosh,v28.0(SPSSInc, Chicago,IL).Thisstudyreceivedinstitutionalreviewboard approvalforwaiverofsignedinformedconsent.

RESULTS

Overhalfofresidents17/32(53%)participatedinthe simulationsessionandhadatleastoneclinicalshiftbefore

ClinicalEducationalTotal

APP,advancedpracticeproviderresident; PGY,postgraduateyear.

andafterthesimulationsession.Thedistributionofthe participatingresidentsasprimarystudyauthorsis demonstratedinthe Table,withasignificantincreaseinthe proportionofaortaPOCUSsubmittedbyinterns.Overall, therewasan86%increaseintotalstudiesandan80% increaseinclinicalstudiesafterthesession.However,when comparingonaper-residentbasiswhileadjustingforclinical opportunities,therewasnosignificantdifferencein mediantotalscanningfrequencyper100shifts

(4.4[interquartilerange(IQR) = 0–15.8]vs8.3[IQR = 3.3–23.6], P = 0.21).Therewasalsonosignificantchangein averagetotalqualityscoresonaperresidentbasis (3.2 ± 0.6vs3.2 ± 0.5, P = 0.92).

Therewerenofalsenegativeorfalsepositivesusing facultyreviewofimagesasthegoldstandard.Therewereno differencesintheproportionofstudieswithagreementvs disagreementwiththeresidentinterpretation(100%vs96%, P = 0.29).Therewasadecreaseinthetotalnumberoflimited orinadequatestudies(12/28(43%)vs10/52(19%), P = .02 [X 2]).Therewasnosignificantchangeintheproportionof clinicalstudiessubmittedas “limited” or “inadequate” (2/5 (40%)vs4/9(44%), P = 0.87[X 2]),butthenumberof educationalstudiessubmittedas “limited” or “inadequate” improved(10/23(44%)vs6/43(14%), P =<.001).

DISCUSSION

Overall,thetotalnumberofaortaPOCUSstudies performedintheEDafterthesimulationincreased,albeit withoutademonstrablechangeinquantityorqualityrating onaper-residentbasis.However,therewereanumberof positive findings,whichsupporttheinclusionofultrasound simulationinresidencytraining,includingasignificant increaseintheproportionofstudiescontributedbyinterns andasignificantdecreaseintheproportionofstudiesthat wereincompleteorlimited.

Itseemsunlikelythatthesignificantincreaseinintern POCUSstudieswasduetocontentmasterybasedon compoundingclinicalexperiencealone.AortaPOCUSisone oftheapplicationsrequiringthemostexperiencetogain proficiency,andwithpreviouslydemonstratedplateau

pointsininterpretationandacquisitionat66studiesand84 studies,respectively,whichwerenotapproachedbyanyone inourstudy.6 ThesamestudyfoundthataortaPOCUS qualityactuallydecreasesinitiallywithincreasingnumberof scansbeforeiteventuallyimprovesabovebaseline,which maybecontributingtotheabsenceofimprovementin medianqualityscoresseeninourstudy.6

MuchofthepublishedresearchregardingPOCUS simulationreportsoutcomesrelatedtotheassessmentof learnerexperienceandskillperformanceoutsidetheclinical context.7,8 Whiletheseoutcomesareimportant,thereisa desiretoassessmoretranslationaloutcomesresultingfrom simulationinterventions.9,10 Therearefewtranslational studiesavailablefordirectcomparisontothecurrentstudy. Oursimulationexperiencewasdeliveredasasinglesession, whichislesstime-intensivethanapriorstudyofEMinterns thatfoundpositiveclinicaleffectsofsimulation-based masterylearningonperformanceoffocusedassessmentwith sonographyintrauma.11 OutsideofEM,arandomizedtrial ofultrasoundsimulationforobstetricsandgynecology residentsthatwasalsomorelongitudinalthanthecurrent studyalsofoundpositiveclinicaleffectsofearlysimulation training,inadditiontoclinicalpracticein first-year residents.4 Furtherstudyisneededtodeterminewhetherthe case-basedsimulationapproachinourstudywouldbemore successfulifthetrainingweremoretimeintensive,more longitudinal,andmosttargetedtowardjuniorlearners.

LIMITATIONS

Our findingsaresubjecttothelimitationsofabefore-andafterstudy,includingthepossibilitythatotherfactorsmay havecontributedtotheobservedchangesotherthanthe simulation;however,wearenotawareofanyothertargeted efforttoeducateourresidentsaboutaortaPOCUSduring thestudyperiod,andbelieveitislikelymostchangeswere associatedwiththesimulation.Second,thiswasastudyof residentswhosecategorizationofstudyintentwassubjectto theirsigningfaculties’ preferences.Therefore,weincluded botheducationalandclinicalarchivestudiestoprovideafair portfolioofeachresident’swork,althoughsomeeducational

Table. Numberofaortapoint-of-careultrasoundstudiessubmittedbeforeandaftersimulationtraining,stratifiedbytraininglevelofthe primarystudyauthorbyarchive.(*):P =< .05withBonferronicorrection.

Wubbenetal. EMResidents’ ClinicalPerformanceofAortaPOCUS

studiesmayhavenotbeenintendedforpatientcare.Third, EMresidentsdonotgettochoosewhichpatientstheytake careofintheED(becauseemergenciesareunplannedand unpredictable);sosomeresidentslikelyhadslightlymore exposuretopatientswithindicationsforaortaPOCUSthan others.Inaddition,whilewewereabletoadjustforclinical opportunitiesbasedontimespentintheED,residentsoften seemorepatientsperhourastheyadvancethroughtraining, andwewereunabletoaccountfortotalpatientsseenduring thestudyperiod.Evenconsideringtheselimitations,we believethedatapresentedprovidesanaccuratereal-world assessmentofscanningfrequencyandqualityonEDpatients byEDresidents.

CONCLUSION

Inthefourmonthsfollowingacase-basedsimulationto diagnoseabdominalaorticaneurysmusingpoint-of-care ultrasound,theproportionofaortaPOCUSstudies performedintheEDbyinternsincreasedsignificantly,and theproportionofstudiesthatwereincompleteorlimited significantlydecreased.Whiletherewasnooverallincrease inthemediannumberofscansormeanqualityscoreswhen adjustedforclinicalopportunitiesonaper-residentbasis, amongresidentsasawholetherewasan86%increasein submittedaortaPOCUSstudies.

ACKNOWLEDGMENTS

TheauthorswouldliketothanktheCarverCollegeof MedicinefortheirEducationalInnovationandScholarship Granttosupportthepurchaseoftheultrasoundphantom usedinthisstudy,JohnKeechforhisassistanceingrant writing,PriyankaVakkalankaforherassistanceingrant writingandmanuscriptreview,andDanielMillerand NathanielShekemfortheirassistanceimplementing thesimulation.

AddressforCorrespondence:BrandonM.Wubben,MD,University ofIowaHospitalsandClinics,DepartmentofEmergencyMedicine, 200HawkinsDrive,IowaCity,IA52242.Email: brandon-wubben@ uiowa.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.This workwassupportedbyanEducationalInnovationandScholarship

GrantprovidedbytheCarverCollegeofMedicineattheUniversity ofIowa.

Copyright:©2024Wubbenetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.AmericanCollegeofEmergencyPhysicians.Policystatement: UltrasoundGuidelines:Emergency,Point-of-Care,andClinical UltrasoundGuidelinesinMedicine.2023.Availableat: https://www. acep.org/siteassets/new-pdfs/policy-statements/ultrasoundguidelines–emergency-point-of-care-and-clinical-ultrasoundguidelines-in-medicine.pdf.AccessedJuly1,2023.

2.RubanoE,MehtaN,CaputoW,etal.Systematicreview:emergency departmentbedsideultrasonographyfordiagnosingsuspected abdominalaorticaneurysm. AcadEmergMed.2013;20(2):128–38.

3.PoSawLL,WubbenBM,BertucciN,etal.Teachingemergency ultrasoundtoemergencymedicineresidents:ascopingreviewof structuredtrainingmethods. JAmCollEmergPhysiciansOpen 2021;2(3):e12439.

4.TolsgaardMG,RingstedC,DreislerE,etal.Sustainedeffectof simulation-basedultrasoundtrainingonclinicalperformance:a randomizedtrial. UltrasoundObstetGynecol.2015;46(3):312–8.

5.ChaikofEL,DalmanRL,EskandariMK,etal.TheSocietyforVascular Surgerypracticeguidelinesonthecareofpatientswithanabdominal aorticaneurysm. JVascSurg.2018;67(1):2–77.e2.

6.BleharDJ,BartonB,GaspariRJ.Learningcurvesinemergency ultrasoundeducation. AcadEmergMed.2015;22(5):574–82.

7.SinghJ,MaternLH,BittnerEA,etal.Characteristicsofsimulationbasedpoint-of-careultrasoundeducation:asystematicreviewof MedEdPORTALcurricula. Cureus.2022;14(2):e22249.

8.LewissRE,HoffmannB,BeaulieuY,etal.Point-of-careultrasound education:theincreasingroleofsimulationandmultimediaresources. JUltrasoundMed.2014;33(1):27–32.

9.McGaghieWC,DraycottTJ,DunnWF,etal.Evaluatingtheimpactof simulationontranslationalpatientoutcomes. SimulHealthc 2011;6(Suppl):S42–7.

10.LienWC,LinP,ChangCH,etal.Theeffectofe-learningon point-of-careultrasoundeducationinnovices. MedEducOnline 2023;28(1):2152522.

11.SmithS,LoboV,AndersonKL,etal.Arandomizedcontrolledtrialof simulation-basedmasterylearningtoteachtheextendedfocused assessmentwithsonographyintrauma. AEMEducTrain 2021;5(3):e10606.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

FoundationsofEmergencyMedicine:Impactofa Standardized,Open-access,CoreContentCurriculum onIn-TrainingExamScores

JaimeJordan,MD,MAEd*

NatashaWheaton,MD*

NicholasD.Hartman,MD,MPH†

DanaLoke,MD,MS‡

NathanielShekem,MPAS§

AnwarOsborne,MD,MPM∥

P.LoganWeygandt,MD,MPH¶

KristenGrabowMoore,MD,MEd∥

SectionEditor:DanielleHart,MD

*UniversityofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,LosAngeles,California

† WakeForestUniversitySchoolofMedicine,DepartmentofEmergency Medicine,Winston-Salem,NorthCarolina

‡ NorthwesternUniversityFeinbergSchoolofMedicine,Departmentof EmergencyMedicine,Chicago,Illinois

§ UniversityofIowa,DepartmentofEmergencyMedicine,IowaCity,Iowa

∥ EmoryUniversity,DepartmentofEmergencyMedicine,Atlanta,Georgia

¶ JohnsHopkinsUniversitySchoolofMedicine,Baltimore,Maryland

Submissionhistory:SubmittedJune28,2023;RevisionreceivedDecember9,2023;AcceptedJanuary12,2023

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18387

Introduction: Learnersfrequentlybenefitfrommodalitiessuchassmall-group,case-basedteaching andinteractivedidacticexperiencesratherthanpassivelearningmethods.Thesecontemporary techniquesarefeaturesofFoundationsofEmergencyMedicine(FoEM)curricula,andparticularlythe FoundationsI(F1)course,whichtargets first-yearresident(PGY-1)learners.TheAmericanBoardof EmergencyMedicineadministersthein-trainingexam(ITE)thatprovidesanannualassessmentofEMspecificmedicalknowledge.WesoughttoassesstheeffectofF1implementationonITEscores.

Methods: WeretrospectivelyanalyzeddatafrominternsatfourEMresidencyprogramsaccreditedbythe AccreditationCouncilforGraduateMedicalEducation.Wecollecteddatain2021.Participatingsiteswere geographicallydiverseandincludedthree-andfour-yeartrainingformats.Wecollecteddatafrominterns twoyearsbefore(controlgroup)andtwoyearsafter(interventiongroup)implementationofF1ateachsite. YearofF1implementationrangedfrom2015–2018atparticipatingsites.Weabstracteddatausinga standardformincludingprogram,ITErawscore,yearofITEadministration,USMedicalLicensingExam Step1score,Step2ClinicalKnowledge(CK)score,andgender.Weperformedunivariableand multivariablelinearregressiontoexploredifferencesbetweeninterventionandcontrolgroups.

Results: Wecollecteddatafor180PGY-1s.Step1andStep2CKscoresweresignificantpredictorsof ITEinunivariableanalyses(bothwith P < 0.001).AfteraccountingforStep1andStep2CKscores,we didnot findF1implementationtobeasignificantpredictorofITEscore, P = 0.83.

Conclusion: ImplementationofF1curriculadidnotshowsignificantchangesinperformanceontheITE aftercontrollingforimportantvariables.[WestJEmergMed.2024;25(2)209–212.]

INTRODUCTION

Residencyprogramsprovideeducationandtrainingto developcompetentphysicians.Boardcertificationin emergencymedicine(EM)requirescompletionofan

AccreditationCouncilforGraduateMedicalEducation (ACGME)-accreditedtrainingprogramandapassingscoreon theQualifyingExamination(QE)andOralCertification Examination(OCE)administeredbytheAmericanBoardof

EmergencyMedicine(ABEM).1,2 TheABEMIn-training Examination(ITE)isanimportanttoolusedbytraining programstoassessmedicalknowledgeandprepareresidents fortheQE.1,3 TheITEisdesignedtoreflectthecontentofthe ModelofClinicalPracticeofEmergencyMedicine(EM Model)andhaspredictivevalueinestimatingthelikelihoodof individualresidentspassingtheQE.3 Priorliteraturesuggests thatclinicalexposurealoneleavessignificantgapsin fundamentalknowledgedefinedbytheEMModel.4 Residency didacticcurriculaprovideanopportunitytosupplementcore knowledge;however,thebestmethodsforproviding instructionoutsideoftheclinicalsettingandpreparingtrainees forsuccessfulperformanceontheITEareunknown.

FoundationsofEmergencyMedicine(FoEM)isa national,free,open-access,onlineEMcurriculumthathas beenwidelyadoptedintheUnitedStates.5,6 FoEMbecame availablein2015;registrationforuseofFoEMcoursesfor the2022–2023academicyearincluded237registered educationalprograms,serving6,326residentphysicians.5,6 FoEMoffersstandardized,level-specific,corecontentfor EMresidentsusinglearner-centriceducationalstrategies thathavebeenshowntobenefitlearningsuchassmall-group discussion,peerlearning,andindividualizedguidance.5–11

FoundationsI(F1)isa flippedclassroom,case-basedcourse targetingpostgraduateyear(PGY)-1residentsthatincludes a30-unit,systems-basedcurriculumoffundamentalcontent intheEMModel.5,6,12 Priorliteraturedemonstratespositive effectsofthe flippedclassroommodelonlearning outcomes.13–15

TheF1curriculumincludescuratedselfstudyresourcescalled “LearningPathways” forlearnersto reviewpriortodidacticmeetings,inwhichresidentswork throughmultipleF1caseswithaknowledgeablefacilitator providinginformationinanoral-boardsstyleformat.6 The F1summarizesessentiallearningpointsandsharesthem withlearnersto fillknowledgegapsandallowforspaced repetition.6 AlthoughtheF1curriculumisnotspecifically designedforITEreview,third-partypairedassessmentsfor eachunithavebeenavailableforusesince2017.6

LimitedoutcomedataofFoEMF1establishedquality anddemonstratedhighsatisfactionamongfacultyleaders andresidentlearners.5,6 However,therehasnotbeenan assessmentofobjectivemeasuressuchasmedicalknowledge andITEperformanceThisinformationcanprovideamore comprehensiveassessmentofthevalueofimplementingsuch aprogram.Inthisstudy,wesoughttoevaluatetheeffectof F1courseimplementationonITEperformanceinthe PGY-1year.Wehypothesizedthatimplementationofthe structuredF1curriculumwouldleadtoimproved performanceontheITE.

METHODS

WeperformedaretrospectivecohortstudyofITEdata collectedfromPGY-1residentsatfourACGME-accredited EMresidencyprogramsintheUnitedStatesbeforeandafter

implementationoftheFoEMF1curriculum.Weselected participatingsitesthatweregeographicallydiverseand included3-and4-yeartrainingformats.Wecollecteddatain December2021.AllPGY-1residentsatparticipatingsites duringthestudyperiodwereeligibletoparticipate.We excludedPGY-1residentswhoweremissingdata.

Wedeterminedthattodetecta5%differenceinITEscore with80%powerandanalphaof0.05,wewouldneedto enroll81participantsineachgroup(controland intervention)foratotalof162participants.Ourcontrol groupconsistedofdatafromPGY-1residentsforthetwo yearspriortoimplementationateachsite.Ourintervention groupconsistedofdatafromPGY-1residentsforthetwo yearsafterimplementationateachsite.YearofF1 implementationrangedfrom2015–2018atparticipating sites.Theleadauthorfromeachsiteabstracteddatausinga standardformthatincludedprogram,ITErawtotalscore, yearofITEadministration,UnitedStatesMedicalLicensing Examination(USMLE)Step1score,USMLEStep2 ClinicalKnowledge(CK)score,andresidentgender.Priorto dataabstraction,theauthorgroupreadeachitemonthe formaloudandtrialedabstractingasmallportionof representativedatatoensureclarityofmeaningand consistencyinprocess.

Wecalculateddescriptivestatisticsfordemographicdata andITEperformance.Weperformedregressionanalysesto exploredifferencesbetweentheinterventionandcontrol groups.We firstperformedunivariablelinearregression analysesforvariablesincludingimplementationofF1, residencyprogram,yearofITEadministration,USMLE Step1score,USMLEStep2score,andresidentgenderwith ITErawscoreasouroutcomeofinterest.Weincluded variableswitha P -value < 0.1intheunivariableregressionin amultivariablelinearregressionwiththesameoutcome variable.Weconsideredvariableswitha P -valueof < 0.05in themultivariablemodelasstatisticallysignificant.We performedallanalysesinSPSSv27.0(IBMCorporation, Armonk,NY).

ThisstudywasapprovedbytheInstitutionalReview BoardoftheDavidGeffenSchoolofMedicineatUCLA.

RESULTS

Weabstracteddatafromatotalof224interns.We excluded44internswhoweremissingdata.Weanalyzed datafrom180interns(88pre-implementationand92postimplementation)whohadcompletedata.Thedemographics ofparticipantswithcompletedataareshownin Table1.The meanITErawscoreforinternsinthecontrolgroupwas 72.15 ± 6.72.ThemeanITEscoreforinternsinthe interventiongroupwas72.74 ± 7.93.Intheunivariable regressionanalyses,onlyUSMLEStep1andUSMLEStep2 CKscoresyielded P -valuesof < 0.1(Table2).Becauseour hypothesiscenteredontheimpactofimplementationofthe F1curriculumonITEscores,weforcedthisvariableasthe

Gender

Controlgroup n(%)totaln = 88

Interventiongroup n(%)totaln = 92

Male3231

Female5660

Non-binary01

MeanUSMLE

MeanUSMLE

232(14.26)232(15.59)

USMLE,UnitedStatesMedicalLicensingExamination.

Residencyprogram0.22 YearofITEadministration0.14

USMLEStep1score

Residentgender0.24

USMLE,UnitedStatesMedicalLicensingExamination; ITE, in-trainingexam; CK,clinicalknowledge.

lastvariableafterblockentryofvariablesofUSMLEStep1 scoreandUSMLEStep2CKscoreinthemultivariable regressionanalysis,despiteithavinga P -valueof0.59inthe univariableanalysis.AftercontrollingforStep1scoreand Step2CKscore,F1implementationwasnotasignificant predictorofITEscore,Rsquarechange = 0, P = 0.83.The datasatisfiedallassumptions.

DISCUSSION

OurstudydemonstratesthatbothStep1andStep2CK weresignificantpredictorsofITEscore.Thisisconsistent withpriorliteratureinmultiplespecialtiesdemonstrating associationsbetweenUSMLEscoresandITE performance.16–19 Wefoundthatourinterventiongrouphad aslightlyhigherrawITEscoreshowever,aftercontrolling forUSMLEscores,thisincreasewasnotstatistically significant,despitebeingadequatelypowered.Thiswas somewhatsurprisinggiventhatF1providesaconsistent structureandcomprehensivecoverageofcontentintheEM modelandalsoincorporatesteachingmethodsthathave beenshowntoenhancelearning.2,6–11 However,ourresults alignwithpreviousstudies,whichhavedemonstratedthat changesincurriculumwerenotassociatedwithsignificant differencesinITEperformance.20,21 Specifically,converting

anhourofsynchronousdidacticconferencetoasynchronous learning,andconvertingconferencelecturestosmallgroup, “flipped-classroom” stylelearninghavepreviouslybeen foundtohavenosignificanteffectonITEscores.20,21

ItisimportanttonotethattheobjectiveofF1isto improveEMcoreknowledgeandapplicationintheclinical environmentandisnotspecificallytargetedtowardsITEtest preparationorperformance.Additionally,performanceon theITEmaynotcomprehensivelyrepresentlearner knowledgeofEM.Thismaybeonereasonthatwedidnot findsignificantchangesinITEperformance.Additionally, variableimplementationandusageofF1atdiffering programscouldinfluencepotentialgains.Althoughthe FoEMcoursesarestandardized,participatingprograms mustaddresstheirownuniqueneedsandbarriers;thismay resultinvariabilityincourseimplementation,including variableuseof flipped-classroomstyleasynchronous resourcesandpairedassessments.Itisalsoimportanttonote thattheITEisadministeredinFebruaryofeachyear;thus, participatingPGY-1residentsinthisstudywereonly exposedtoapproximatelysevenmonthsoftheyear-longF1 curriculumpriortotheITE.

Itispossiblethatadditionalimprovementsmaybeseenwith additionaltimespentinthecurriculum.Thenonsignificant improvementseeninthisstudymaybeaugmentedwith implementationofFoundationsII(F2),whichisdesignedfor PGY-2residents,andFoundationsIII(F3),whichisdesigned forPGY-3andPGY-4residents.Theseoutcomesmeritfurther investigation.Whileourstudydidnot findasignificant increaseinITEscorescomparedtostandardcurricula,itwas notworsethanstandardpracticeandhasadditionalbenefitsof afree,standardized,pre-packaged,high-quality,adaptable formatwithuseracceptability.6

Overall,theresultsofthisstudyprovideimportant insightsforboththenumerousprogramsalreadyusing FoEMandthoseEMresidenciesconsideringincorporating itintotheirtrainingprograms.6 Inadditiontoprior feasibilityanduseracceptabilitydata,thisstudyprovidesan evaluationofobjectiveoutcomes,namelyknowledge,the firstlevelinMiller’spyramidofclinicalcompetence.6,22 Therearestillmanyunansweredquestions.Further investigationintotheeffectoftheF1curriculumonABEM QEandOCEperformanceshouldbepursued.Additionally, asFoEMisdesignedtosupportknowledgeapplicationinthe clinicalspace,futureworkcouldevaluatetheimpactof FoEMonotherdomainsofresidentperformance.

LIMITATIONS

Thisstudyhaslimitations.Theremaybeconfoundersnot accountedforinouranalysisthatcouldhaveinfluenced results.WedidnotcollectdataonspecificITEpreparation curriculaatparticipatingsites,individualusageofexternal ITEpreparationmaterialsoutsideoftrainingprogram curricula,timespentusingF1curriculum,useofpaired

Table1. Demographicdataofparticipatinginterns.

Table2. Resultsofunivariableregressionanalysisof recordedvariables.

Jordanetal. ImpactofStandardized,Open-access,CoreContentCurriculumonITEScores

assessments,totalnumberofF1unitscompletedby participatingresidents,ortimespentstudyingforITEin general.However,tothebestofourknowledge,therewere noothermajorchangestothesite’sdidacticcurriculumor methodsofpreparingtraineesfortheITEduringthestudy period.AlthoughtheF1courseincludesstandardized content,participatingprogramsmustaddresstheirown uniqueneedsandvariablesthatimpacttheconsistencyof courseadministration.Theremaybedifferencesinthe personnelwhodeliverthecontent,attendancerequirements, etc,whicharenotaccountedforinourstudy.Theresultsseen inthisstudymaynottransfertoothersiteswhereadherence toimplementationguidelinesismoreorlessconsistent.

CONCLUSION

OurstudysuggeststhattheFoEMF1curriculumisnot associatedwithsignificantchangesinperformanceonthe ITEinEMtrainingprogramsaftercontrollingforimportant variables.Theseresultsmayinformtheuseand implementationofFoEMcoursesinEMtrainingprograms.

AddressforCorrespondence:JaimeJordan,MD,MAEd,University ofCaliforniaLosAngeles,DavidGeffenSchoolofMedicine, DepartmentofEmergencyMedicine,924WestwoodBlvd.,Suite 300,LosAngeles,CA90024.Email: jaimejordanmd@gmail.com

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Jordanetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.ACGME.MilestonesbySpecialty:EmergencyMedicine.Accreditation CouncilforGraduateMedicalEducation.2021.Availableat: https://www.acgme.org/globalassets/pdfs/milestones/ emergencymedicinemilestones.pdf.AccessedNovember,2022.

2.AmericanBoardofEmergencyMedicine.BecomeCertified. Availableat: https://www.abem.org/public/become-certified AccessedJune23,2023.

3.AmericanBoardofEmergencyMedicine.In-TrainingExam.Available at: https://www.abem.org/public/for-program-directors/in-trainingexamination.AccessedJune23,2023.

4.BischofJJ,EmersonG,MitzmanJ,etal.Doestheemergencymedicine in-trainingexaminationaccuratelyreflectresidents’ clinical experiences? AEMEducTrain. 2019;3(4):317–22.

5.FoundationsofEmergencyMedicine.Availableat: https:// foundationsem.com/.AccessedJune23,2023.

6.MooreKG,KettererA,WheatonN,etal.Development,implementation, andevaluationofanopenaccess,level-specific,corecontent curriculumforemergencymedicineresidents. JGradMedEduc. 2021;13(5):699–710.

7.TwengeJM.Generationalchangesandtheirimpactintheclassroom: teachingGenerationMe. MedEduc. 2009;43(5):398–405.

8.Moreno-WaltonL,BrunettP,AkhtarS,etal.Teachingacrossthe generationgap:aconsensusfromtheCouncilofEmergencyMedicine ResidencyDirectors2009AcademicAssembly. AcadEmergMed. 2009;16(Suppl2):S19–24.

9.HartDandJoingS.TheMillennialGenerationand “thelecture.” Acad EmergMed. 2011;18(11):1186–7.

10.RobertsDH,NewmanLR,SchwartzsteinRM.Twelvetipsforfacilitating Millennials’ learning. MedTeach. 2012;34(4):274–8.

11.CooperAZandRichardsJB.Lecturesforadultlearners:breakingold habitsingraduatemedicaleducation. AmJMed. 2017;130(3):376–81.

12.BeesonMS,AnkelF,BhatR,etal.The2019modeloftheClinical PracticeofEmergencyMedicine. JEmergMed. 2020;59(1):96–120.

13.KingAM,GottliebM,MitzmanJ,etal.Flippingtheclassroomingraduate medicaleducation:asystematicreview. JGradMedEduc. 2019;11(1):18–29.

14.ChenF,LuiAM,MartinelliSM.Asystematicreviewoftheeffectiveness of flippedclassroomsinmedicaleducation. MedEduc. 2017;51(6):585–97.

15.ChenKS,MonrouxeL,LuYH,etal.Academicoutcomesof flipped classroomlearning:ameta-analysis. MedEduc. 2018;52(9):910–24.

16.FeningK,HorstAV,ZirwasM.CorrelationofUSMLEStep1scoreswith performanceondermatologyin-trainingexaminations. JAmAcad Dermatol. 2011;64(1):102–6.

17.NelsonMandCalandrellaC.DoesUSMLEStep1&2scorespredict successonITEandABEMQualifyingExam:areviewofanemergency medicineresidencyprogramfromitsinception. AnnEmergMed. 2017;70(4):58–9.

18.PatzkowskiMS,HauserJM,LiuM,etal.Medicalschoolclinical knowledgeexamscores,notdemographicorotherfactors,associated withresidencyin-trainingexamperformance. MilMed. 2023;188(1-2):e388–91.

19.PerezJAJrandGreerS.CorrelationofUnitedStatesMedicalLicensing examinationandinternalmedicinein-trainingexaminationperformance. AdvHealthSciEducTheoryPract. 2009;14(5):753–8.

20.KingAM,MayerC,BarrieM,etal.Replacinglectureswithsmallgroups: theimpactof flippingtheresidencyconferenceday. WestJEmergMed. 2018;19(1):11–7.

21.WrayA,BennettK,Boysen-OsbornM,etal.Efficacyofan asynchronouselectroniccurriculuminemergencymedicineeducation intheUnitedStates. JEducEvalHealthProf. 2017;14:29.

22.MillerGE.Theassessmentofclinicalskills/competence/performance. AcadMed. 1990;65(9Suppl):S63–7.

EDUCATION SPECIAL ISSUE:BRIEF RESEARCH REPORT

IntegratingHospiceandPalliativeMedicineEducationWithinthe AmericanBoardofEmergencyMedicineModel

RebeccaGoett,MD*

JasonLyou,MD†

LaurenR.Willoughby,MD†

DanielW.Markwalter,MD‡§

DianeL.Gorgas,MD†

LaurenT.Southerland,MD,MPH†

*RutgersNewJerseyMedicalSchool,DepartmentofEmergencyMedicine, Newark,NewJersey

† TheOhioStateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,Columbus,Ohio

‡ UniversityofNorthCarolinaSchoolofMedicine,DepartmentofEmergency Medicine,ChapelHill,NorthCarolina

§ UniversityofNorthCarolinaSchoolofMedicine,UNCPalliativeCareProgram, ChapelHill,NorthCarolina

SectionEditor:ChrisMerritt,MD

Submissionhistory:SubmittedAugust29,2023;RevisionreceivedNovember20,2023;AcceptedJanuary12,2023

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18448

Background: Hospiceandpalliativemedicine(HPM)isaboard-certi fiedsubspecialtywithinemergency medicine(EM),butpriorstudieshaveshownthatEMresidentsdonotreceivesufficienttraininginHPM. ExpertsinHPM-EMcreatedaconsensuslistofcompetenciesforHPMtraininginEMresidency.We evaluatedhowtheHPMcompetenciesintegratewithintheAmericanBoardofEmergencyMedicine Milestones,whichincludetheModeloftheClinicalPracticeofEmergencyMedicine(EMModel)andthe knowledge,skills,andabilities(KSA)list.

Methods: ThreeemergencyphysiciansindependentlymappedtheHPM-EMcompetenciesontothe 2019EMModelitemsandthe2021KSAs.Discrepancieswereresolvedbyafourthindependent reviewer,andthe finalmappingwasreviewedbyallteammembers.

Results: TheEMModelincluded78%(18/23)oftheHPMcompetenciesasadirectmatch,andwe identifiedrecommendedareasforincorporatingtheother five.TheKSAsincluded43%(10/23).Most HPMcompetenciesincludedintheKSAsmappedontoatleastonelevelB(minimalnecessaryfor competency)KSA.ThreeHPMcompetencieswerenotclearlyincludedintheEMModelorintheKSAs (treatingend-of-lifesymptoms,caringfortheimminentlydying,andcaringforpatientsunder hospicecare).

Conclusion: ThemajorityofHPM-EMcompetenciesareincludedinthecurrentEMModelandKSAs andcorrespondtoknowledgeneededtobecompetentinEM.ProgramsrelyingontheEMMilestonesto plantheircurriculumsmaymisstraininginsymptommanagementandcareforpatientsattheendoflife orwhoareonhospice.[WestJEmergMed.2024;25(2)213–220.]

INTRODUCTION

Athirdofadultswhodiewillreceiveemergency departmentcareinthemonthpriortotheirdeath.1 Emergencyphysiciansneedtrainingtoprovidethehighquality,goal-concordantcarethatthesepatientsdeserve. Hospiceandpalliativemedicine(HPM)isasubspecialtyof emergencymedicine(EM)thataddsanadditionalfocuson

symptommanagement,goal-concordantcare,andqualityof life,especiallyforpatientswithchronicdiseaseorlifethreateningconditions,orwhoareattheendoflife.2 Prior researchhasshownthatcurrentEMresidencytraininglacks instructioninHPM.3–7 Toaddressthis,theAmerican CollegeofEmergencyPhysiciansPalliativeMedicineSection publishedalistof23criticaldevelopmentalmilestonesin

HPMtrainingforEMresidents.8 However,itisunclearhow besttointegratetheserecommendationsintoanEM residencycurriculum.

ManyEMresidencycurriculumsarebasedonthe knowledgeneededtopasstheEMboardcertificationexams. ThisknowledgeiscodifiedintheAmericanBoardof EmergencyMedicine(ABEM)ModeloftheClinical PracticeofEmergencyMedicine(EMModel)andalistof knowledge,skills,andabilities(KSA).9,10 TheEMmodel alongwiththeKSAsarethefoundationaldocumentsusedto createtheEMMilestones,acompendiumubiquitously employedinbothEMtrainingandassessment.Ourgoalin thisstudywastodeterminewheretheHPMcompetencies fit orcould fitwithintheEMModelandKSAs.Thismapping couldhelpguidecurriculumdesignortheincorporationof theHPMcompetenciesintotestingcontent.

METHODS

Thisstudywasnothumansubjectsresearchandwas deemedexemptfrominstitutionalreviewboardreview.We comparedthe2019EMModelandthe2021KSAstothe HPMcompetencies.TheHPMcompetencieswereassigned numerals.TheEMModelitemswereannotatedbytheir numberandcategory.ThenotationsfortheKSAcategories andcodeswereuseddirectlyfromthe2021document.We dividedtheKSAsintooverarchingcategories(eg,diagnosis, pharmacotherapy,reassessment)whichwethenfurther dividedintosetsofcompetencieswithinthatcategory.10 Eachcompetencywasgivenahierarchyintraining correspondingtoanalphabeticlevel(withAthemost advancedlevelofcompetencyandEtheleast).LevelAis reservedforadvancedknowledgeorskills.LevelBisthe minimalcompetencylevel,definedastheminimumskilllevel everyEMresidentshouldattaintograduate.LevelsC,D, andEareskillsinthedevelopmentofreachinglevelB.

Asthistypeofanalysishasnotbeendonebefore,weuseda sequentialapproachwithinitialindependentreviewers,a mediatorstep,andthen finalconsensusgroupdiscussion. Theconsensusgroupresultswerethenreviewedbytwo independentexternalexperts.Inthe firstphaseofconsensus mapping,tworesidents(EMpostgraduateyear(PGY)-2and EM/internalmedicine(PGY-4)andanEMattending independentlymappedpalliativecarecompetenciesusinga MicrosoftExcelspreadsheet(MicrosoftCorporation, Armonk,NY).Thethreeinitialconceptmappershad independentdatasheetsandwereblindedtoeachother’ s determinations.Acompetencycouldmapontomorethan oneareaoftheEMModel.First,keywordsfromeachHPM competencyweresearchedforintheEMModel.Ifno matcheswerefound,theEMModelwasreviewedlinebyline todeterminewhethertherewereconceptualmatches.Ifthere wasnodirectmatch,buttheHPMcompetencycouldbe incorporatedunderatopic,thiswaslistedasapotentialarea forincorporation.

Anytopicthatdidnothaveatleast2/3agreementonthe initialindependentreviewwasreviewedbyafourth emergencyphysicianwithexpertiseinEMresidenteducation andEMModeldevelopment.Shewasblindedtotheinitial reviewer’snamesbutdidhavetheirresults.Thefullgroup metandreviewedallthemappinguntilconsensuswas reached.Theconsensustableswerethenreviewed independentlybytwoadditionalexternalHPMboardcertifiedEMattendingsinvolvedinresidenteducationattwo differentEMresidencyprograms.Thesameprocesswasused formappingtheKSAs.

RESULTS

IncorporationintotheEmergencyMedicineModel

Fifty-oneof963EMModelitemsweretaggedinthe independent firstroundofmapping,with98.7%consensus (951/963)betweentheinitialthreeindependentreviewerson whetheranitemwasorwasnottaggedasamatch.The final reviewbytheindependentHPM-boardedEMattendingsdid notresultinadjustmentstoanyoftheexistingmappingbut didaddtothepotentialareasof fitfortheHPM competenciesthatdidnotdirectlymatchontotheEM Model. Table1 liststhecompetenciesincludedinthe2019 EMModel(18/23,78%).Manycompetencies fitinto EM Modelcategory20:OtherCoreCompetencies section, whichincludescommunicationskills,transitionsofcare, culturalcompetency,andhealthcarecoordination.

Discrepancydiscussionscenteredaroundmanagementvs diagnosis.Thecompetency HPM2:Treatingdistressing symptoms(eg,nausea/vomiting,dyspnea) wasfeltto fitby keywordmatchunderEMModelcategory 1.0Signs, SymptomsandPresentations .However,thatcategorydoes notmentiontreatmentofsymptomsdirectly.Similarly, HPM18:ComplicationsofCancer couldmaptomany itemsintheEMmodel,butagainreferstopalliative managementofcancercomplicationsratherthandiagnosis.

PotentialAreasofFitintheEmergencyMedicineModel

FiveHPMcompetenciesdidnot fitintotheEMModel. The firsttwo, HPM7:Treatingcommonend-of-life symptoms and HPM8:Carefortheimminentlydying (expectingdeathwithinhourstodaysorrecentlydeceased patientandtheirfamilymembers) ,couldbetaughtunder EMModelitem 20.4.4.2.2:Systems-basedPractice: Withdrawalofsupport. ThisEMModelitemcouldbe clarifiedtoensurethatitincludessymptomcontrolandendof-lifecare.Thenext, HPM11:Caringforpatientsunder hospicecare, couldbetaughtwhenteaching 20.4.4.2.3: Systems-basedPractice:HospiceReferral. However,the hospice-referralEMModelitembettermappedonto HPM 17 ,whichincludesassessingforandinitiatinghospice referrals.Theteamfeltthatidentifyingandreferringpatients tohospicewasaseparateskillsetthancaringforpatientson hospice.ThelasttwoHPMcompetencieswithoutaclear

Hospiceand palliative competencyDescriptionEMmodelitem

1Paincontrol:a.chronicpain,b.malignant andnon-malignantpain.

19.3.1Anesthesiaandacutepain management-regionalanesthesia

19.3.2Anesthesiaandacutepain management-proceduralsedation

19.3.3Anesthesiaandacutepain management-analgesia

2Treatingdistressingsymptoms (eg,nausea/vomiting,dyspnea)

3Difficultcommunication:a.deliveryofbad news(eg,prognosisanddeathtelling) b.conflictresolution(eg,betweenfamily members

4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking

1.3.32Nausea/vomiting 1.3.42Shortnessofbreath

*unclearwhethertheseEMmodelelementsreferto treatingthesesymptomsordevelopingadifferential diagnosisforthesesymptoms,butbothshould betaught.

20.1.2.2Interpersonalandcommunication skills-conflictmanagement

20.1.2.4Interpersonalandcommunicationskillsdeliveringbadnews/deathnotifications

20.4.4.1Healthcarecoordination-advancedirectives

5Caregiversupport20.3.4.6Well-beingandresilience-careforthecaregiver

6Non-initiationorstoppingofnonbeneficial interventions

19.2Resuscitation-cardiopulmonaryresuscitation

20.1.1.3Interpersonalskills-patientandfamily experienceofcare

20.4.4.2.2Healthcarecoordination-withdrawalofsupport

9Bereavementandgrieving14.2.4Mooddisordersandthought disorders-griefreaction

10Family-witnessedresuscitation19.2Resuscitation-cardiopulmonaryresuscitation

12Copingandself-care20.3.4.1Well-beingandresilience-fatigue andimpairment

20.3.4.1.1Well-beingandresilience-sleephygiene

20.3.4.3Well-beingandresilience-work/lifebalance

13End-of-lifemanagementinthemass casualtyincident/event

16Screeningforpalliativecareneeds:

a.identifyingpatientswhomaybenefit fromHPMspecialistreferral,b.identifying theimminentlydyingpatient(expected deathwithinhours-days).

17Rapidpalliativecareassessment:

a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.

20.4.2.2.1Patienttriageandclassification

20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare

20.4.4.2.3Healthcarecoordination-hospicereferral

20.3.4.6Well-beingandresilience-careforthecaregiver 20.4.4.2.3Healthcarecoordination-hospicereferral

20.4.4.3.1Healthcarecoordination-activitiesofdaily living/functionalassessment

Table1. ThehospiceandpalliativeemergencymedicineresidencyeducationcompetenciesmappedontotheAmericanBoardof EmergencyMedicineEMModel.

Table1. Continued.

Hospiceand palliative competencyDescriptionEMmodelitem

18Complicationsofcancer:a.disease complications(eg,spinalcord compression,hypercalcemia), b.treatmentcomplications (eg,pancreatitis,tumorlysis,neutropenia, acuterenalfailure).

2.9.2.3Largebowel-radiationcolitis

2.9.2.5Largebowel-neutropenicenterocolitis/typhlitis

3.6.1Diseasesofthepericardium-pericardial tamponade

8.7Oncologicemergencies

8.7.1Oncologicemergencies-febrileneutropenia

8.7.2Oncologicemergencies-hypercalcemia ofmalignancy

8.7.3Oncologicemergencies-hyperviscositysyndrome

8.7.4Oncologicemergencies-malignant pericardialeffusion

8.7.5Oncologicemergencies-spinalcordcompression

8.7.6Oncologicemergencies-superiorvena cavasyndrome

8.7.7Oncologicemergencies-tumorhemorrhage

8.7.8Oncologicemergencies-tumorlysissyndrome

11.1.4.2Bonyabnormalities-tumor-relatedfractures

16.2.3Disordersofthepleura,mediastinum,andchest wall-pleuraleffusion

16.6.2Pulmonaryembolism/infarct-venous thromboembolism

16.6.2.1Pulmonaryembolism/infarct-massiveand submassiveembolism

19Ethical,spiritual,andculturalissues aroundend-of-lifeanddeath

20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.

21ethicalandlegalissues:a.decisionmakingcapacity,b.futility.

22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.

23Transitionsacrosscaresettings,eg, inpatientvshomehospice,palliative careunit

associationwiththeEMModelwere HPM14:Trajectories ofdying:a.Terminalillness,b.OrganFailure,c.Frailty,d. SuddenDeath, and HPM15:Prognostication. Whilethese competenciesnecessitatehavingsoundunderstandingofthe naturalhistoryofdiseaseaswellasphysicalexaminationand

20.1.2.5Interpersonalandcommunication skills-culturalcompetency

20.4.4.1Healthcarecoordination-advancedirectives

20.3.2.4Professionalism-medicalethics

20.4.5.4Regulatory/legal-consent,capacityandrefusal ofcare-consent,capacityandrefusalofcare

20.1.1.1Interpersonalskills-inter-departmentaland medicalstaffrelations

20.1.1.2Interpersonalskills-intra-departmentalrelations, teamwork,andcollaborationskills

20.4.2.4.1EDadministration-alliedhealthprofessionals

20.4.4.2.1Healthcarecoordination-patientidentification forpalliativecare

20.4.4.2.3Healthcarecoordination-hospicereferral

clinicalworkupcomponentsinformingprognosis,theseare alsoskillsforexplainingthelikelihoodofdeathand communicatingwithpatientsandfamilies.Theteam consensuswasthatthesecouldbetaughtwithintheEM Modelitems 20.1.2.4InterpersonalandCommunication

Table2. Thepalliativeemergencymedicinecompetenciesincorporatewiththe2021AmericanBoardofEmergencyMedicineknowledge, skills,andabilities.

Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel

3Difficultcommunicationa.deliveryofbad news(eg,prognosisanddeathtelling)

b.conflictresolution (eg,betweenfamilymembers)

4Goalsofcarediscussions:a.assisting familieswithdecisionmaking.b.assisting patientswithdecisionmaking.

CS17Use flexiblecommunicationstrategiesto negotiateeffectivelywithstaff,consultants, patients,families,andotherstoprovideoptimal patientcare,recognizingandresolving interpersonalconflicts

CS3Elicitpatients’ reasonsforseekinghealthcareand theirexpectationsfromtheEDvisit

CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication

CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

B

13End-of-lifemanagementinthemass casualtyincident/event

14Trajectoriesofdying:a.terminalillness, b.organfailure,c.frailty, d.suddendeath.

D

B

C

B

DM11Participateinamasscasualtydrilloreventinan EDinvolvingmultiplepatients,prioritizingcare, containingpotentialexposures,andappropriately assigningresources C

17Rapidpalliativecareassessment: a.aligningdiagnosticsandtherapeuticsto patientgoals,b.functional,psychosocial, andspiritualassessment,c.assessingfor andinitiatinghospicereferrals,d.toolkits tohelpidentifypatientneedsfor appropriatereferrals/resources, e.caregiverburden.

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate

CS7Considertheexpectationsofthosewhoprovide orreceivecareintheEDandusecommunication methodsthatminimizethepotentialforstress, conflict,andmiscommunication

Table2. Continued.

Hospiceandpalliative medicinecompetencyKSAcodeDescriptionLevel

20Advancedirectives:a.physicianorderfor life-sustainingtreatment(POLST), b.medicalorderforlife-sustaining treatment(MOLST),c. fivewishes.

21Ethicalandlegalissues:a.decisionmakingcapacity,b.futility.

22Multidisciplinaryteamandsupport systems.(understandingteamrolesand systemresources):a.spiritualchair, b.socialchair,c.hospicecareeligibility, d.continuingcare,e.importanceoflocal andcommunitysupportsystems.

23Transitionsacrosscaresettings, eg,inpatientvshomehospice, palliativecareunit

CS6Elicitinformationfrompatients,families,andother healthcaremembersusingverbal,nonverbal, written,andtechnologicalskills

ES15Elicitthepatient’sgoalsofcarepriortoinitiating emergencystabilization,includingevaluatingthe validityofadvanceddirectives

CS15Solicitpatientparticipationinmedicaldecisionmakingbydiscussing,risks,benefits,and alternativestocareprovided

ES6Recognizeinatimelyfashionwhenfurther clinicalinterventionisfutile

D

B

C

B

LI12Balancepatientautonomywithpatientprotection andadvocacywhenaddressingconsentand refusalofcareinaccordancewithlegaland ethicalstandards B TI9Obtaininformedconsentfromthepatientor

TM1OrganizepatientcareteamsB

CS5Communicateinformationtopatientsandfamilies usingverbal,nonverbal,written,and technologicalskills,andconfirmunderstanding B

CS10Communicatepertinentinformationtohealthcare colleaguesineffectiveandsafetransitions ofcare C

TC11Determine,summarize,andcommunicatethe diagnosisordiagnosticuncertainty,anticipated course,prognosis,dispositionplan,medications, futurediagnostic/therapeuticinterventions,signs andsymptomsforwhichtoseekfurthercareand follow-uptopatientorsurrogate B

TC15Ensuretransitionsofcareareaccuratelyand efficientlycommunicatedbetweencliniciansusing bestpractices

Skills:Deliveringbadnews/DeathNoti fi cations and 20.1.1.3InterpersonalandC ommunicationSkills:Patient andfamilyexperienceofcare.

IncorporationintotheKnowledge,SkillsandAbilities

Thirtyitemsof214weretaggedinthe firstroundwith87% consensus(187/214)betweentheinitialthreeindependent reviewersonwhetheranitemwasorwasnottaggedasa match.Tenofthe23HPMcompetencies(43%)mappedonto 16differentKSAs(Table2).Ofthe16matcheswithinthe KSAs,nonewereadvancedskills(levelA).AllbutHPM13

mappedontoatleastonelevelBskill.Atableshowingallthe HPMcompetenciesandtheirincorporationwithinthe EMModelandKSAstogetherisincludedas SupplementalDataA.

PotentialAreasofFitintotheKnowledge,Skills andAbilities

ThreeadditionalKSAswereidentifiedashavingareasof potential fitorincorporation. HPM5:Caregiversupport and HPM12:Copingandself-care couldbetaughtwhile discussing CS2:Establishrapportwithanddemonstrate

empathytowardpatientsandtheirfamilies .Finally, HPM 16:Screeningforpalliativecareneeds couldbetaughtwith TC18:Correctlydeterminetheappropriatedisposition .

DISCUSSION

ThisstudyshowedfairtogoodinclusionofHPM competencieswithinthepublishedEMKSAsandEM Model,demonstratingthattheHPMcompetenciesare representedintheMilestones.However,keytopicareaswere identifiedthatcouldimprovethefocusofEMtrainingin HPM.DemonstratingtheoverlapoftheHPMandEM contentmayhelpEMeducatorsensurethatHPMtrainingis incorporatedintotheircurriculums.Lackoftrainingon thesetopicsisaconsistent findinginnationaland internationalstudies,andeducatorsneedbetterwaysto incorporateHPM-EMtrainingintoresidency curriculums.3–7,11–13 ImprovedteachingoftheHPM-EM competencieshasthepotentialtodecreasethecaregaps seeninEDsymptommanagementandend-of-lifecare, includinglackofgoalsofcareconversationsforcritically illpatients.14,15

AlimitationoftheHPMcompetenciesisthattheyhave notbeenexternallyassessedorinvestigatedandarebasedon expertconsensus.Noneoftheinitialfourreviewerswere involvedinthedevelopmentoftheHPMcompetenciesand theyfoundthemtoalmostallmapontotheEMModelor identifiedplacesintheEMModelthatcouldbeexpandedto includethemmoreexplicitly.Additionally,theHPM competenciesthatmappedontoKSAsallmetatleastone KSAontheminimalcompetencylevel.These findingsimply thattheHPMcompetenciesareskillsthatareat residentlevel.

ThedescriptionsintheHPMcompetenciescanadddepth tothecorrespondingEMMilestonesforcurriculum developmentandsummativeevaluation.Forexample,most residenciesprovidetrainingorsimulationsofmasscasualty care.Thestudygroupenvisionedwaysinwhichend-of-life managementcouldbeaddedintothattraining(HPM13).

Likewise,alectureonpost-cardiacarrestcarecould incorporatetrainingonthenon-initiationorcompassionate discontinuationofinterventionssuchasmechanical ventilation(HPM6).Summativecompetencyassessmentsat endoftrainingtogainboardcertificationcouldalso incorporatemoreHPMcompetency-basedquestions.

MuchoftheoverlapbetweentheHPMcompetenciesand theEMModelandKSAswasin Interpersonaland CommunicationSkills (EMModel)andthe CS –Communication&InterpersonalSkills (KSAs). Communicationskills,althoughchallengingtoteach,are criticalinpatient-centeredcareandwilllikelyhavean increasedemphasisasartificialintelligenceandmachine learningbecomemoreuniversallyintegratedintoclinical care.Currentmodelsforcommunicationinstructionrely heavilyonrolemodeling.16 Residentshavesuggestedthat

formaltrainingincommunicationshouldfocusongeneral communicationskillsandshouldprovidesyntaxtousein futurediscussions.17 Developingcommunicationskills requiresdeliberatepracticeoftechniques,includingNURSE statements(naming,understanding,respecting,supporting, andexploring)andAsk-Tell-Ask.17,18 Additionally, educatorsmustbecomefamiliarwithmethodsfor real-timeteachingofcommunication,suchas “CouldI addsomething?”19

Trajectoriesofdying(HPM7)andprognostication (HPM8)aretwoskillsusedtocounselpatients/familieswith seriousillnessorattheendoflifethatdidnot fitclearlywithin theEMModel.Thesearedifficultskills,andpriorstudies haveidentifiedsomediscordancebetweenwhatfamilies/ caregiversunderstandaboutaperson’sdeathandthe underlyingcausesofdeathidentifiedbythephysician-led team.20 Thus,thisskillshouldbehonedthroughouttraining. ItisourexperiencethatEMresidentsrarelyreceiveexplicit educationonprognostication,andsowerecommendits incorporationintocurriculums.Ourresultsfurthersuggest thattrainingontreatingend-of-lifesymptoms,careforthe imminentlydying,andcaringforpatientsunderhospicecare couldbeoverlookedbycurrentresidentcurriculumswith strictadherencetotheEMModel.

LIMITATIONS

Alimitationofthisprojectisthateventhoughaconsensus processwasusedwithexpertsinresidencyeducationand HPM,othereducationexpertsmayinterpretthedomains andcompetenciesdifferently.Forexample,the EMModel item20.3.4.6Well-beingandResilience-Careforthe caregiver wasmatchedtoHPM5and17aboutpatient caregivers.However,thiscouldalsobeinterpretedas residentself-careasitisunderthewell-beingsection.Finally, whiletrainedHPMemergencyphysiciansreviewedallthe mapping,theinitialmappingdidincluderesidentinput.This couldbeconsideredanadvantage,astheyareexperiencing lecturesweekly,orareapotentialsourceofbias,astheyhave nothadafullEMcurriculumyet.

CONCLUSION

WeidentifiedareasofoverlapwheretheHPM-EM subspecialtycompetenciescanbeemphasizedorintegrated intoEMModel-basedresidencycurriculums.This knowledgecanbeusedforcurriculumplanningand incorporatingHPMintodefinitionsforcompetencyinEM. Thesecouldalsobereflectedin finalsummativeevaluations forcertification.

AddressforCorrespondence:LaurenT.Southerland,MD,TheOhio StateUniversityWexnerMedicalCenter,DepartmentofEmergency Medicine,725PriorHall,376W10thAve.,Columbus,OH43210. Email: Lauren.Southerland@osumc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy.

LaurenT.SoutherlandhasNIHgrantfundingnotpertainingtothis study.DianeL.GorgasisaboardmemberontheAmericanBoardof EmergencyMedicine.LaurenT.SoutherlandandDanielW. Markwalterhavecontributedtosomeofthefreeeducational websitesmentionedinthediscussion.Therearenootherconflictsof interestorsourcesoffundingtodeclare.

Copyright:©2024Goettetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.ElmerJ,MikatiN,ArnoldRM,etal.Deathandend-of-lifecarein emergencydepartmentsintheUS. JAMANetwOpen. 2022;5(11):e2240399.

2.GeorgeN,BowmanJ,AaronsonE,etal.Past,present,andfutureof palliativecareinemergencymedicineintheUSA. AcuteMedSurg. 2020;7(1):e497.

3.WoodsEJ,GinsburgAD,BellolioF,etal.Palliativecareinthe emergencydepartment:asurveyassessmentofpatientandprovider perspectives. PalliatMed. 2020;34(9):1279–85.

4.MeoN,HwangU,MorrisonRS.Residentperceptionsofpalliative caretrainingintheemergencydepartment. JPalliatMed. 2011;14(5):548–55.

5.KrausCK,GreenbergMR,RayDE,etal.Palliativecareeducationin emergencymedicineresidencytraining:asurveyofprogramdirectors, associateprogramdirectors,andassistantprogramdirectors. JPain SymptomManage. 2016;51(5):898–906.

6.BaylisJ,HarrisDR,ChenC,etal.Palliativeandend-of-lifecare educationinCanadianemergencymedicineresidencyprograms: anationalcross-sectionalsurvey. CJEM. 2019;21(2):219–25.

7.AdeyemiOJ,SimanN,GoldfeldKS,etal.Emergencyproviders’ knowledgeandattitudestowardhospiceandpalliativecare:acrosssectionalanalysisacross35emergencydepartmentsintheUnited States. JPalliatMed. 2023;26(9):1252–60.

8.ShoenbergerJ,LambaS,GoettR,etal.Developmentofhospiceand palliativemedicineknowledgeandskillsforemergencymedicine

residents:usingtheAccreditationCouncilforGraduateMedical EducationMilestoneframework. AEMEducTrain. 2018;2(2):130–45.

9.AmericanBoardofEmergencyMedicine.The2019ModeloftheClinical PracticeofEmergencyMedicine.Availableat: https://www.abem.org/ public/resources/em-model.AccessedNovember3,2020.

10.AmericanBoardofEmergencyMedicine.2021Knowledge,Skills,& Abilities.Availableat: https://www.abem.org/public/resources/ emergency-medicine-milestones-ksas.AccessedNovember17,2022.

11.SandersS,CheungWJ,BakewellF,etal.Howemergencymedicine residentshaveconversationsaboutlife-sustainingtreatmentsincritical illness:aqualitativestudyusinginductivethematicanalysis. AnnEmerg Med. 2023;82(5):583–93.

12.ZengH,EugeneP,SupinoM.Wouldyoubesurprisedifthispatient diedinthenext12months?Usingthesurprisequestiontoincrease palliativecareconsultsfromtheemergencydepartment. JPalliatCare. 2020;35(4):221–5.

13.BeneschTD,MooreJE,BreyreAM,etal.Primarypalliativecare educationinemergencymedicineresidency:amixed-methodsanalysis ofayearlong,multimodalintervention. AEMEducTrain. 2022;6(6):e10823.

14.WalkerLE,StanichJA,BellolioF.Aqualitativeassessmentofaguide forgoalsofcareconversationsintheED. AmJEmergMed. 2023:75:185–7.

15.YilmazS,GrudzenCR,DurhamDD,etal.Palliativecareneedsand clinicaloutcomesofpatientswithadvancedcancerintheemergency department. JPalliatMed. 2022;25(7):1115–21.

16.OldeBekkinkM,FarrellSE,TakayesuJK.Interprofessional communicationintheemergencydepartment:residents’ perceptions andimplicationsformedicaleducation. IntJMedEduc. 2018;9:262–70.

17.RisingKL,PapanagnouD,McCarthyD,etal.Emergencymedicine residentperceptionsabouttheneedforincreasedtrainingin communicatingdiagnosticuncertainty. Cureus. 2018;10(1):e2088.

18.TalkVital.QuickGuideResources.Availableat: www.vitaltalk.org/ resources/.AccessedAugust14,2023.

19.BackAL,ArnoldRM,TulskyJA,etal. “CouldIaddsomething?”: teachingcommunicationbyinterveninginrealtimeduringaclinical encounter. AcadMed. 2010;85(6):1048–51.

20.MoonF,KissaneDW,McDermottF.Discordancebetweenthe perceptionsofcliniciansandfamiliesaboutend-of-lifetrajectories inhospitalizeddementiapatients. PalliatSupportCare. 2021;19(3):304–11.

EDUCATION SPECIAL ISSUE:BRIEF EDUCATIONAL ADVANCES

TheEffectofaSimulation-basedInterventiononEmergency MedicineResidentManagementofEarlyPregnancyLoss

ShawnaD.Bellew,MD,MPH*

EricaLowing,MD†

LeahHolcomb,MPH,MS‡

*PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofEmergencyMedicine,Greenville,SouthCarolina

† PrismaHealth-Upstate,UniversityofSouthCarolinaSchoolofMedicineGreenville, DepartmentofObstetricsandGynecology,Greenville,SouthCarolina

‡ ClemsonUniversity,DepartmentofPublicHealthSciences,Clemson,SouthCarolina

SectionEditor:AbraFant,MD

Submissionhistory:SubmittedNovember22,2023;RevisionreceivedJanuary3,2024;AcceptedJanuary17,2024

ElectronicallypublishedMarch25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18596

Background: Theevaluationofpatientswith first-trimestervaginalbleedingandconcernforearly pregnancyloss(EPL)frequentlyoccursintheemergencydepartment(ED),accountingforapproximately 1.6%ofallEDvisits.1 Unfortunately,thesepatientsconsistentlyreportnegativeexperienceswithED care.2–8 Inadditiontoenvironmentalconcerns,suchaslongwaittimes,patientsoftendescribenegative interactionswithstaff,includingaperceivedlackofempathy,theuseofinsensitivelanguage,and inadequatecounseling.2,3 ThesepatientsandtheirpartnersoftenviewEPLasatraumaticlossoflifeand commonlyexperienceprolongedgriefreactions,includinganxietyanddepression.9–11 Poorsatisfaction withcarehasbeenassociatedwithworsementalhealthoutcomes.12 Thesecomplaintsrepresentan importantopportunityforimprovementinemergencymedicine(EM)training.13

WhilenopublishedliteraturetodatedescribestheperformanceofEMresidentsinmanaging patientspresentingwithEPL,studiessuggestthatevenobstetricsandgynecology(OB/GYN)residents findtheseinteractionschallenging.14,15 Simulation-anddidactic-basedtraininghasbeenshowntobe beneficialinimprovingOB/GYNresidentEPLcounselingandhasbeenassociatedwithimprovedpatient outcomes.16 Toourknowledge,thishasyettobereplicatedinEMresidencytraining.

Objectives: Weaimedtodevelopandevaluateasimulation-basededucationalinterventiontoimprove EMresidentmanagementofpatientspresentingwithEPL.[WestJEmergMed.2024;25(2)221–225.]

CURRICULARDESIGN

Theeducationalinterventionconsistedofthreephases (Figure1)andwasdesignedtooptimizelearningbasedon Kolb’slearningcycle.17,18 Residentswerepresentedwitha challengingscenario(concreteexperience)andthen promptedtoreflectonareasforimprovement(reflective observation).Theythencompletedanasynchronousmodule followedbyaninteractivegroupdiscussion(abstract conceptualization).Thelearningcyclecontinuedthrough activeexperimentationviaarepeatedopportunitytodothe simulation,followedbydebriefing.Thisformofrepetitive simulationhasbeenshowntobemoreeffectivewhen comparedwithnon-repeatedsimulation.19,20

WeimplementedtheinterventioninMay2023and conductedapre/poststudyofitsimmediateimpact,which

wasdeemedexemptbyourinstitutionalreviewboard.The interventiontookplaceatthesimulationcenterofthe affiliatedmedicalschool,duringthetwo-hourperiod typicallyallottedformonthlyresidentsimulation-based education.Postgraduateyear(PGY)1–3EMresidentswere recruitedbasedonaconveniencesampleincludingall residentsattendingsimulationthatday.Theresidents werenotinformedofthetopicoftheintervention priortothedayofthestudy,whichistypicalofour simulationcurriculum.

Sixstandardizedpatients(SP)werehiredtoportray patientsexperiencingEPL.Sixvolunteerfacultyemergency physicians(twomen,fourwomen)observedandevaluated thesimulationsandprovidedinstructionanddebriefing.One facultyOB/GYNphysicianandonefacultyemergency

physician(bothwomen)co-facilitatedtheguidedgroup discussion-basededucation.

First,residentsparticipatedina10-minutesimulated patientencounterinwhichtheywereinstructedtocarefora SPwhoportrayedapatientwhowaseightweekspregnant andpresentedwithvaginalbleeding.Priortoevaluatingthe patient,eachresidentwasprovidedwithultrasoundresults indicatingthepregnancywasnonviable(presumably obtainedintriage).

Followingtheencounter,residentsindividuallydebriefed withanEMfacultyobserver.Residentsthenhad30minutes tocompleteanasynchronousonlineeducationalmodulethat includedcontentabouttheassessmentofearlypregnancy bleeding;diagnosingandmanagingectopicpregnancy; preventingalloimmunization;andEPLcounseling.

Particularattentionwaspaidtooptimizingcaretoaddressa patient’sphysical,emotional,andcognitiveneeds,a frameworkrecommendedbyEmondetal.21 Themodulewas deliveredviaaninteractiveeducationalplatform,Rise360 Articulate(Articulate,NewYork,NY).22 Aftercompleting themodule,residentsparticipatedina15-minuteguided groupdialogwithEMandOB/GYNfaculty,discussingbest practicesandmodelingpracticalcommunicationskills. Facilitatorsgaveexamplesofhowtheywouldaddress patientsinvariousscenariostocommunicateclearlywhile alsousingsensitivelanguage.

Followingthisdiscussion,residentsrepeatedthesame 10-minutesimulatedpatientencounterfollowedby individualdebriefingwithEMfaculty.Theinterventionwas designedtoaccommodateupto24residentswiththe resourcesdescribed.

IMPACTANDEFFECTIVENESS

Tostudytheimmediateimpactoftheintervention, residentperformancewasevaluatedusingfourmeasures: 1)completionofcriticalactionsduringthesimulationviaan 11-itemchecklist;2)self-reportedconfidence;3)a10-item multiple-choicetestoffoundationalEPLknowledge;and4) SPperceptionsofresidentempathyduringthesimulationvia

themodifiedJeffersonScaleofEmpathy(JSE).23,24 Allfour evaluativemeasuresweredeliveredimmediatelyfollowing theinitialsimulatedencounter(Phase1)andafterthe final simulationencounter(Phase3).Inadditiontothese measures,residentswereinvitedtoparticipateinabrieffocus groupinterview,conductedbyanon-facultyfacilitator (woman),aftertheinterventiontodiscusstheirimpressions oftheintervention.

FacultyinEMandOB/GYNdevelopedthetaskchecklist toincludecriticalactionsandevidence-basedbestpractices intreatingpatientsexperiencingEPL.Thislistwasadapted fromachecklistemployedinasimilarstudyandmodifiedto reflectEDcare.25 Residentswereaskedtoratetheir perceivedconfidencelevelfromleast(1)tomost(10) confidentregardingthefollowing:knowledgeaboutthe evaluationandmanagementofpatientswith first-trimester bleeding;abilitytocommunicateinasensitiveandempathic mannerwithpatientswithEPL;andabilitytocounsela patientexperiencingEPLregardingwhattoexpectafter discharge.Theyalsocompleteda10-questionmultiplechoicetest,whichEMandOB/GYNfacultydevelopedto assessbasicobjectiveknowledge.Aftereachsimulated encounter,SPscompletedthemodifiedJSE,avalidatedtool forSPevaluationofclinicianempathyandcommunication. ThemodifiedJSEincludes fivequestionsonaseven-point Likertscalerangingfromstronglydisagree(1)tostrongly agree(7).23,24 Anoutlineofthesimulatedcase,themodule, andtheassessmenttoolsareincludedinthe supplemental material accompanyingtheonlinearticle.

Ofthe16residentswhoparticipated,75%identifiedas men,andtherewasrelativelyequalrepresentationofPGY-1 (31.3%),PGY-2(37.5%),andPGY-3(31.3%)residents. Residentsimprovedfrompre-topost-interventionacrossall fourevaluativemeasures(Table1).Beforetheintervention, fewresidentsprovidedinformationaboutwhattoexpect afterdischarge,includingthepotentialpainlevel,the likelihoodofpassingtissue,returnprecautions,andlongtermemotionalramifications.Aftertheintervention, residentsweresignificantlymorelikelytousesensitive

Figure1. Sequenceofaneducationalinterventionforearlypregnancylosscounseling. EPL,earlypregnancyloss.

Bellewetal. Simulation-basedInterventiononEMResidentManagementofEPL

5.Counselspatientabouttheamountofexpectedbleeding2(12.5)11(68.8)0.004

6.Counselspatientonexpectedpain1(6.3)10(62.5)0.004

7.Counselspatientonthepossibilityofpassingtissue2(12.5)12(75.0)0.006

8.Counselspatientonreturnforseverebleeding3(18.8)14(87.5)0.003

9.Counselspatientonreturnforfever2(12.5)15(93.8)0.001

10.NormalizesemotionalramificationsofEPL5(31.3)13(81.3)0.008

11.Discussesfollow-upplan10(62.5)16(100)0.030 EPL,earlypregnancyloss.

languageandtoincludeinformationaboutexpected outcomesandreturnprecautions(Table1b).

Theseresultsindicatethatfocusedtrainingresultedin immediateimprovementsinresidentperformance, particularlyregardingcounselingandcommunication. Giventhepositiveresultsofsimilarinterventionsundertaken inotherlearnerpopulations,thisimmediateimpactlikely indicatesimprovedabilitytocareforpatientsinclinical practice.Verhaegheetalpublishedtheimpactofathreehourin-situsimulationtrainingforOB/GYNresidents, whichresultedinlong-termimprovementsinpsychologic outcomesaswellasreducedneedforreturnvisits.16 As comparedtothesepreviousinterventions,ourcurriculum enhancedefficiencybyemployinganonlinetrainingmodule, whichcoveredadditionalfoundationalknowledgeofearly pregnancybleedingcare(includingectopicpregnancyand threatenedEPL).Thisefficiencyisparticularlyimportantin EMgiventhebreadthofrequiredknowledge.

Whiletheeightresidentswhoparticipatedinthefocus groupinterviewgenerallyreportedpositivefeedback,two residentsdidnotethattheywereconfusedbytheorderofthe simulationsuchthattheyhadadiagnosispriortoany interactionwiththepatient.Inthefuture,thismaybe

amelioratedbyprovidingtheresidentswithmorecontextto thecaseorsimplyrevisingthescenariosothattheultrasound reportisreceivedafteraninitialevaluationandrequestfor imaging.Additionally,thetimeallottedforthe asynchronousmodulewas30minutes,butmostresidents completeditinabout20minutes,indicatingthepossibility ofadditionalcontentorexpansionofanotheraspectof theintervention.

LIMITATIONSANDCONCLUSION

Thisstudydescribesresidentperformanceinasimulated patientencounter,andwecannotconcludethatthisreflects actualclinicalcare.Thisstudyonlyassessedtheimpactofthe trainingonlearning(Kirkpatricklevel2)anddidnotattempt toevaluatetheresidents’ ongoingclinicalbehaviororits effectonpatients.26 Thestudywasconductedduringone sessionand,therefore,wecannotinferinformationabout retentionoflearning.Futureworkshouldassesstheeffectof interventionssuchasthisonclinicianbehaviorandresultant patientoutcomes.Facultyevaluatorswerenotblinded duringthesimulatedpatientencounters,whichcouldhave introducedbiasintotheevaluationprovidedviathe checklist.Thisconcernissomewhataddressedbythebinary

Table1. Residentassessmentoutcomespre-topost-intervention.

Table1b. Residentchecklistperformance.

natureofthechecklist,inwhicheitherataskwasperformed oritwasnot.Oftheassessmenttools,onlythemodifiedJSE hasbeenexternallyvalidated.CreatingandvalidatingEMspecificmeasurementtoolsforEPLcarewouldensuremore robustdatagoingforward.

“Participantsdisproportionatelyidentifiedasmen(75%), ascomparedtothenationalaverageinemergencyresidencies of62%.27 Giventhesmallpopulationfromwhichthestudy samplewasderived,wedidnotaskparticipantswhetherthey werecis-ortransgendertoavoidlossofanonymity. Similarly,wedidnotaskparticipantsaboutpersonal experienceswithEPL.Futureworkcouldexplorethe relationshipofthesecharacteristicsandexperienceswith clinicalperformance.Despitetheselimitations,theresultsof thisstudyindicateaneedforEPL-specificeducationinEM residencyandthatabrief,simulation-basedinterventionwas effectiveinproducingimmediateimprovements.

Consideringtheresultsofsimilarstudiesconductedinother populations,aninterventionsuchasthismayresultin improvedclinicalcareandlong-termpatientoutcomesinthis common,butdevastating,presentation.

5.HoAL,HernandezA,RobbJM,etal.Spontaneousmiscarriage managementexperience:asystematicreview. Cureus. 2022;14(4):e24269.

6.MeluchAL.Waitingtobeseen:provider-patientcommunicationinthe emergencyroomaboutmiscarriage. HealthCommun. 2021;37(11):1452–4.

7.MillerCA,RoeAH,McAllisterA,etal.Patientexperienceswith miscarriagemanagementintheemergencyandambulatorysettings. ObstetGynecol. 2019;134(6):1285–92.

8.PunchesBE,JohnsonKD,AcquavitaSP,etal.Patientperspectivesof pregnancylossintheemergencydepartment. IntEmergNurs. 2019;43:61–6.

9.PrettymanRJ,CordleCJ,CookGD.Athree-monthfollow-upof psychologicalmorbidityafterearlymiscarriage. BrJMedPsychol. 1993;66(Pt4):363–72.

10.ZaccardiR,AbbottJ,Koziol-McLainJ.Lossandgriefreactionsafter spontaneousmiscarriageintheemergencydepartment. AnnEmerg Med. 1993;22(5):799–804.

11.BellhouseC,Temple-SmithM,WatsonS,etal. “Thelosswas traumatic somehealthcareprovidersaddedtothat”:Women’s experiencesofmiscarriage. WomenBirth. 2019;32(2):137–46.

AddressforCorrespondence:ShawnaD.Bellew,MD,MPH,Prisma Health-Upstate,UniversityofSouthCarolinaSchoolofMedicine Greenville,DepartmentofEmergencyMedicine,701GroveRoad, Greenville,SC29605.Email: Shawna.bellew@prismahealth.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Bellewetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.WittelsKA,PelletierAJ,BrownDF,etal.UnitedStatesemergency departmentvisitsforvaginalbleedingduringearlypregnancy, 1993–2003. AmJObstetGynecol. 2008;198(5):523.e1–6.

2.Larivière-BastienD,deMontignyF,VerdonC.Women’sexperiencesof miscarriageintheemergencydepartment. JEmergNurs.

2019;45(6):670–6.

3.MacWilliamsK,HughesJ,AstonM,etal.Understandingtheexperience ofmiscarriageintheemergencydepartment. JEmergNurs. 2016;42(6):504–12.

4.DaintyKN,SeatonMB,McLeodS,etal.Reframinghowearlypregnancy lossisviewedintheemergencydepartment. QualHealthRes. 2021;31(6):1119–28.

12.deMontignyF,VerdonC,MeunierS,etal.Women’spersistent depressiveandperinatalgriefsymptomsfollowingamiscarriage:the roleofchildlessnessandsatisfactionwithhealthcareservices. Arch WomensMentHealth. 2017;20(5):655–62.

13.EvansCS.Earlypregnancylossintheemergencydepartment:lessons learnedasaspouse,newfather,andemergencymedicineresident. Ann EmergMed. 2021;77(2):233–6.

14.BrannMandButeJJ.Communicatingtopromoteinformeddecisionsin thecontextofearlypregnancyloss. PatientEducCouns. 2017;100(12):2269–74.

15.ButeJJandBrannM.Tensionsandcontradictionsininterns’ communicationaboutunexpectedpregnancyloss. HealthCommun. 2020;35(5):529–37.

16.VerhaegheC,GicquelM,BouetPE,etal.Positiveimpactofsimulation trainingofresidentsonthepatients’ psychologicalexperience followingpregnancyloss. JGynecolObstetHumReprod. 2020;49(3):101650.

17.KolbDA. ExperientialLearning:ExperienceastheSourceofLearning andDevelopment.2nd ed.Hoboken,NJ:PearsonEducation,2015.

18.StockerM,BurmesterM,AllenM.Optimisationofsimulatedteam trainingthroughtheapplicationoflearningtheories:adebatefora conceptualframework. BMCMedEduc. 2014;14:69.

19.AuerbachM,KesslerD,FoltinJC.Repetitivepediatricsimulation resuscitationtraining. PediatrEmergCare. 2011;27(1):29–31.

20.NgC,PrimianiN,Orchanian-CheffA.Rapidcycledeliberatepracticein healthcaresimulation:ascopingreview. MedSciEduc. 2021;31(6):2105–20.

21.EmondT,deMontignyF,GuillaumieL.Exploringtheneedsof parentswhoexperiencemiscarriageintheemergencydepartment: Aqualitativestudywithparentsandnurses. JClinNurs. 2019;28(9-10):1952–65.

Bellewetal. Simulation-basedInterventiononEMResidentManagementofEPL

22.FirsttrimesterbleedingandEarlyPregnancyLoss.Availableat: https:// rise.articulate.com/share/CfDnfG-CI6UEdqU1t-FEUDtPnu_UU5mQ AccessedMay9,2023.

23.HojatM. EmpathyinHealthProfessionsEducationandPatientCare 1st ed.NewYorkCity,NY:SpringerInternationalPublishing,2016.

24.MalloryL,FloyedR,DoughtyC,etal.ValidationofamodifiedJefferson ScaleofEmpathyforobserverstoassesstrainees. AcadPediatr. 2021;21(1):165–9.

25.MarkoEK,Buery-JoynerSD,SheridanMJ,etal.Structuredteaching ofearlypregnancylosscounseling. ObstetGynecol. 2015;126(Suppl4):1s–6s.

26.JohnstonS,CoyerFM,NashR.Kirkpatrick’sevaluationofsimulation anddebriefinginhealthcareeducation:asystematicreview. JNursEduc. 2018;57(7):393–8.

27.DeFazioCR,CloudSD,VerniCM,etal.Womeninemergencymedicine residencyprograms:ananalysisofdatafromAccreditationCouncilfor GraduateMedicalEducation-approvedresidencyprograms. AEMEduc Train. 2017;1(3):175–8.

28.LeeL,MaW,DaviesS,etal.Towardoptimalemotionalcareduringthe experienceofmiscarriage:anintegrativereviewoftheperspectivesof women,partners,andhealthcareproviders. JMidwiferyWomens Health. 2023;68(1):52–61.

RootCauseAnalysisofDelayedEmergencyDepartment ComputedTomographyScans

ArjunDhanik,MD*†∘

BryanA.Stenson,MD*†∘

RobinB.Levenson,MD*†

PeterS.Antkowiak,MD,MPH*†

LeonD.Sanchez,MD,MPH†‡

DavidT.Chiu,MD,MPH*†

SectionEditor:MarkLangdorf,MD,MPHE

*BethIsraelDeaconessMedicalCenter,DepartmentofEmergencyMedicine, Boston,Massachusetts

† HarvardMedicalSchool,HarvardUniversity,Boston,Massachusetts

‡ BrighamandWomen’sFaulknerHospital,DepartmentofEmergencyMedicine, Boston,Massachusetts

∘ Co- firstauthorswhoequallycontributedtothiswork

Submissionhistory:SubmittedJanuary23,2023;RevisionreceivedOctober25,2023;AcceptedDecember5,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17831

Introduction: Asolutionforemergencydepartment(ED)congestionremainselusive.Asrelianceon imaginggrows,computedtomography(CT)turnaroundtimehasbeenidentifiedasamajorbottleneck.In thisstudywesoughttoidentifyfactorsassociatedwithsignificantlydelayedCTintheED.

Methods: WeperformedaretrospectiveanalysisofallCTimagingcompletedatanurban,tertiarycare EDfromMay1–July31,2021.Duringthatperiod,5,685CTswereperformedon4,344patients,witha mediantimefromCTordertocompletionof108minutes(Quartile1[Q1]:57minutes,Quartile3[Q3]:182 minutes,interquartilerange[IQR]:125minutes).Outliersweredefinedasstudiesthattooklongerthan 369minutestocomplete(Q3 + 1.5 × IQR).Wesystematicallyreviewedoutlierchartstodetermine factorsassociatedwithdelayandidentified fivefactors:behaviorallynon-compliantormedically unstablepatients;intravenous(IV)lineissues;contrastallergies;glomerular filtrationrate(GFR) concerns;anddelaysrelatedtoimagingprotocol(eg,needforIVcontrast,requestfororaland/orrectal contrast).Wecalculatedconfidenceintervals(CI)usingthemodifiedWaldmethod.Inter-raterreliability wasassessedwithakappaanalysis.

Results: Weidentifiedatotalof182outliers(4.2%oftotalpatients).Fifteen(8.2%)caseswereexcluded forCTtime-stampinconsistencies.Ofthe167outliersanalyzed,38delays(22.8%,95%confidence interval[CI]17.0–29.7)wereduetobehaviorallynon-compliantormedicallyunstablepatients;30 (18.0%,95%CI12.8–24.5)wereduetoIVissues;24(14.4%,95%CI9.8–20.6)wereduetocontrast allergies;21(12.6%,95%CI8.3–18.5)wereduetoGFRconcerns;and20(12.0%,95%CI7.8–17.9) wererelatedtoimagingstudyprotocols.Thecauseofthedelaywasunknownin55cases(32.9%,95% CI26.3–40.4).

Conclusion: Ourreviewidentifiedbothmodifiableandnon-modifiablefactorsassociatedwith significantlydelayedCTintheED.Patientfactorssuchasbehavior,allergies,andmedicalacuitycannot becontrolled.However,institutionalpoliciesregardingdifficultIVaccess,contrastadministrationinlow GFRsettings,andstudyprotocolsmaybemodified,capturingupto42.6%ofoutliers.[WestJEmerg Med.2024;25(2)226–229.]

INTRODUCTION

Asolutionforemergencydepartment(ED)congestion remainselusive.Asrelianceonimaginggrows,computed

tomography(CT)turnaroundtime fromCTorderto completion hasbeenidentifiedasamajorbottleneck.1,2 Onestudyshowedthatpatientswhohadradiological

diagnosticswere4.4timesmorelikelytostayoverfourhours intheEDthanthosewhodidnothavethesetests.2

NumerousstudieshaveidentifiedstrategiestodecreaseCT turnaroundtimes.Whiteetalmappedthecomplexprocessof EDradiologytransportandappliedsystemsengineering principlestoimproveefficiencywithoutincreasingresource use. 3 Perotteetalassembledamultidisciplinarystakeholder teamtoidentifybarriersandimplementsolutionstoreduce CTturnaroundtimefrom5.8to4.6hoursdespitea 13.8%increaseinthenumberofscans.1 Variousstudieshave demonstratedtheefficacyofapplyingLeanand SixSigmaprinciples.4,5 Finally,queuingtheoryhasbeen usedtomodelEDdelayswithvaryinglevelsof resourceutilization.6,7

Therehasnotyetbeenadedicatedoutlieranalysisof delayedCTscansintheED.Inthisstudywesoughtto identifyfactorsassociatedwithsignificantlydelayedCT. ThisisconsequentialgiventhatpatientswithEDstayslonger thansixhoursdirectlycontributetocrowding.8

METHODS

WeperformedaretrospectiveanalysisofallCTs completedatanurban,tertiarycareEDinBoston, Massachusetts,fromMay1–July31,2021.Duringthat period,5,685scanswereperformedon4,344patients,witha mediantimefromCTordertocompletionof108minutes (Quartile1[Q1]:57minutes,Quartile3[Q3]:182minutes, interquartilerange[IQR]:125minutes).Outlierswere definedasstudiesthattooklongerthan369minutesto complete(Q3 + 1.5 × IQR).WedefinedCTcompletion timeasthepointatwhichtheCTtechnologistmarksthe studyascompleted,therebyremovingtheconfounderof radiologistreadtime.

Wesystematicallyreviewedoutlierchartsand communicationsbetweenmembersofthecareteamto determinefactorsassociatedwithdelayandidentified five factors:behaviorallynon-compliantormedicallyunstable patients;intravenous(IV)lineissues(eg,IVinfiltration, difficultIVaccess,inadequateIVsize);contrastallergies; glomerular filtrationrate(GFR)concerns;anddelaysrelated toimagingprotocol(eg,need/requestforcontrast administration,includingIV,oral,and/orrectal).

Confidenceintervals(CI)werecalculatedusingthemodified Waldmethod.Weperformedakappaanalysistoassessfor inter-raterreliability.Thiswasdoneoneachcategory individuallyassomeoutliercaseshadmultiplecontributing factors.Thisstudydesignwasapprovedbyourinstitutional reviewboardwithadeterminationofexemption.We observed10ofthe12methodsofhealthrecordreviewas outlinedbyWorsteretal,withtheexceptionsofabstractor performancemonitoringandabstractorblinding tohypothesis.9

RESULTS

Weidentified182outliers(4.2%oftotalpatients)and excluded15cases(8.2%)forCTtime-stampinconsistencies. Ofthe167outliersanalyzed,38delays(22.8%,95%CI 17.0–29.7)wereduetobehaviorallynon-compliantor medicallyunstablepatients;30(18.0%,95%CI12.8–24.5) wereduetoIVissue;,24(14.4%,95%CI9.8–20.6)weredue tocontrastallergies;21(12.6%,95%CI8.3–18.5)weredueto GFRconcerns;and20(12.0%,95%CI7.8–17.9)wererelated toimagingstudyprotocol.Thecauseofthedelaywas unknownin55cases(32.9%,95%CI26.3–40.4).The distributionofCTtypesforoutliercasesisillustrated in Table1

Kappavaluesrangedfrom0.69–0.98forallthecategories (Table2).Intravenousissueshadthelowestdegreeof agreement,whiledelaysduetoallergyprotocolshadthe highestdegreeofagreement.

DISCUSSION

Ourreviewidentifiedbothmodifiableandnon-modifiable factorsassociatedwithsignificantlydelayedCTintheED. Patientfactorssuchasbehavior,allergies,andmedicalacuity cannotbecontrolled.However,institutionalprotocols regardingdifficultIVaccess,contrastadministrationinlow GFRsettings,andstudyprotocolsmaybemodified.Oneof thesemodifiablefactorsisIVaccess:earlyinvolvementofan

Table1. Distributionofoutliersinemergencydepartment computedtomography.

ComputedtomographytypeNumber(%total)

Torso(anychest/abdomen/pelvisimaging)124(62.0%)

Non-contrasthead37(18.4%)

Spine15(7.5%)

Angiogramheadandneck13(6.5%)

Face,orbits,softtissueneck7(3.5%)

Extremity5(2.5%)

Kappaanalysisoffactorsassociatedwithsignificantly delayedcomputedtomography.

Factorsassociatedwith delay

Kappa(95%confidence interval)

Intravenouslineissues0.69(0.55–0.83)

Contrastallergy0.98(0.93–1.00)

Renalfunctionconcerns0.86(0.74–0.98)

Behaviorallyormedically unstablepatient 0.85(0.75–0.94)

Imagingprotocol0.83(0.70–0.96)

Unknown0.86(0.78–0.95)

Table2.

IVteamorutilizationofultrasoundforIVplacementmay expediteimaging.Ourdatasuggeststhat18.0%ofoutliers canbemoreefficientlyimagedbyimprovingIVplacement strategies.Studieshaveshownthatnearly9%ofEDpatients havedifficultIVaccess,definedinonepaperasrequiring ≥3 attemptsoranultrasound-guidedline.Thesepatients experiencestatisticallysignificantdelaysinestablishingIV access,obtaininglabresults,andreceivinganalgesia,aswell asexperiencinglongerEDlengthofstay.10 Therefore,the benefitsofexpeditiousIVplacementextendsbeyond enhancedCTthroughput.

Thesecondmodifiablefactorpertainstocontrast administrationinlowGFRsettings.Thereisgrowing evidencethattheriskofacutekidneyinjuryresultingfrom contrastadministrationinpatientswithreducedGFRmay havebeenoverestimated.11 Thisshifthasbeenattributedto thefactthatmuchoftheexistingliteraturewasnot sufficientlycontrolledtodistinguishbetweencontrastinducedandcontrast-associatednephropathy.11 Institutions mayconsiderrevisingpolicies,suchasforgoingmandatory pre-hydrationorradiologistconversationsandamending exitingGFRcutoffs,toexpediteimaging.

JudiciousprotocolingofCTmayaddressaproportionof outliers.Onestudyfoundthatpatientswhohadan abdominal/pelvicCTwithonlyIVcontrasthadan approximatelytwo-hourshorterEDlengthofstaywhen comparedtopatientswhoreceivedaCTwithoralandIV contrast.12 Thisdifferencewasevenmorepronouncedwhen comparingpatientswhounderwentCTwithoralcontrast withthosewhowereimagedwithnocontrast:patientswho receivednocontrasthadanapproximatelyfour-hourshorter lengthofstay.13 Finally,eliminationoftheroutineuseoforal contrastinabdominal/pelvicCThasbeenshowntoshorten EDlengthofstaywithoutaffectingdiagnosticaccuracy.14 ConsidereduseofcontrastmayimproveCTthroughput.

WemodifiedEDworkflowtoimproveCTthroughput andaddresssomeoftheoutliersidentifiedinthisstudy.We revisedinstitutionalpoliciesregardingcontrast administrationinlowGFRpatientsandstreamlined communicationbetweentheEDandradiologyteams.

Previously,CTinapatientwithaGFRof45–60milliliters perminute(mL/min)triggeredaconversationbetween radiologyandtheEDcareteamregardingoralhydration. Underthenewguidelines,patientswithaGFR ≥45mL/min mayproceeddirectlytoCTwithIVcontrast.ForGFR 30–45mL/min,radiologywillcalltheEDteamanddiscuss themeritsofadministeringIVcontrast.IftheEDteamelects toproceedwithIVcontrast,thevolume,timing(pre-orpostCT),androuteof fluidhydrationareallatthediscretionof theED.Computedtomographythroughputistherefore maximizedaspatientsmaybehydrated after receivingCT. ForcaseswithaGFR ≤30mL/min,radiologywilldiscussthe meritsofIVcontrastwiththeEDteam.Ifcontrastistobe administered,onehourofIVpre-hydrationisrecommended

priortoimagingifthereisnocontraindication. CommunicationbetweentheEDandradiologyteamshas beenstreamlinedwiththeintroductionofautomated messagesthatindicatewhenpre-hydrationhasbeeninitiated andcompleted.

Analysisofthecommunicationbetweentheradiologyand EDteamsrevealedthattherewereoftenmultiplecalls regardingapatient’shydrationstatus.Weintendtorepeata similaranalysiswiththeaboveinterventionstoassess forachangeinthenumberofdelayedCTstudiesdueto GFRconcerns.Werecommendthatinstitutions performtheirownanalysisofoutlierstounderstand opportunitiesforimprovementandtoexpediteoverall EDthroughput.

LIMITATIONS

Limitationsofourstudyincludethefactthatitwas conductedatasingle,urban,academic,tertiary-careED. Thispopulationmaynotbeindicativeofthatseenbyother EDs.Furthermore,residentsinourEDtakeownershipof difficultIVplacementaspartoftheirtraining.Thus,difficult IVplacementmaynotbeassociatedwithdelayedCTinother EDsthathavededicatedIVaccessteams.TheGFRcutoffs forcontrastadministrationinourEDareadmittedly stringent.Otherinstitutionswithlessstringentcutoffsmay notseeasmanysignificantlydelayedCTstudiesdueto GFRconcerns.

Forthepurposesofthisanalysis,patientfactorssuchas behavior,allergies,andmedicalacuitywereconsiderednonmodifiable.Futurestudiesmayconsiderreviewingprotocols forallergyprophylaxisorbehavioralde-escalation.Finally, weexcludedatotalof41.1%ofoutliers:8.2%duetoCTtimestampinconsistenciesand32.9%becausethecauseofthe delaycouldnotbeidentifieddespitethoroughreviewof outliercharts.

CONCLUSION

Weidentifiedmodifiableandnon-modifiablefactors associatedwithsignificantlydelayedCTintheED. InterventionssuchaspromptIVteaminvolvementor utilizationofultrasoundforIVplacement,revisionof institutionalpoliciesregardingcontrastadministrationin lowGFRsettings,andCTprotocolconsiderationmay addressupto42.6%ofoutliers.Theseinterventionsmay improveCTturnaroundtimesandEDthroughput. Futureresearchwillextendthisanalysisbymeasuringthe effectofrevisedinstitutionalpoliciesregarding contrastadministration.

AddressforCorrespondence:BryanA.Stenson,MD,BethIsrael DeaconessMedicalCenter,RosenbergBuilding-2,One DeaconnessRoad,Boston,MA02215.Email: bstenson@bidmc. harvard.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Dhaniketal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.PerotteR,LewinGO,TambeU,etal.Improvingemergencydepartment flow:reducingturnaroundtimeforemergentCTscans. AMIAAnnu SympProc. 2018;2018:897–906.

2.RyanA,HunterK,CunninghamK,etal.STEPS:leanthinking,theoryof constraintsandidentifyingbottlenecksinanemergencydepartment. Ir MedJ. 2013;106(4):105–7.

3.WhiteBA,YunBJ,LevMH,etal.Applyingsystemsengineeringreduces radiologytransportcycletimesintheemergencydepartment. WestJ EmergMed. 2017;18(3):410–8.

4.RachhP,PendleyAM,DuongPT,etal.DecreasingCTacquisitiontime intheemergencydepartmentthroughleanmanagementprinciples. Radiographics. 2021;41(3):E81–9.

5.KleinDandKhanV.Utilizingsixsigmaleanstrategiestoexpedite emergencydepartmentCTscanthroughputinatertiarycarefacility. JAmCollRadiol. 2017;14(1):78–81.

6.JosephJW.Queuingtheoryandmodelingemergencydepartment resourceutilization. EmergMedClinNorthAm. 2020;38(3):563–72.

7.StensonBA,JosephJW,AntkowiakPS,etal.Estimatingmaximum patientthroughputatanEDscanner:isthreepatientsperhourthemagic number? AmJEmergMed. 2021;40:159–61.

8.HennemanPL,NathansonBH,LiH,etal.Emergencydepartment patientswhostaymorethan6hourscontributetocrowding. JEmerg Med. 2010;39(1):105–12.

9.WorsterA,BledsoeRD,CleveP,etal.Reassessingthemethodsof medicalrecordreviewstudiesinemergencymedicineresearch. Ann EmergMed. 2005;45(4):448–51.

10.DavisEM,FeinsmithS,AmickAE,etal.Difficultintravenousaccessin theemergencydepartment:performanceandimpactofultrasoundguidedIVinsertionperformedbynurses. AmJEmergMed. 2021;46:539–44.

11.DavenportMS,PerazellaMA,YeeJ,etal.Useofintravenousiodinated contrastmediainpatientswithkidneydisease:consensusstatements fromtheAmericanCollegeofRadiologyandtheNationalKidney Foundation. Radiology. 2020;294(3):660–8.

12.HopkinsCL,MadsenT,FoyZ,etal.Doeslimitingoralcontrastdecrease emergencydepartmentlengthofstay?. WestJEmergMed. 2012;13(5):383–7.

13.HuynhLN,CoughlinBF,WolfeJ,etal.Patientencountertimeintervals intheevaluationofemergencydepartmentpatientsrequiring abdominopelvicCT:oralcontrastversusnocontrast. EmergRadiol. 2004;10(6):310–3.

14.LevensonRB,CamachoMA,HornE,etal.Eliminating routineoralcontrastuseforCTintheemergencydepartment: impactonpatientthroughputanddiagnosis. EmergRadiol. 2012;19(6):513–7.

forOlderPatients:AQualitativeStudy

MackenzieA.McKnight,MD*

MelissaK.Sheber,MS*

DanielJ.Liebzeit,PhD†

AaronT.Seaman,PhD‡

EricaK.Husser,PhD§ HarleahG.Buck,PhD†

HeatherS.Reisinger,PhD‡ SangilLee,MD,MS‡∥

*UniversityofIowa,RoyJ.andLucilleA.CarverCollegeofMedicine,IowaCity,Iowa

† UniversityofIowa,CollegeofNursing,IowaCity,Iowa

‡ UniversityofIowa,RoyJ.andLucilleA.CarverCollegeofMedicine,Departmentof InternalMedicine,IowaCity,Iowa

§ PennStateUniversity,RossandCarolNeseCollegeofNursing, UniversityPark,Pennsylvania

∥ DepartmentofEpidemiology,UniversityofIowaCollegeofPublicHealth, DepartmentofEmergencyMedicine,IowaCity,Iowa

SectionEditors:StephenMeldon,MD,andMarkLangdorf,MD,MHPE

Submissionhistory:SubmittedApril21,2023;RevisionreceivedOctober30,2023;AcceptedNovember21,2023

ElectronicallypublishedJanuary25,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18088

Introduction: Olderadultsoftenhavemultiplecomorbidities;therefore,theyareathighriskforadverse eventsafterdischarge.The4Msframework whatmatters,medications,mentation,mobility hasbeen usedinacuteandambulatorycaresettingstoidentifyriskfactorsforadverseeventsinolderadults, althoughithasnotbeenusedintheemergencydepartment(ED).Weaimedtodeterminewhether1)use ofthe4Msworksheetwouldhelpemergencycliniciansunderstandolderadultpatients’ goalsofcareand 2)useoftheworksheetwasfeasibleintheED.

Methods: Weconductedaqualitative,descriptivestudyamongpatientsaged ≥60yearsand emergencycliniciansfromJanuary–June2022.Patientswereaskedto fillouta4Msworksheet; followingthis,semi-structuredinterviewswereconductedwithpatientsandcliniciansseparately. Weanalyseddatatocreatecodes,whichweredividedintocategoriesandsub-categories.

Results: Atotalof20olderpatientsand19emergencyclinicianswereinterviewed.Weidentifiedtwo categoriesbasedonouraims:understandingpatientgoalsofcare(sub-categories:clinician/patient concordance;understandingunderlyinggoalsofcare;underlyinggoalsofcarediscrepancy)anduseof 4MsWorksheet(sub-categories:worksheettodiscussiondiscrepancy;challengesusingworksheet; challengecompletingworksheetbeforedischarge).Ratesofconcordancebetweenpatientandclinician onmainconcern/goalofcareandunderlyinggoalsofcarewere82.4%and15.4%,respectively.

Conclusion: Wefoundthatmostpatientsandemergencycliniciansagreedonthemaingoalofcare, althoughcliniciansoftenfailedtoelicitpatients’ underlyinggoal(s)ofcare.Additionally,manypatients preferredtohavetheinterviewer fillouttheworksheetforthem.Therewasoftendiscrepancybetweenwhat waswrittenandwhatwasdiscussedwiththeinterviewer.Moreresearchisneededtodeterminethebest waytointegratethe4Msframeworkwithinemergencycare.[WestJEmergMed.2024;25(2)230–236.]

INTRODUCTION

Olderadultswillaccountforover20%oftheUS populationinthenextdecade,andtheyaremorelikelyto havemultiplecomorbidities,takemoremedications,anduse morehealthcareresourcesthanindividualsinyoungerage

groups. 1 Thevisitratetoemergencydepartments(ED)inthe USin2019totaled43visitsper100forindividuals 65–74yearsofageand66visitsper100forindividuals ≥75years.Thevisitrateforthose ≥75washigherthanall otheragegroups(rangingfrom19–25visitsper100

individuals)exceptforthoseunderoneyearofage.Several studieshaveshownthatthereisalackofrecognitionofrisk factors(eg,polypharmacy,fallrisk,delirium)foradverse outcomesamongolderadultsintheED.2–4 Several assessmenttoolshavebeendevelopedintheEDtoevaluate riskfactors(eg,identificationofseniorsatrisk5),butthereis notooltoeffectivelycommunicatetheneedsof olderadults.6–8

The4Msframeworkofage-friendlyhealthsystemswas createdbytheJohnA.HartfordFoundationincollaboration withtheInstituteforHealthcareImprovement,American HospitalAssociation,andCatholicHealthAssociation.It incorporatesfourkeyelements: whatmatters,medication, mentation ,and mobility 9 Thesefourelementswere developedwithcurrentevidence-basedpracticeswiththe intentionofguidingclinicianconversationswitholderadults. The4Msframeworkconsidersthecommonriskfactorsfor adverseoutcomesinolderadultpatients(eg,riskfor delirium,potentiallyinappropriatemedications,and challengeswithmobility).

Whiletherehasbeenafocusonimplementingthe4Ms frameworkinacuteandambulatorycaresettings,therehave beenfewstudiesonitspotentialapplicationintheED.10 The EDisoneclinicalsettinginwhichitcouldbeimportantto discussthe4Mssothatallpotentialriskfactorsareidentified andcareistailoredtotheneedsofolderpatients.Thereis oftenatimeconstraintfortheemergencycliniciantoengage inalengthyconversationaboutwhatmatterstothepatient.

(“Whatmatters” entailsdiscussingthespecificdetailsthat mattertoeachpatientonadeeplevel,includingtheirgoals andpreferencesforcare,whichcanguidethecareteamand aligncaretowhattrulymatterstothepatient.)Useofthe 4Msframeworkcouldpotentiallypreventadverseoutcomes forolderadultpatientsreceivingcareintheEDby recognizingriskfactorssuchaspolypharmacy,fallrisk, anddelirium.

The4MsworksheetwasdevelopedbytheteamatAgeFriendlyCare,PA,aGeriatricWorkforceEnhancement ProgramatthePennStateRossandCaroleNeseCollegeof Nursing.Theworksheetisapatient-facingtoolthatallows individualstoidentifywhatmatterstothemandwhat questionstheymayhaveaboutpotentialproblemswith mobility,mentation,andmedications.Thetoolwas developedtohelpfacilitateconversationsbetweenpatients andclinicians,butitsusehasnotyetbeenevaluatedinthe ED(Appendix1).Weaimedtoevaluatethepotential usabilityofthe4MsworksheetintheEDtofacilitate conversationsaboutwhatmatters,medications,mention, andmobilitybetweenolderpatientsandemergency clinicians,andtoassesswhetherthe4Msworksheetmay supportemergencyclinicians’ understandingofpatients’ goalsofcare,includingbarriersandfacilitatorstousing theworksheet.

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Olderadultswithmultiplecomorbiditiesface highriskspost-discharge.The4Ms framework whatmatters,medication, mentation,andmobility isusedin varioussettings.

Whatwastheresearchquestion?

Cantheuseofthe4MsframeworkintheED settinghelpcliniciansunderstandolderadult patients ’ goalsofcare,andisitfeasible?

Whatwasthemajor findingofthestudy?

Ratesofconcordancebetweenpatientand clinicianonthemainconcern/goalofcareand underlyinggoalsofcarewere82.4%and 15.4%,respectively.

Howdoesthisimprovepopulationhealth?

Integrationofthe4Msframeworkin emergencycarecouldenhanceunderstanding andalignmentwitholderadults ’ underlying goalsofcare.

METHODS

StudyDesign

Weconductedthestudyusingaqualitative,descriptive approachinvolvingsemi-structuredinterviewswitholder adultpatients(aged ≥60years)andtheiremergencyclinician fromJanuary–June2022.Thisstudyispartofalargerstudy thatexaminedpatientgoalsreportedthroughthe4Ms worksheetusingthequalitativemethod.11–13 Inthepresent analysiswefocusedonusabilityoftheworksheetbypatients, aswellasemergencyclinicians’ understandingofpatients’ goalsofcare.11 Thelocalinstitutionalreviewboardapproved thisstudyanddeterminedittobeexempt.Weadheredtothe ConsolidatedChecklistforReportingQualitativeResearch (COREQ)guidelines.(Appendix2).

StudySetting

ThisstudytookplaceinasingleacademicEDwithLevelI traumaaccreditationandanannualcensusofapproximately 60,000patients.ClinicianspracticinginthisspecificED includeattendingphysicians,residents,andadvanced practiceproviders(APP).TheEDhasathree-year emergencymedicineresidencyprogramandan18-month fellowshipforAPPs;anyEMresidentorAPPwhomthe patientssawwouldbeenrolledinofoneofthesetraining

programs.Theinstitutioniscurrentlyapartofthe age-friendlyhealthsystemsmovement.

4MsWorksheet

The4MsworksheetwascreatedbyAge-FriendlyCare, PAatthePennStateRossandCarolNeseCollegeof Nursing,whichisaCenterofGeriatricNursingExcellence. Theworksheetexplainseachcategoryofthe4Ms framework whatmatters,medications,mentation,and mobility aprogramhousedintheage-friendlyhealthcare systemandprovidesspaceforthepatienttowriteashort answerresponseabouteach.

ParticipantsandSampling

Participantswererecruitedforaninterviewif1)theywere aged ≥60years;2)theywerecurrentlyreceivingcareinthe ED;3)theirchiefcomplaintwasnotrelatedtoalteredmental status;4)theywereabletoreadandunderstandthe4Ms worksheetwritteninEnglish,and5)theyhadalreadybeen evaluatedbyaclinicianintheED.Patientswhomet eligibilitycriteriawereapproachedbyamemberofthe researchteamtoprovidestudyinformation.Consentwas obtainedfromallparticipantsbeforebeginning theinterview.

DataCollection

Twomedicalstudents,MMandMS whoweretrained bytheprimaryinvestigatorSLandbyDLwhohasextensive experiencewithqualitativeresearch conductedsemistructuredinterviewswithpatientsandclinicians.The researchteam(SL,MS)developedaninterviewguide (Appendix3).Interviewresponseswerecapturedas handwrittennotesinsteadofbydigitalrecordingduetocost. Theresearchteamcollectedpatientage,gender,andtypeof clinicianinterviewed(staffphysician,residentphysician,or APP).Interviewersalsotook fieldnotes,whichcontainedthe reasonforthevisitandcontextualfactors(symptomrelief, diagnostictest,disposition,non-verbalaspectsofthe interview).TheinterviewerenteredalldataintoREDCap,a secureelectronicdatacapturetoolshostedatTheUniversity ofIowaHospitalsandClinics.14 Nocompensationwas providedforinterviews.

Patientinterviewstookplaceintheroomwherethe patient’sEDevaluationtookplace.Theemergencyclinician wasnotpresentforthepatientinterview.First,thepatient completedthe4Msworksheeteitherindependentlyor verballytoamemberoftheresearchteam,followedby discussion.(Patientswerenotaskedwhethertheyhad completeda4Msworksheetinpriorhealthcareencounters.) DiscussionincludedpatientgoalsforthecurrentEDvisit, questionsabouteachcategorywithinthe4Msframework, andhowthepatientfelttheirvisithadgoneoverall.Atthe endoftheinterview,theinterviewerofferedthepatientto keepthe4Msworksheetforuseasareferenceinfuture

healthcareencounters.Patientinterviewslasted 30–60minutes(includingthetimespentcompletingthe4Ms worksheet).Afterthepatientinterviewwascomplete,the patient’sclinicianwasinterviewedabouttheirperceptionof thepatient’sgoalsofcareandhowthosegoalswereelicited duringpatientassessment.Clinicianinterviewsoccurred withoutthepatientpresentandlasted1–5minutes.

Analysis

InterviewdatastoredinREDCapwasanalyzedby researchteammembersSL,MM,MS,andDLusingcontent analysis.15 Eachinterviewwasinitiallycodedbytwo members(MS,MM),whoreceivedabrieftrainingon qualitativecontentanalysis.Twofacultyinvestigators(SL andDL)reviewedthesecodesandmadefurther recommendationbeforeconsensuswasreached.Theentire researchteammettodiscusscoding;anydiscrepancieswere resolvedthroughgroupdiscussion.Identifiedcodeswere enteredintoacodebook(Appendix4),whichwas maintainedandupdatedthroughoutdataanalysis.Codes weregroupedintocategoriesandsub-categoriestodescribe thedata.Datacollectionandanalysisfollowedaniterative processandoccurredsimultaneously,whichallowedfor revisionstotheinterviewguidetoaddressgapsinthedata. Interviewscontinueduntiltheresearchteamjointly determinedthatnonewinformationrelevanttotheresearch aimswasemerging.

Rigor

Datawasanalyzedbyaresearchteamwithavarietyof backgroundstoreduceindividualbiasandimprove credibilityoftheresults.16,17

RESULTS

Weapproached21patientstoconductsemi-structured interviews;onedeclinedduetounknownreasons.Intotalwe interviewed20patientsand19cliniciansduringthesixmonthperiod.Itshouldbenotedthatthereasonforthesmall samplesizeinasix-monthperiodwasduetointerviewers MMandMSalsohavingmedicalschoolduties.Allbutone patienthadacliniciantointerview;theoriginalclinicianfor thatonepatientwasnolongerintheED,andthenew clinicianwasunabletoanswerthequestions,asthepatient wasbeingdischarged.Nineattendingphysicians,eight residents,andoneAPPparticipatedinthestudy(Table1). Theinterviewprocessforpatienttookabout30–60minutes (median45minutes),includingtimeto filloutthe4Ms worksheet,and1–5minutes(median3minutes)tointerview emergencyclinicians.

Patientandclinicianinterviewsresultedinthreethemeson thetopicofunderstandingpatientgoalsofcare:clinician/ patientconcordance;understandingunderlyinggoalsof care;andunderlyinggoalsofcarediscrepancy(summarized in Table2).The4Msworksheetwasusedtofacilitate

conversationsaboutwhatmatters,mentation,mobility,and medications,whichsupportunderstandingofpatients’ goals ofcare,includingbarrierstoandfacilitatorsofits use(Appendix4).

UnderstandingPatientGoalsofCare Clinician/patientconcordance

Inmanycases,patientsandcliniciansarrivedat concordantperceptionsofthegoalsofcare.Thiswas indicatedwhenpatientandclinicianagreedonthemain concernandgoalsofcareforthevisit,suchasinthe followingexamples:

Ininterview2,boththepatientandclinicianagreedthat themainconcernwasaddressingthepatient’sfeverand coordinatingcaretoaddressrecurrentfeversrelatedto chemotherapy;ininterview3,boththepatientandclinician agreedthatthegoalofcarewasevaluationafterfalland beingabletocontinuelivingindependently;ininterview4, boththepatientandclinicianagreedthatthegoalofcarewas addressingsymptomsofconstipationandabdominalpain;in interview9,boththepatientandclinicianagreedthatthe

mainconcernthatbroughtthepatienttotheEDwaschest pain;ininterview10,boththepatientandclinicianagreed thatthegoalofcarewasaddressingsymptomsoffatigue;and ininterview11,boththepatientandclinicianagreedthatthe goalofcarewasrulingoutseriouscardiacpathology.Inthis lastcase,boththepatientandclinicianidentifiedafearthat thepatient’schestpainmayhavepointedtoseriouscardiac pathologygiventhepatient’shistory.Theclinicianelicited theforebodingfeelingthatthepatientwashaving.

Ininterview14,boththepatientandclinicianunderstood thatthepatient’schestpainwaswhatmatteredmost.In interview15,boththepatientandclinicianwantedtoaddress abdominalpain.Ininterview16,boththepatientand clinicianagreedthatthemaingoalofcarewaspain management.Ininterview17,thepatientfeltthattheywere treatedwellandwhenaskedwhethertheclinicianhad addressedtheirconcerns,answered, “Yeah,everyonewas nice.” However,forthispatient,therewasnoclinician perspectivetocompareto.Ininterview20,thepatient reportedthatshewaskeptuptodate(onhercare)andfelt thattheEDcliniciandid “just fine ” inaddressinghergoalsof care,questions,andconcerns.

Overall,14of19patientsandcliniciansagreedonthe mainconcernorgoalsofcareforthevisit(Table3).Further, wegroupedtheseintosymptomevaluation(Interviews2,4, 11);symptommanagement(Interview16);symptom evaluation/management(Interview3);genericagreement (Interviews9,10,14);andmiscellaneous(interviews17,20).

UnderstandingUnderlyingGoalsofCare

Insomecases,thepatientandclinicianagreedon underlyinggoalsofcareforthepatient.Anunderlyinggoal ofcareisdefinedasaspectsofwhatmatterstothepatientin theirdailylifeorhealththataffecttheirmainconcernand goalofcare.Examplesareasfollows:Ininterview2,boththe patientandcliniciansuggestedthatarelatedgoalofcarewas tocoordinatewiththeteammanagingthepatient’scancerto developacareplangoingforwardthatwouldaddresstheir recurrentfeversandchemotherapyissues;andininterview3, boththepatientandclinicianagreedthatanunderlyinggoal ofcarewastobeabletocontinuelivingindependently. Overall,2of19patientsandcliniciansagreedonunderlying goalsofcare(Table3).

UnderlyingGoalsofCareDiscrepancy

Clinician/patient

Understandingunderlying

Underlyinggoalsofcare

Despiteagreeingonmainconcernsandgoalsofcare, patientsandcliniciansoftendidnotagreeonunderlying goalsofcare.Thiswasthecasewheninterpretationofwhat mattersforthepatientandclinicianwasdiscrepant,asshown intheseexamples:Ininterview5,theclinicianmentionedthat themaingoalofcarewaspainrelief,andthatthepatient wouldratherbeathomeand “soilhimself” thanbeatthe [deidentified]hospital.Thepatient’smaingoalofcarewasto workonphysicaltherapy,gainstrength,andgetoffsome

Table1. Patientandemergencycliniciandemographicdata.

Table2. Categoriesandsub-categoriesforunderstandingpatient goalsofcareandutilizationofthe4Msworksheet.

Attendingn = 10,(%) response = yes

Resident(MD/DOorAPP) n = 8,(%)response = yes

APPn = 1,% response = yes Concordance (%)

medications.Thus,weidentifiedthemesofmedication concernsandmaintainingindependencefromthisinterview. Ininterview4,theclinicianidentifiedimprovingsymptoms asthemaingoalofcare,butthepatientidentifiedthemain goalasimprovingindependence.Thus,weidentifiedtheme ofmedicationconcernsfromthisinterview.

Duringthe19clinicianinterviews,17wereabletoidentify themainconcernthatbroughtthepatienttotheED.Ofthese 17clinicians,14(82.35%)mentionedamainconcernorgoal thatmatchedwiththepatient’sgoal.Thirteenclinicians mentionedanunderlyinggoalofcareforthepatient.Ofthese 13,onlytwoclinicians(15.38%)mentionedanunderlying goalofcarethatmatchedwiththepatient’sunderlyinggoals (Table3).Theresponsesonpatientperspective,main concern,andunderlyinggoalsofcareweresimilarbetween attendingandresidentphysicians.

Utilizationofthe4MsWorksheet

Implementationofthe4Msworksheetrevealedmultiple potentialbarrierstoandfacilitatorsofitsuse,including worksheettodiscussiondiscrepancy,challengesusingthe worksheet,andchallengecompletingtheworksheetbefore discharge(Table2).

WorksheettoDiscussionDiscrepancy

Insomecases,patients’ answerstopromptsonthe4Ms worksheetdidnotmatchinformationobtainedthrough discussionwiththeresearchteam,asshowninthese examples:Ininterview4,thepatientanswered “ no ” to medicationconcerns,butdiscussedmanyissuesrelatedto medications;ininterview3,thepatientdiscussedinformation relevanttodischarge/dispositionandrevealedopportunityto learnaboutmedicationsthatwasnotcapturedinthe worksheetanswers;andininterview20,thepatient wrote “ no ” tomedicationconcerns,buthadconcernsabout twooftheirmedications.Theseexamplescouldindicate thattherewasnotenoughspaceontheworksheetto providetheinformation,orthatthepatientdidnotcareto fill outtheworksheetindetail.Apossibilityalsoexists thatthepatientwasremindedofmoredetails throughdiscussionthattheydidnotthinkaboutbefore.

ChallengesUsingtheWorksheet

Manypatientspreferredaverbaldiscussionaboutwhat matteredtothemintheircareasopposedto fillingoutthe worksheet.Nineof20participantsdidnotfeelcomfortable with fillingoutthe4Msworksheet,andinterviewersoffered to fillitinforthem.Someappearedtobeuneasycompleting theworksheet,asshownintheseexamples:Patient2began fidgetingwiththeworksheetandexpresseddiscomfortwith fillingitout,statingthatuneasewithworksheetsextended backtobeinginschoolasachild;patient5couldnotreadthe questions,andtheinterviewerreadthemtothepatientand also filledouttheworksheetwiththeiranswers;patient11did notwantto fillouttheworksheetalonebutwashappyto allowtheinterviewertodoso;patient19hadParkinsonism and,therefore,wasunabletowriteontheworksheet;the interviewer filledouttheworksheetforthispatient.

ChallengeCompletingtheWorksheetBeforeDischarge

Therewerealsologisticalchallengeswithcompletingthe 4Msworksheet,includingnothavingenoughspaceonthe worksheettoadequatelyanswerthequestions,asthereare onlysomanylinesavailabletowriteundereachelement. Limitedtimewasanotherchallenge,asshowninthis example:Patient17’sinterviewwasperformedjustbefore theyweredischarged,soitfeltrushedasthepatientwas gettingreadytoleave.Theinterviewerandpatientwerealso interruptedtwiceduringtheinterview.

Overall,theuseofthe4Msworksheetrequiredadditional personneltohelpinterpretquestionsand filloutquestion items.Anydowntimewasusedto finishthissentence,which providedopportunitytocompletetheworksheet.

DISCUSSION

Wefoundthatemergencyclinicianshaveagood understandingofproblem-orientedgoalsofcarebutnot underlyinggoals.Further,thepotentialusability of4Msworksheettofacilitatetheconversationbetween patientsandemergencycliniciansfaceschallenges.A successfulimplementationofthe4Msframeworkin theEDiskeyinintegratingemergencycareintothe age-friendlyhealthsystem.Themesweidentifiedhighlight

Table3. Clinicianunderstandingofpatientgoalsofcare.

theuniqueaspectof4Msandtheworksheettofacilitate suchcare.

Intermsofunderstandingpatientgoalsofcare,we identifiedclinician/patientconcordance,understanding underlyinggoalsofcare,andunderlyinggoalsofcare discrepancy.Inmostinterviews,patientsandclinicians agreedonaproblem-orientedgoalrelatedtothepatient’ s reasonforpresentingtotheED.However,wefoundthat mostemergencycliniciansdidnotevokethepatient’ s underlyinggoalsforthevisit.Theliteratureonusingthe4Ms frameworktoelicitgoalsofcareamongolderadultsinthe EDislimited.Onestudyfoundthatwhendiscussing “what matters” tothepatient,emergencycliniciansandpatients agreedthatdischarginghomeorreduction/resolvingof symptomswasahighpriority,butemergencycliniciansoften didnotidentifythepatients’ desiretoreturntoprior functionalability.18 Ourstudy findingsaresimilarinthat manypatientsinterviewedhadunderlyinggoals,butthese goalswerenotdescribedbytheclinician.

Intermsofbarriersinusingthe4Msworksheet,we identifiedworksheet-to-discussiondiscrepancy,challenges usingtheworksheet,andchallengescompletingthe worksheetbeforedischarge.Nearlyhalfofthepatientsdid notfeelcomfortable fillingouttheworksheetandrequired theinterviewertoassistthemindoingso.Thiswasduetoa varietyofreasons.Ininstancesinwhichthepatientsdid fill outtheworksheet,therewasoftenadiscrepancybetween whatwaswrittenontheworksheetandwhatwasdiscussed duringtheinterview.Inoneinstancetheusageofthe worksheetanddiscussionfeltrushedduetothepatient’ s impendingdischarge.Inthesecases,patientsmaynothave fullydescribedtheirgoals,andinterviewersmaynothave askedmorein-depthquestionsabouttheirgoals.10

Thereislimitedliteratureontheuseofthe4Msworksheet inthecontextoftheED.Onepapersuggestedusingateam approachtoevaluatingthe4MsinelderlypatientsintheED (eg,pharmacistsshoulddiscussmedications,andsocial workersshoulddiscussmobility).Anotherstudyused transitionalcarenursesintheEDtoevaluateelderlypatients forcognitionandmobilityandfoundthatusingsuchcare nursesdecreasedadmissionsandreadmissionstothe hospital.19,20 Ourstudyisuniqueinthatweusedaworksheet basedonthe4Msframeworkandevaluateditsfeasibilityfor useintheED.Giventheamountoftimethatthediscussions take,wesuggestusingateamapproachorhaveadedicated persontohave4Msdiscussionswithpatients.

StrengthsandLimitations

Strengthsofourstudyincludethein-depthdiscussionwith patients,whichallowedustounderstandtheirgoalsand whatmatterstothem.Anotherstrengthisthatwehad multiplemembersoftheresearchteamcodingthesame interviews;thisallowedustoaddmoreobjectiveanddiverse pointsofview.However,therewerealsoseverallimitations

toourstudy.Thesamplesizewaslimited,andwewere unabletorecruitasufficientnumberofAPPs.Sincewe recruitedthesesubjectsduringactiveclinicalcare,thetime thatphysicians/APPshadforthisinterviewwasabout1-5 minutes,whichmayhavecausedbias.Patientswereenrolled fromasinglecenter,limitingtransferabilitytootherhealth systemsandgeographicregions.Theinterviewswerenot recorded;soverbatimquoteswerenotpossibleinallcases, whichmayhavecausedrecallbias.Also,wedidnothave accesstodemographicvariables,againaffecting transferabilityof findings.Therewasnoquantitative measurementofdiscrepancyinthecodingresults.

FutureImplications

Implicationsofuseofthe4Msworksheetforclinicians includeincreasedtimespentwithpatientsandgreaterpatient satisfaction,butalsoincludesincreasedprobabilityoffalling behindinpatientcare.Implicationsofuseofthe4Ms worksheetforpatientsincludeincreasedsafetyandneeds beingmet,potentialavoidanceofhospitaladmission,and improvedpatientoutcomes.Itwouldbeinterestingtosee whetherearlyuseofthe4MsworksheetintheEDcourse withsubsequentavailabilityforEDcliniciansallowsgreater concordanceinthegoalsofcare.Theuseofthe4Ms frameworkforemergencycareisnotfullydeveloped,andthe useoftheworksheetcanfacilitatesituation-specificcare(eg, dischargeplanning).

CONCLUSION

Wefoundthatusingthe4Msframeworkasaguideforthe careofolderadultpatientsintheEDcanhelpelicit underlyinggoalsofcare.Wewereabletoanswerwhether patients’ goalsofcarewerecongruentwithwhatthe emergencyclinicianbelievedthepatients’ goalswererelated tothepresentingproblemandwiththepatients’ underlying goals.Ourstudyalsofoundthattheuseofthe4Msworksheet intheEDneedsmoreresearchonhowtobestincorporateit intothecareofolderadultpatients,asmanyolderadults mayneedadditionalassistanceto fillitout.Wesuggestthat the4Msworksheetcanbeusedwitholderpatientswho presenttotheEDtoguideconversationswithclinicians.This studyispreliminary,andrequiresavalidationstudyto furthertesttheworksheet’sutilityandacceptabilityin theED.

AddressforCorrespondence:SangilLee,MD,MS,Universityof Iowa,RoyJ.andLucilleA.CarverCollegeofMedicine,Department ofInternalMedicine,200HawkinsDR,IowaCity,IA52242.Email: sangil-lee@uiowa.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources

and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024McKnightetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.AminzadehFandDalzielWB.Olderadultsintheemergency department:asystematicreviewofpatternsofuse,adverseoutcomes, andeffectivenessofinterventions. AnnEmergMed. 2002;39(3):238–47.

2.LaMantiaMA,MessinaFC,HobgoodCD,etal.Screeningfordeliriumin theemergencydepartment:asystematicreview. AnnEmergMed. 2014;63(5):551–60.e2.

3.CarpenterCR,AvidanMS,WildesT,etal.Predictinggeriatricfalls followinganepisodeofemergencydepartmentcare:asystematic review. AcadEmergMed. 2014;21(10):1069–82.

4.HusteyFM,WallisN,MillerJ.InappropriateprescribinginanolderED population. AmJEmergMed. 2007;25(7):804–7.

5.AsomaningNandLoftusC.Identificationofseniorsatrisk(ISAR) screeningtoolintheemergencydepartment:implementationusingthe plan-do-study-actmodelandvalidationresults. JEmergNurs. 2014;40(4):357–64.e1.

6.HanJH,ZimmermanEE,CutlerN,etal.Deliriuminolderemergency departmentpatients:recognition,riskfactors,andpsychomotor subtypes. AcadEmergMed. 2009;16(3):193–200.

7.HusteyFM,MeldonSW,SmithMD,etal.Theeffectofmentalstatus screeningonthecareofelderlyemergencydepartmentpatients. Ann EmergMed. 2003;41(5):678–84.

8.BatenV,BuschH-J,BuscheC,etal.Validationofthebriefconfusion assessmentmethodforscreeningdeliriuminelderlymedicalpatientsin

aGermanemergencydepartment. AcadEmergMed. 2018;25(11):1251–62.

9.CacchionePZ.Age-friendlyhealthsystems:the4Msframework. Clin NursRes. 2020;29(3):139–40.

10.BambachKandSoutherlandLT.Applyinggeriatricprinciplesto transitionsofcareintheemergencydepartment. EmergMedClinNorth Am. 2021;39(2):429–42.

11.SheberM,McKnightM,LiebzeitD,etal.335 “4Ms” conversationinthe emergencydepartment:aqualitativestudy. AnnEmergMed. 2022;80(4):S143.

12.SandelowskiM.Whateverhappenedtoqualitativedescription? Res NursHealth. 2000;23(4):334–40.

13.HsiehH-FandShannonSE.Threeapproachestoqualitativecontent analysis. QualHealthRes. 2005;15(9):1277–88.

14.GarciaKKSandAbrahãoAA.ResearchdevelopmentusingREDCap software. HealthcInformRes. 2021;27(4):341–9.

15.GaleNK,HeathG,CameronE,etal.Usingtheframeworkmethodfor theanalysisofqualitativedatainmulti-disciplinaryhealthresearch. BMCMedResMethodol. 2013;13:117.

16.SandelowskiM.Rigororrigormortis:theproblemofrigorinqualitative researchrevisited. ANSAdvNursSci. 1993;16(2):1–8.

17.SandelowskiM.Theproblemofrigorinqualitativeresearch. ANSAdv NursSci. 1986;8(3):27–37.

18.GettelCJ,VenkateshAK,DowdH,etal.Aqualitativestudyof “what matters” toolderadultsintheemergencydepartment. WestJEmerg Med. 2022;23(4):579–88.

19.HwangU,DresdenSM,RosenbergMS,etal.Geriatricemergency departmentinnovations:transitionalcarenursesandhospitaluse. JAm GeriatrSoc. 2018;66(3):459–66.

20.DresdenSM,HwangU,GarridoMM,etal.Geriatricemergency departmentinnovations:theimpactoftransitionalcarenurses on30-dayreadmissionsforolderadults. AcadEmergMed. 2020;27(1):43–53.

NationalCharacteristicsofEmergencyCareforChildrenwith NeurologicComplexChronicConditions

KaileenJafari,MD*†

KristenCarlin,MPH†

DeryaCaglar,MD*†

EileenJ.Klein,MD,MPH*†

TamaraD.Simon,MD,MSPH‡

SectionEditor:RonnieWaldrop,MD

*UniversityofWashington,DepartmentofPediatrics,Seattle,Washington † SeattleChildren’sResearchInstitute,CenterforClinicalandTranslational Research,Seattle,Washington

‡ UniversityofSouthernCalifornia,KeckSchoolofMedicine,Departmentof Pediatrics,LosAngeles,California

Submissionhistory:SubmittedJanuary25,2023;RevisionreceivedNovember16,2023;AcceptedNovember21,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17834

Introduction: Mostpediatricemergencycareoccursingeneralemergencydepartments(GED),where lesspediatricexperienceandlowerpediatricemergencyreadinessmaycompromisecare.Medically vulnerablepediatricpatients,suchasthosewithchronic,severe,neurologicconditions,arelikelytobe disproportionatelyaffectedbysuboptimalcareinGEDs;however,littleisknownaboutcharacteristicsof theircareineitherthegeneralorpediatricemergencysetting.Inthisstudyourobjectivewastocompare thefrequency,characteristics,andoutcomesofEDvisitsmadebychildrenwithchronicneurologic diseasesbetweengeneralandpediatricEDs(PED).

Methods: Weconductedaretrospectiveanalysisofthe2011–2014NationwideEmergencyDepartment Sample(NEDS)forEDvisitsmadebypatients0–21yearswithneurologiccomplexchronicconditions (neuroCCC).Wecomparedpatient,hospital,andEDvisitscharacteristicsbetweenGEDsandPEDs usingdescriptivestatistics.Weassessedoutcomesofadmission,transfer,criticalprocedure performance,andmortalityusingmultivariablelogisticregression.

Results: Therewere387,813neuroCCCEDvisits(0.3%of0–21-year-oldEDvisits)inoursample.Care occurredpredominantlyinGEDs,andvisitswereassociatedwithahighseverityofillness(30.1% highestseverityclassificationscore).ComparedtoGEDvisits,PEDneuroCCCvisitswerecomprisedof individualswhowereyounger,morelikelytohavecomorbidconditions(32.9%vs21%, P < 0.001),and technologyassistance(65.4%vs.45.9%)butunderwentfewerproceduresandhadlowerEDcharges ($2,200vs$1,520, P < 0.001).VisitstoPEDshadloweradjustedoddsofcriticalprocedures(adjusted oddsratio[aOR]0.74,95%confidenceinterval[CI]0.62–0.87),transfers(aOR0.14,95%CI0.04–0.56), andmortality(aOR0.38,95%CI0.19–0.75)comparedtoGEDs.

Conclusion: CareforchildrenwithneuroCCCsinapediatricEDisassociatedwithlessresource utilizationandlowerratesoftransferandmortality.IdentifyingfeaturesofPEDcareforneuroCCCscould leadtolowercostsandmortalityforthispopulation.[WestJEmergMed.2024;25(2)237–245.]

INTRODUCTION

Approximately30millionemergencydepartment(ED) visitsaremadebychildrenintheUnitedStatesannually.1 Previousstudieshaveshownthatmostpediatricpatients receiveemergencycareingeneralEDs(GED),whereless pediatricexperienceandlowerpediatricemergency

readinessmaycompromisecare.2,3 CareinGEDsmaybeless likelytofollowevidence-basedguidelinesforcommon pediatricconditionssuchasheadtrauma,croup,and asthma.Inaddition,GEDsmayhavehigherratesof potentiallyavoidabletransfers(giventhelikelihoodof transferwhenthepatientdidnotpresentinitially

toaspecialtyhospital)andhighermortalityin criticalillness.4–6

Childrenandyoungadultswithchronic,severeneurologic conditionsareamedicallyvulnerablepopulationwithpoor functionalstatusandhighratesofcomorbiddiseaseandneed fortechnologyassistance(ie,gastrostomytubeorventricular peritonealshunts).Thesecharacteristicsmakethis populationatriskoffrequentEDutilization,uniquemedical presentations,andhighacuityofillness.7–11 Priorstudies havefoundchildrenwithneurologicdiseaseaccountforup to13%ofinpatientpediatricadmissions,andonethirdof inpatientpediatrichealthcarecosts.7,8 AlthoughGEDs providemostemergencycareforchildren,emergency medicine(EM)traineesandGEDclinicianshaveless experienceandconfidenceinprovidingcaretopediatric patientscomparedtotreatingadults.12,13 Despitegrowing evidencethatchildrenwithchronicneurologicconditionsare atriskoffrequentEDusage,thereislimiteddatacomparing thedistribution,characteristics,andoutcomesofemergency careforthisspecificpopulationbetweenGEDsandpediatric EDs(PED).

Inthisstudyweaimedtodescribethenationalestimatesof childrenandyoungadultpatientswithneurologiccomplex chronicconditions(neuroCCC)andcomparethe characteristicsandoutcomesofEDvisitsinGEDsand PEDs.WehypothesizedthatPEDvisitswouldbeassociated withlowerratesofresourceutilization,transfer, andmortality.

METHODS

StudyDesignandDataSource

Weconductedacross-sectionalanalysistoidentifythe frequencyandcharacteristicsofEDvisitsamongpatients aged0–21yearswithneuroCCCsbetween2011–2014inthe NationwideEmergencyDepartmentSample(NEDS).The NEDSispartoftheHealthcareCostandUtilizationProject databasethatissponsoredbytheAgencyforHealthcare ResearchandQuality.Itisthelargestall-payer,nationally representativeEDdatabaseintheUnitedStatesandcontains a20%stratifiedsampleofhospital-basedEDs.Thesampling strategydeliberatelyencompassesbetween945–955hospitalbasedEDsin24–34statesandapproximately 120–137millionweightedvisitstotheEDannually.14 Hospital-basedEDsarestratifiedbyUScensusregion, traumadesignation,urban-rurallocation,hospital ownership,andteachingstatus.Additionally,asnopatient identifiersareavailable,individualpatientscannotbe trackedlongitudinally,andencountersthatoriginated inonefacilityandweretransferredwillhaveaseparate encounterinthereceivingfacility.Theunitofanalysisisthe EDvisit.ThestudywassubmittedtotheSeattleChildren’ s InstitutionalReviewBoard(IRB)andwasdetermined tobeIRBexempt.

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Childrenwithchronicneurologicdiseasesare atriskforsevereillnessandpooroutcomesin theemergencydepartment.

Whatwastheresearchquestion?

Doesemergencycaredifferbetweengeneral (GED)andpediatricEDs(PED)for childrenwithchronicneurologicdisease?

Whatwasthemajor findingofthestudy?

ChronicneurologicalpediatricvisitsinPEDs hadloweroddsofmortalityintheED(aOR 0.4,95%CI0.2 – 0.8)comparedtoGEDs (0.04%vs0.13%).

Howdoesthisimprovepopulationhealth?

IdentifyingfeaturesofpediatricEDcarefor childrenwithchronicneurologicconditions couldimprovemortalityinthis high-riskpopulation.

WeincludedallEDvisitsmadebypatients0–21yearsof agewithanInternationalClassificationofDiseases,9th Rev (ICD-9)diagnosisconsistentwithaneuroCCC.15 The classificationofCCCs,originallydevelopedbyFeudnterand colleagues,isanorgan-systembasedclassificationof complexdiseasesofchildhood.15 WeexcludedEDvisitsthat weremissingage,primarydiagnosis.ortheprimary diagnosiswasinvalid.

OurprimarypredictorvariablewasthecategoryofED, eitherGEDorPED.TheNEDSdoesnotspecifically designatePEDs;thus,consistentwithsimilarpublished reports,wedesignatedPEDsasthosewhere ≥75%ofvisits werechildren <18yearsofage.AllotherEDswere categorizedasGEDs.3,5,16–18 Ofnote,whilewedefinedPED baseduponproportionofvisitsmadebychildren <18years, weincludedencountersinthisstudyuptoage21,aspatients withchronicmedicalconditionsoftencontinuetoseekcare inPEDsintoyoungadulthoodastheytransitiontoadult care.2,3 Patient-levelpredictorsincludedthefollowing: 1)demographics,insurancepayer,medianincomequartile, andtheurban-ruralclassificationofthepatient’sareaof residence;2)neuroCCCdiagnosticcategory;3)numberof non-neuroCCCs);4)presenceoftechnologyassistanceand CCCICD-9codes.15 Weidentifiedspecifictechnologies, includingventricularshunts,feedingtubes(gastrostomy, gastro-jejunostomy,andjejunostomytubes)and tracheostomies,specificallyusingthecorresponding

technologyassistancecodes.15 Hospital-levelpredictors includedteachingstatusoftheED,traumacenter designation,andhospitalregion.

Ourprimaryoutcomesofinterestwereresource utilization,severityofillnesspresentation,disposition (admissionandtransfer),andmortality.Resourceutilization wasassessedthroughEDcharges,frequencyofprocedures performed,anddiagnosticimaging.AstheNEDSprovides onlyfacilitycharges,andcost-to-chargeratioswerenot availablefortheyearsselected,wereporttotalchargesforthe EDandinpatientstay.Thisapproachisconsistentwithprior publishedstudies.19,20 Weusedtotalnumberofcurrent proceduralterminology(CPT)codes,ratherthanICD-9 proceduralcodes,toassessprocedurefrequencyasa significantlyhigherproportionofEDvisitshadCPTcodes available.Diagnosticimaging(includingradiograph, ultrasound,computedtomography(CT)ormagnetic resonanceimaging(MRI)andcross-sectional imaging(CTorMRIonly)wasreportedbasedontheCPT andICD-9proceduralcodesassociatedwiththe EDvisit.

Weassessedtheoutcomeofseverityofclinical presentationusingseverityclassificationscores(SCS)and criticalprocedureperformance.2,21 TheSCSisaPediatric EmergencyCareAppliedResearchNetworkconsensusderiveddiagnosticsystemthatreliesonthemostsevereICD9diagnosticcodesattachedtoeachrecordtoassigneachED visitaseverityscore.Theseverityscorerangesfrom1 (minimalresourcesused)to5(maximalresourcesused).21

CriticalproceduresweredefinedasthepresenceofanICD-9 codeforendotrachealintubation,centrallineplacement,and chesttubeplacementaspreviouslydescribedinthe literature.22,23 Mortalitywascategorizedas(1)EDmortality and(2)visitmortality(deathatanypointduringED visitorhospitalization).

StatisticalAnalysis

Weincorporatedsamplingweightstoconsiderthe significantsurveydesignandsamplingproceduresofthe NEDS.Descriptivestatistics,includingfrequencies, proportionsandsumsasappropriate,wereusedto summarizepatientandhospitalcharacteristics.Wemade comparisonsusingchi-squareorANOVAtestforcategorical variables,and t -testsforcontinuousvariables.Multivariable logisticregressionwasperformedfor fivedifferentED outcomevariables(admission,transfer,transferor admission,mortality,andcriticalprocedureperformance). Predictorvariablesincludedinlogisticregressionwere patient-levelvariables(demographics,numberofCCCs, technologyassistance,SCSscore),andhospital-level predictors(traumacenterdesignation,geographic location,PEDvsGED).Resultswerereportedas adjustedoddsratios(aOR)and95%confidence intervals(CI).

RESULTS

Oftheestimated141millionweightedEDvisitsmadeby patientsaged0–21yearsinthe2011-2014NEDS,387,987 (0.3%)hadaneuroCCCdiagnosis.MostneuroCCCED visitsoccurredinGEDs(74.9%),andtheremainderoccurred inPEDs(25.1%).NeuroCCCsvisitsrepresented proportionatelymoreofall0–21-year-oldPEDvisits comparedtoGEDvisits(1.0%vs0.2%).Thepatient-level characteristicsofneuroCCCEDvisitsareshownin Table1. Youngerpatients(ages ≤9years)represented proportionatelymoreofPEDthanGEDvisits(63vs48%, P < 0.01).TherewasapredominanceofmalesinbothGED andPEDvisits(55.8%vs55.9%).Theprimarypayerformost GEDandPEDvisitswasMedicaid(56.1%vs60.6%),and incomequartile(notshown)wasnotsignificantlydifferent betweenGEDsandPEDs.

Therewasahighrateofcomorbidchronicconditions overallinchildrenwithneuroCCCs,withoneinfourED visitsassociatedwithatleastonenon-neurologicCCC (93,075,24%, Table1).Themedicalcomplexityofneuro CCCvisitswashigherinPEDscomparedtoGEDs,with 32.9%ofpatientsinPEDswithatleastoneadditionalCCC comparedto21%inGEDs.Technologyassistancewasmore frequentinPEDthanGEDencounters(65.5%vs49.5%)and wascomprisedmostlyofventricularshunts(161,868, 41.7%),feedingtubes(56,568,14.6%)and,lesscommonly, tracheostomies(17,653,4.6%). SupplementalTable1

demonstratesthefrequencyofsubcategoriesofneuroCCCs andthemostcommoncategoriesofnon-neuroCCCs.

Hospitalcharacteristicsarealsoshownin Table1.Over 80%ofPEDvisitswereinmetropolitanlocations,teaching facilities,andLevelI/IItraumacenters,consistentwith underlyingdifferencesbetweenthesetwocategoriesof EDs.2,18 Regionally,PEDvisitswerepredominantlyfrom theWest,whileGEDvisitswerepredominantlyfromthe South.TheNortheastregionaccountedforthelowest proportionofvisits,18.9%ofneuroCCCEDvisitsoverall andonly1.4%ofPEDvisits(datanotshown).

CharacteristicsofemergencyvisitcareforneuroCCCED visitsaredemonstratedin Table1.Severeillness presentationswerecommon;30.1%ofvisitshadaSCS5 indicatingcriticalillness.ThePEDshadfewerSCS5 presentationsthanGEDs,(26.4%vs31.3%vs. P = 0.002), andmoreSCS4presentationsthanGEDs(55.8%vs.47.5%, P =< 0.001).PEDsvisitshadfeweroverallprocedures performed(0proceduresperformedin34.9%PEDvs13.4% inGEDs(P = 0.048)andlessimaging(45.7%vs24.4%,not significant).Endotrachealintubationwasthemost frequentlyperformedcriticalprocedureandoccurredless frequentlyinPEDscomparedtoGEDs(5.7%vs8.3%, P = 0.003).

MedianEDchargesweresignificantlylowerinPEDs comparedtoGEDs(P < 0.001).VisitstoPEDS,hadhigher proportionofadmissions(55.1%vs42.8%, P < 0.001)and

N(%)GeneralED(n = 290,641)PediatricED(n = 97,346)AllEDvisits(n = 387,987)Pvalue

Patientcharacteristics

Ageinyears

0–9139,249(47.9%)61535(63.2%)200,706(51.7%) <0.001

10–1779,886(27.5%)29,138(29.9%)109,024(28.1%)

18–2171,505(24.6%)6,712(6.9%)78,257(20.2%)

Urbanicity

Largemetro142,792(49.1%)72,560(74.5%)215,353(55.5%)0.18

Medium/smallmetro100,482(34.5%)15,978(16.4%)116,460(30.0%)

Non-metro/unknown47299(16.3%)8808(9.0%)56107(14.5%)

Primarypayer

Medicaid162,965(56.1%)58,950(60.6%)221,916(57.2%)0.50

Privateinsurance98780(34.0%)30,832(31.7%)129,612(33.4%)

Medicare/other28,807(9.9%)7393(7.6%)11,868(9.4%)

Complexity

>1additionalCCC69,038(21.0%)32,037(32.9%)93,044(24.0%) <0.001

Technologyassistance143,788(49.5%)63,707(65.45%)207,495(53.5%) <0.001

Hospitalcharacteristics

Teachinghospital195,506(67.3%)97,035(99.7%)292,542(75.4%) <0.001

Traumacenter(I/II)165,244(56.9%)90,905(93.4%)256,149(66.2%) <0.001

Largemetrolocation150,888(51.9%)80,818(83.0%)231,744(59.7%)0.015

Visitcharacteristics

Disposition

Admission124,350(42.8%)53,659(55.1%)178,008(45.9%) <0.001

Transfer27,392(9.4%)1,242(1.3%)28,633(7.4%) <0.001

DeathintheED380(0.13%)47(0.04%)427(0.11%)0.003

Criticalprocedures1

Endotrachealtube20,059(8.3%)4,220(5.7%)24,278(7.7%)0.003

Centralvenousline11,210(4.6%)3,873(5.3%)15,115(4.8%)1,763(0.6%)0.32

Chesttube1,511(0.6%)255(0.3%)77,574(20.0%)0.004

Severityclassificationscore

<360,545(20.8%)17,030(17.5%)192,480(49.6%)0.007

4138,084(47.5%)54,357(55.8%)116,823(30.1%) <0.001

591,060(31.3%)25,762(26.4%)$2,031(1170–3743)0.002 EDcharges,median(IQR)$2,200(1237–3943)$1,520(873–2783)$2,031(1170–3743) <0.001

1Criticalproceduresincludedbothcurrentproceduralterminology(CPT)andInternationalClassificationofDiseases,9thRev(ICD-9) procedurecodes.VisitswithCPT/ICD-9procedurecodeslisted,Totaln = 367,108;generalEDn = 241,401;pediatricEDn = 73,947. ED,emergencydepartment; CCC,complexchronicconditions; IQR,interquartilerange.

lowerproportionoftransfers(1.3%vs9.4%, P < 0.001).In thecombinedoutcomeofadmissionortransfer,therewere nosignificantdifferencesbetweenPEDsandGEDs(52.2% vs56.4%, P = 0.09).DeathintheEDwasaninfrequent outcome,representingonly0.11%ofvisits.However,ED

mortalitywaslowerforPEDvisitscomparedtoGEDvisits (0.02%vs0.11%, P = 0.003).Visitmortality(deathatany pointduringEDorinpatientstay)wassimilarly lowerforPEDvisitscomparedtoGEDs (1.27%vs2.39%, P = 0.003).

Table1. Selectedpatient,hospital,andvisitcharacteristicsinvolvingvisitstogeneralandpediatricemergencydepartmentsfor neuromuscularcomplexchronicconditions.

Jafarietal. EmergencyCareforChildrenwithChronicNeuroComplaints

Table2 summarizesresultsofourlogisticregression modelstoexploretherelationshipbetweenthecategoryof EDandvisitdispositionandcriticalprocedureperformance. ThePEDvisitshadsignificantlyloweradjustedoddsof transfercomparedtogeneralEDs(aOR0.14,95%CI 0.04–0.56).Conversely,PEDshadasignificantlyhigher adjustedoddsofadmission(aOR1.52,95%CI1.19–1.96). Additionalpredictorsinadmissionandtransfermodels includedpresenceofnon-neurologicCCCs,increased severityofillness,rural/fringemetropolitanresidences,and inthosewhoseinsurancewasself-pay.Forthecombined outcomeofadmissionortransfer,therewasnosignificant differencebetweenPEDsandGEDs(aOR1.07,95% CI0.86–1.33).

Intheadjustedmodelsofcriticalprocedures(Table3), PEDvisitshadloweroddsofcriticalprocedures(aOR0.74, CI0.62–0.87)comparedtoGEDvisits.Overall,increased severityofillnesswasassociatedwithadramatically increasedoddsofcriticalprocedures(aOR11.9,95%CI 10.3–13.6).Non-neurologicCCCswerealsoassociatedwith increasedofcriticalproceduresperformance(aOR1.51,95% CI1.42–1.59).TheEDvisitsinwhichapatienthada tracheostomyhadlowerodds(aOR0.48,95%CI0.38–0.59) ofcriticalprocedureascomparedtothosewithouta tracheostomy;otherformsoftechnologyassistancewerenot significantlydifferent.

ThelogisticmodelsforEDmortalityandvisitmortality arealsoshownin Table3.Severityofillnessscoreswerenot

includedintheadjustedmodelsofmortality,astherewas collinearitywiththisvariableandtheoutcome.Theadjusted oddsofEDmortalitywassignificantlylowerforPEDvisits (aOR0.37,CI0.19–0.73)comparedtoGEDvisits.Patients withventriculorperitonealshuntshadaloweradjustedodds ofmortalitycomparedtothosewithout.Similarly,inthe modelofoverallvisitmortality,PEDvisitshadalowerodds ofvisitmortalitycomparedtoGEDs(aOR0.62, P < 0.001). Thepresenceofnon-neurologicCCCswaspredictiveof increasedoddsofmortality,whileallformsoftechnology assistancehadloweradjustedoddsofvisitmortality.

DISCUSSION

InthisnationalsampleofEDvisits,weestimate387,000 annualEDvisitsweremadebypatientsaged0–21yearswith neurologiccomplexitybetween2011–2014.NeuroCCC patientshadhighratesofmedicalcomplexityandtechnology dependence,andoftenpresentedwithsevereillnesstothe ED.Mostoftheemergencycareforthispopulationoccurred inGEDs,wherevisitshadhigherratesofdiagnostictesting, criticalprocedures,andED-associatedcharges.After adjustmentfordifferencesindemographics,comorbidities, andseverityofillnesspresentation,GEDshadhigherratesof transfer.However,therewerenosignificantdifferences betweenGEDsandPEDsinacombinedmodelofadmission ortransfer.Adjustedoddsofcriticalprocedureperformance, EDmortality,andoverallvisitmortalitywerehigherin GEDscomparedtoPEDs.

–1.00)

Femalegender0.93(0.89–0.97)1.06(1.00–1.13)0.95(0.92–0.99)

Insurancepayer

PrivateinsuranceReferentReferentReferent

Medicaid0.85(0.78–0.93)1.10(0.99–1.23)0.88(0.82–0.95)

Medicare0.88(0.71–1.10)0.84(0.57–1.24)0.86(0.69–1.07)

Urbanicity

CentralmetroReferentReferentReferent

Smallmetro0.74(0.58–0.96)2.33(1.75–3.09)0.96(0.78–1.18)

Non-metro0.79(0.54–0.89)2.98(2.28–3.9)1.01(0.84-1.21)

Additionalnon-neurologicCCCs2.56(2.31–2.86)0.43(0.38–0.49)2.25(2.05–2.47)

Feedingtube1.30(1.17–1.45)0.82(0.69–0.98)1.23(1.1–1.37)

Ventricularshunt1.01(0.89–1.14)0.54(0.46–0.63)0.84(0.76–0.93)

Tracheostomy0.59(0.48–0.71)2.8(2.11–3.70)0.71(0.6–0.85)

Severityclassificationscore2.52(2.40–2.64)1.9(1.75–2.06)3.0(2.86–3.31)

PediatricED1.52(1.19–1.96)0.14(0.04–0.56)1.07(0.86–1.33)

OR,oddsratio; CI,confidenceinterval; CCCs,complexchronicconditions; ED,emergencydepartment.

Table2. Logisticmodelsforoutcomeofadmission,transfer,andthecombinedoutcomeofadmissionortransferforemergencydepartment visitsforneurologiccomplexchronicconditions.

–

Femalegender0.81(0.76–

Insurancepayer

PrivateinsuranceReferentReferentReferent Medicaid1.03(0.96–1.11)1.04(0.65–1.67)0.66(0.58–0.75)

–1.15)1.09(0.15–8.24)0.19(0.09–0.36)

Urbanicity

CentralmetroReferentReferentReferent

Smallmetro0.81(0.70–0.94)1.12(0.54–2.31)0.94(0.74–1.12)

Non-metro0.65(0.56–0.75)2.0(0.89–4.54)0.90(0.72–1.12)

Non-neurologicCCCs1.51(1.42–1.59)0.86(0.55–1.33)2.73(2.54–2.94)

Feedingtube0.92(0.83–1.02)1.34(0.49–3.67)0.23(0.18–0.30)

Ventricularshunt0.91(0.81–1.02)0.34(0.18–0.64)0.33(0.28–0.40)

Tracheostomy0.48(0.38–0.59)2.65(0.77–9.17)0.22(0.15–0.31)

Severityclassificationscore11.81(10.28–13.56)N/AN/A

PediatricED0.74(0.62–0.87)0.38(0.19–0.75)0.62(0.46–0.83)

OR,oddsratio; CI,confidenceinterval; CCCs,complexchronicconditions; ED,emergencydepartment.

Priorresearchhasdemonstratedthatchildrenwith complexchronicillnesseshaveahighriskofcriticalillness andpooroutcomesduringemergencies.17,18,24,25 Ourstudy addstotheliteraturebydemonstratingthatamonga nationalsampleofneuroCCCEDvisits,anestimated30% presentedwithcriticalseverityofillness(SCS5)and7.7% requiredendotrachealintubation.Incontrast,inanational sampleofall-comerPEDvisitsintheNEDSinwhichonly 5%ofpatientshad ≥1complexchroniccondition,only0.5% ofvisitshadacriticalseverityscoreof5andonly0.15% requiredintubation.2 Thecomparativelymuchhigher severitymeasuresfoundinourstudypopulationfurther supportsthehigh-acuityEDneedsamongchildrenwith neurologiccomplexityascomparedtoageneral pediatricpopulation.

Ourworkdemonstratesthatmostemergencycarefor childrenwithneuroCCCsoccursinGEDsratherthan specializedpediatriccenters,congruentwithprior characterizationsofemergencycareforchildrenwithCCCs.2

Priorresearchusingnationallyrepresentativedatasetshas shownthatPEDsmayperformbetteracrossseveralqualityof-caremetrics,includinglessdiagnostictestinginasthma, fewerantibioticsforviralinfections,andlowermortalityin criticalillnessessuchascardiacarrestandsepsis.4,5,25 Our studyexpandsuponthisexistingliterature,bycharacterizing thedisparitiesincharacteristicsofemergencycare inGEDsforchildrenwithchronicneurologicdiseases.

These findingssuggesttheremaybesomespecific benefitstoPEDcareforcertainhigh-risk,medically fragilepopulations,suchasthosewith neurologiccomplexity.

SomeofthedifferencesweobservedbetweenGEDsand PEDsmaybeduetounmeasuredinfluencesofapre-transfer evaluationandstabilization.Asthisdatasethasnopatient identifiersanddoesnotallowforlongitudinalassessmentof patientcare,wewereunabletoidentifywhichEDvisitswere self-referredvstransferredfromanotherED.However, amongthe28,633transferredencountersinthisstudy,27,392 (95.7%)originatedinaGED,anditislikelymanyofthese encountersweretransferredtoaPED.Oncethesepatients reachedthereceivingfacility,theylikelyhadreduced requirementsforadditionaldiagnostictestingorcritical interventions,whichcouldaccountforthecomparatively lowerEDcostsandprocedurefrequencyweobservedinPED encounters.Additionally,otherfactorsrelatedtothetransfer processmayhaveinfluencedprocedureratesinGEDs.For instance,referringemergencyphysiciansmighthavechosen tointubatepatientswithahigherriskofrespiratory decompensationbeforethetransfer,potentiallycontributing totherelativelyhigherintubationfrequencyseeninGEDs. Togainadeeperunderstandingoftheoriginsoftheobserved variationsinEDcostsandoutcomesbetweenGEDsand PEDs,futurestudiesincorporatinglongitudinalpatientdata areneeded.

Table3. Logisticmodelsfortheoutcomesofcriticalprocedureperformance,deathintheemergencydepartmentordeathatanypointinthe visit(visitmortality).

ItisworthnotingthatEDvisitsmadebychildrenwith neuroCCCshadhighratesoftechnologyassistanceoverall, andinourlogisticmodelspatientswithtracheostomyand ventricularshuntshadloweroddsofvisitmortality.

Althoughtechnologyassistanceinlargepopulationstudies ofall-comerpediatricpatientshasbeenidentifiedasarisk factorforsevereillnessandmortality,inchildrenwith neurologicdiseasesthereisevidencethattechnology assistancemaybeprotective.26,27 Ina2019Canadianstudy ofchildrenwithmedicalcomplexity,technologyassistance wasassociatedwithloweroddsofvisitmortalityinchildren withneurologicimpairmentandthosewithmultipleCCCs.8 Similarly,a2015analysisfromHongKongfoundthatin childrenwithsevereneurologicdiseases,tracheostomywas associatedwithloweroddsofmortality.28 Additionally, inherentdifferencesinthetypeofneurologiccomplexity betweenpatientswithandwithouttheseformsoftechnology assistancemayhelpexplaintheobserveddifferences inmortality.

Wehypothesizethehigherratesoftransferandlowerrates ofadmissioninGEDsarelikelysecondarytolimited inpatientpediatriccapabilitiesatthesecenters,thus necessitatingtransfer.6,7,29 Thishypothesisissupportedby our findingthatthecombinedoutcomeofadmission andtransferinourlogisticmodelsshowednodifferences betweenGEDsandPEDs.Thereisincreasingevidence thatpediatricinpatientcareisincreasinglylimitedin communityhospitals,resultinginincreasedregionalization ofhospitalpediatriccare.30 Thisislikelytobeparticularly trueforchildrenwithneurologiccomplexity,whomay requirespecialistconsultationonlyavailablein pediatriccenters.

These findingshaveimportantimplicationsforthe deliveryofpediatricemergencycaretomedicallyvulnerable patientsintheUnitedStates.Despitetheincreasing regionalizationofinpatientpediatriccare,emergencycare forchildrenislikelytocontinuetooccurpredominantlyin GEDsgiventhegeographiclimitationsinaccessto specializedpediatricemergencycentersformanypatients. Thus,ensuringadequateeducationandpreparationfor emergencyconditionsincomplexpediatricpatientsin communityandruralEDsiscritical.Experiencecaringfor criticallyill,medicallycomplexpediatricpatientsislacking formanyEMtraineesandrepresentsatargetforongoing educationalefforts.12,13

Simulationinterventions,suchasthosedeliveredbythe IMPACTSnetwork.areanotherpossibleinterventionto helpimprovethecareofthiscomplexpopulationbynonpediatriccliniciansincommunityEDsettings. 31 Pediatric emergencytelemedicinemaybeanotherpotentialstrategy toimprovethequalityofcarereceivedbycomplex pediatricpatientsinGEDs.Improvementsinthis technology,wideravailabilityoftelemedicineclinicians, andincreasingacceptanceofthisformatofcare

mayultimatelyaddressdisparitiesinaccesstocareby makingspecializedpediat ricemergencyphysicians moreavailable. 32 – 34

LIMITATIONS

Thisstudyhasseveralimportantlimitations.First,we useddatafrom2011–2014,whichmayimpacthow translatablethese findingsaretothepresent.Increasing regionalizationofcareinthelast10yearsmayhaveimpacted overalldistributionofpediatricneuroCCCEDcarebetween GEDsandPEDsandpotentiallyanincreasedfrequencyof transfers.Increasedeffortstowardpediatricreadinessin GEDsduringthistimeframecouldalsohaveimproved criticalillnessoutcomesinsomeGEDs.Additionally,this workreliesonlargeamountsofadministrativedata,whichis susceptibletoerrorsindataprocessingandvariabilityin coding.WeusedICD–9codestoidentifythepopulationof neuroCCCvisits,andtheICD–9codesascribedtoan encounteronlypertaintothecurrentlyrecordedED diagnosesandmaynotrepresentallpre-existingconditions. Thus,thisworklikelyunderestimatesthetruefrequencyof neuroCCCEDvisits,particularlyforloweracuitytreatand-releasevisits.2

Additionally,usingtheproportionofpediatricpatients seenwithinanEDtodeterminePEDdesignationhasitsown limitations.Specifically,ifaPEDandGEDare financially linked(commoninacademicinstitutionsthatsharethesame campus)thevisitsfromthesetwoinstitutionswilloftenbe groupedasasinglehospitalintheNEDS.Thisresultsin somePEDsbeinggroupedtogetherwithGEDs,usingour categorizationsystem.Giventhecollinearityoftheoutcome ofmortalitywithSCS,wedidnotincludethisinour modeling,andthusdifferencesinmortalitybetweenGEDs andPEDsmayinpartbeduetounmeasureddifferencesin severityofillness.Lastly,mortalitywasoverallaninfrequent outcome,andthusanybroadinterpretationsofthis finding shouldbetakenwithcaution.

CONCLUSION

Thisstudyisthe firstinourknowledgetodescribethe nationalstateofemergencycareforchildrenwithneurologic complexchronicconditionsinbothpediatricandgeneral EDs.Our findingsdemonstratethatmostemergencycarefor childrenwithneuroCCCsoccursinGEDs,andthatGEDs hadhigherratesofproceduresandcharges,transfers,and mortalityascomparedtoPEDs.Asthesepatientsarelikely tocontinuetopredominantlyreceiveemergencycarein GEDsettings,interventionstoensureappropriatetraining andpreparationofgeneralemergencyphysiciansforchildren withneurologiccomplexityisneeded.Additionally,further researcheffortstoexploretheimpactofpediatricemergency telemedicinesupportonimprovedqualityofcarefor medicallycomplexpatientsisneeded.

AddressforCorrespondence:KaileenJafari,MD,Universityof Washington,DepartmentofPediatrics,4800SandPointWayNE, M/SMB.7.520,Seattle,WA98105.Email: kaileen.jafari@ seattlechildrens.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Jafarietal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.MooreBJ,StocksC,OwensPL.Trendsinemergencydepartmentvisits, 2006–2014.HCUPStatisticalBrief#227.2017.Availableat:~https:// hcup-us.ahrq.gov/reports/statbriefs/sb227-EmergencyDepartment-Visit-Trends.jsp#:~:text=Overall%2C%20the%20number %20of%20ED,U.S.%20population%20grew%206.9%20percent December1,2020.

2.MurtaghKurowskiE,ByczkowskiT,Grupp-PhelanJM.Comparisonof emergencycaredeliveredtochildrenandyoungadultswithcomplex chronicconditionsbetweenpediatricandgeneralemergency departments. AcadEmergMed. 2014;21(7):778–84.

3.MichelsonKA,LyonsTW,HudginsJD,etal.Useofanationaldatabase toassesspediatricemergencycareacrossUnitedStatesemergency departments. AcadEmergMed. 2018;25(12):1355–64.

4.MichelsonKA,HudginsJD,MonuteauxMC,etal.Cardiacarrestsurvival inpediatricandgeneralemergencydepartments. Pediatrics. 2018;141(2):e20172741.

5.CushingAM,BucholzE,MichelsonKA.Trendsinregionalizationof emergencycareforcommonpediatricconditions. Pediatrics. 2020;145(4):e20192989.

6.BerryJG,PoduriA,BonkowskyJL,etal.Trendsinresourceutilization bychildrenwithneurologicalimpairmentintheUnitedStatesinpatient healthcaresystem:arepeatcross-sectionalstudy. PLoSMed. 2012;9(1):e1001158.

7.CohenE,BerryJG,CamachoX,etal.Patternsandcostsofhealthcare useofchildrenwithmedicalcomplexity. Pediatrics. 2012;130(6):e1463–70.

8.MassinMM,MontesantiJ,GérardP,etal.Childrenwithchronic conditionsinapaediatricemergencydepartment. ActaPaediatr. 2006;95(2):208–13.

9.O’MahonyL,O’MahonyDS,SimonTD,etal.Medicalcomplexityand pediatricemergencydepartmentandinpatientutilization. Pediatrics. 2013;131(2):e559–65.

10.FeudtnerC,LevinJE,SrivastavaR,etal.Howwellcanhospital readmissionbepredictedinacohortofhospitalizedchildren?A retrospective,multicenterstudy. Pediatrics. 2009;123(1):286–93.

11.JainPN,ChoiJ,KatyalC.Pediatriccareinthenonpediatric emergencydepartment:providerperspectives. HospPediatr. 2019;9(3):216–9.

12.LanghanM,KeshavarzR,RichardsonLD.Howcomfortableare emergencyphysicianswithpediatricpatients? JEmergMed. 2004;26(4):465–9.

13.HealthcareCostandUtilizationProject(HCUP).NEDSOverview. Availableat: https://www.hcup-us.ahrq.gov/nedsoverview.jsp AccessedDecember2,2020.

14.FeudtnerC,FeinsteinJA,ZhongW,etal.Pediatriccomplexchronic conditionsclassificationsystemversion2:updatedforICD–10and complexmedicaltechnologydependenceandtransplantation. BMC Pediatr. 2014;14:199.

15.HansenM,FleischmanR,MecklerG,etal.Theassociationbetween hospitaltypeandmortalityamongcriticallyillchildreninUSEDs. Resuscitation. 2013;84(4):488–91.

16.WhitfillT,AuerbachM,ScherzerDJ,etal.Emergencycareforchildren intheUnitedStates:epidemiologyandtrendsovertime. JEmergMed. 2018;55(3):423–34.

17.BourgeoisFTandShannonMW.Emergencycareforchildrenin pediatricandgeneralemergencydepartments. PediatrEmergCare. 2007;23(2):94–102.

18.ChandwaniHS,StrasselsSA,RascatiKL,etal.Estimatesofcharges associatedwithemergencydepartmentandhospitalinpatientcarefor opioidabuse-relatedevents. JPainPalliatCarePharmacother.

2013;27(3):206–13.

19.ChannaR,ZafarSN,CannerJK,etal.Epidemiologyofeyerelatedemergencydepartmentvisits. JAMAOphthalmol.

2016;134(3):312–9.

20.AlessandriniEA,AlpernER,ChamberlainJM,etal.Developinga diagnosis-basedseverityclassificationsystemforusein emergencymedicalservicesforchildren. AcadEmergMed. 2012;19(1):70–8.

21.SotoF,MurphyA,HeatonH.Criticalproceduresinpediatric emergencymedicine. EmergMedClinNorthAm. 2013;31(1):335–76.

22.MittigaMR,GeisGL,KerreyBT,etal.Thespectrumandfrequencyof criticalproceduresperformedinapediatricemergencydepartment: implicationsofaprovider-levelview. AnnEmergMed. 2013;61(3):263–70.

23.AmericanAcademyofPediatrics,CommitteeonPediatricEmergency MedicineandCouncilonClinicalInformationTechnology,American CollegeofEmergencyPhysicians,PediatricEmergencyMedicine Committee.Policystatement–emergencyinformationformsand emergencypreparednessforchildrenwithspecialhealthcareneeds. Pediatrics. 2010;125(4):829–37.

24.BurkeRTandAlversonB.Impactofchildrenwithmedicallycomplex conditions. Pediatrics. 2010;126(4):789–90.

25.AmesSG,DavisBS,MarinJR,etal.Emergencydepartmentpediatric readinessandmortalityincriticallyillchildren. Pediatrics. 2019;144(3):e20190568.

26.HeneghanJA,ReederRW,DeanJM,etal.Characteristics andoutcomesofcriticalillnessinchildrenwithfeedingand respiratorytechnologydependence. PediatrCritCareMed. 2019;20(5):417–25.

27.RussCMandAgusM.Triageofintermediate-carepatientsinpediatric hospitals. HospPediatr. 2015;5(10):542–7.

28.LamKFandLamP.Tracheostomyinchildrenwithsevereneurological impairment:asinglecentrereview. JPaediatrRespirolCritCare. 2015;11(4):4–11.

29.FrançaULandMcManusML.Trendsinregionalizationofhospitalcare forcommonpediatricconditions. Pediatrics. 2018;141(1):e20171940.

30.MichelsonKA,HudginsJD,LyonsTW,etal.Trendsincapabilityof hospitalstoprovidedefinitiveacutecareforchildren:2008to2016. Pediatrics. 2020;145(1):e20192203.

31.AbulebdaK,WhitfillT,MontgomeryEE,etal.Improvingpediatric readinessingeneralemergencydepartments:aprospective interventionalstudy. JPediatr. 2021;230:230–7.e1.

32.GattuR,TeshomeG,LichensteinR.Telemedicineapplicationsforthe pediatricemergencymedicine:areviewofthecurrentliterature. Pediatr EmergCare. 2016;32(2):123–30.

33.McConnochieKM,WoodNE,HerendeenNE,etal.Acuteillnesscare patternschangewithuseoftelemedicine. Pediatrics. 2009;123(6):e989–95.

34.DayalP,HojmanNM,KisseeJL,etal.Impactoftelemedicineonseverity ofillnessandoutcomesamongchildrentransferredfromreferring emergencydepartmentstoachildren'shospitalPICU. PediatrCritCare Med. 2016;17(6):516–21.

PediatricOutcomesofEmergencyMedicalServices

Non-TransportBeforeandDuringtheCOVID-19Pandemic

LoriPandya,MD*

BrandonMorshedi,MD,DPT,NREMT-P,FP-C†

BrianMiller,MD‡

HalimHennes,MD*

MohamedBadawy,MD*

*UniversityofTexasSouthwesternMedicalCenter,Department ofPediatrics,DivisionofEmergencyMedicine,Dallas,Texas

† UniversityofArkansasforMedicalSciences,Deprtmentof EmergencyMedicine,LittleRock,Arkansas

‡ UniversityofTexasSouthwesternMedicalCenter,Department ofEmergencyMedicine,Dallas,Texas

SectionEditors:JabeenFayyaz,MD,MCPS,MHPE,PhD,IHP,andYaninaPurim-Shem-Tov,MD,MS

Submissionhistory:SubmittedJuly24,2023;RevisionreceivedNovember22,2023;AcceptedNovember27,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18408

Introduction: Pediatricpatientsaccountfor6–10%ofemergencymedicalservices(EMS)activationsin theUnitedStates.Approximately30%ofthesechildrenarenottransportedtoanemergencydepartment (ED).Adultdataintheliteraturereportshigherhospitalizationandcomplicationsfollowingnon-transport. Fewstudiesdiscussepidemiologyandcharacteristicsofpediatricnon-transport;however,dataon outcomeislimited.Ourprimaryaimwastodetermineoutcomesofnon-transportedchildrenwithinour urbanEMSsystembeforeandduringtheCOVID-19pandemic.Oursecondaryobjectivewastoexplore reasonsfornon-transport.

Methods: Thiswasaprospective,descriptivepilotstudy.WecomparedEMSdataforSeptember2019 (pre-COVID-19)toSeptember2020(pandemic).Includedwerechildrenaged0–17yearswhoactivated EMSanddidnotreceivetransporttotheprimaryhospitalfortheEMScapturearea.Wedefined outcomesasrepeatEMSactivation,EDvisits,andhospitaladmissions,allwithin72hours.Datawas obtainedviaelectroniccapture.Weuseddescriptivestatisticstoanalyzeourdata,chisquarefor categoricaldata,stepwiselogisticregression,andunivariatelogisticregressiontotestforassociationof covariateswithnon-transport.

Results: Therewere1,089pediatricEMSactivationsinSeptember2019and780inSeptember2020. Non-transportoccurredin633(58%)inSeptember2019and412(53%)inSeptember2020.Emergency medicalserviceswasreactivatedwithin72hoursinthefollowing:9/633(1.4%)in2019;and5/412(1.2%) in2020(P = 0.77).VisitstotheEDoccurredin57/633(9%)in2019and42/412(10%)in2020(P = 0.53). Hospitaladmissionsoccurredin10/633(1.5%)in2019and4/412(0.97%)in2020(P = 0.19).One non-transportedpatientwasadmittedtotheintensivecareunitinSeptember2020(<1%)andsurvived. Hispanicethnicity,age >12years,andfeverwereassociatedwithrepeatEMSactivation.Themost commonreasonfornon-transportinbothstudyperiodswasthattheparentfeltanambulance wasnotnecessary(47%).

Conclusion: Inoursystem,non-transportofpediatricpatientsoccurredin >50%ofEMSactivationswith nosignificantadverseoutcome.Age >12years,fever,andHispanicethnicityweremorecommonin repeatedEMSactivations.Themostcommonreasonfornon-transportwasparentsfeelingitwasnot necessary.FuturestudiesareneededtodevelopreliableEMSguidelinesforpediatricnon-transport. [WestJEmergMed.2024;25(2)246–253.]

INTRODUCTION

Pediatricpatientshistoricallyaccountforupto10%of emergencymedicalservices(EMS)activationsintheUnited States,1 withmorerecentliteraturesuggesting6%.2 A nationalEMSdatareviewnotedthat30%ofpediatric patientsarenottransportedtoamedicalfacilityforfurther evaluationandcare.2–6 Thereasonsfornon-transportare broad,includingfactorssuchasparentalrefusalandtypeof complaint(ie,musculoskeletaltrauma,respiratoryillness). Whiledataexistsregardingratesofandfactorsrelatedto pediatricEMSnon-transport,outcomesarelimitedtoafew studies.Intheadultliterature,non-transportwasassociated witha16%hospitalizationrate7 andinsomecasesseriousor fataloutcomes.8 Onepediatricstudynotednon-transported patients <3yearsofagewere1.3timesmorelikelytohavea subsequentemergencydepartment(ED)visit,9 whileanother reporteda10%hospitalizationrateafterpediatricnontransportforparentalrefusal.10 DuringtheCOVID-19 pandemic,datasuggeststhatEMScallvolumesandnontransportrateschanged,withadeclineinoverallEMS responsevolumesandanincreaseintherateofnontransports.11 Littleisknownaboutwhetherthisimpacted outcomesforchildrenwhowerenon-transported.

Ourprimaryobjectivewastodeterminepediatric outcomesofnon-transportwithinourlargeEMSsystem beforeandduringtheCOVID-19pandemic.Outcomeswere definedasrepeatEMSactivation,in-personEDvisits,and/or hospitaladmissions,allwithin72hoursofinitialEMS activation.Wealsoaimedtodescribedemographicfactors associatedwithsubsequentlyneedingmedicalattentionafter EMSnon-transport.Asecondaryobjectivewastoidentify reasonsfornon-transportwithinoursystembothpre-and duringtheCOVID-19pandemic.Wechosetocomparepreandduringthepandemictodeterminewhethertherewasa changeinutilizationorinEMSclinicians’/parents’ behavior duringapandemictobetterprepareoursystemsfor thefuture.

MATERIALSANDMETHODS

StudySetting

Thiswasaprospective,descriptivepilotstudyatalarge, urban, fire-basedEMSsystemintheCityofDallas,Texas, with59stationsand~1,800EMSrespondersservingatotal populationof1.3million,withapproximately25%ofthat population <18yearsofage.Thestudywasapprovedbythe institutionalreviewboard.

InclusionofPatients

Weincludedchildrenaged0–17yearswithEMS activationwhodidnotreceiveEMStransportduringthe studyperiod.Weselectedtwoone-monthtimeperiods, September2019(pre-COVID-19)andSeptember2020 (COVID-19pandemic).Duringthestudyperiod,allnontransportsofpediatricpatientswere,perprotocol,required

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Upto30%ofpediatricEMSactivationsare nottransportedtoanED.Adultdatareports adverseoutcomesfollowingnon-transport; pediatricdataislimited.

Whatwastheresearchquestion?

WeaimedtodetermineoutcomesofnontransportedchildrenwithinourEMSsystem beforeandduringCOVID-19.

Whatwasthemajor findingofthestudy?

Therewasnodifferenceinoutcomespre/ duringCOVID-19:EMSreactivation (1.3%ofallpatients)(P = 0.77);EDvisits (P = 0.53);andadmission(P = 0.19).

Howdoesthisimprovepopulationhealth?

Futurestudiesareneededtodevelopreliable guidelinesforpediatricnon-transport,which coulddecreaseburdenonthemedicalsystem especiallyduringpandemics.

tohaveonlinemedicalcontrol(OLMC)consultationand/or audiorecording.Audio-recordedrefusalwasobtainedvia handheldtabletusingastandardizedscript.AnyEMSinitiatednon-transportswerenotallowedinthesystem,and allnon-transportswereinitiatedbytheparentorguardian.

DataAcquisition

WeobtainedandcomparedEMSdatathrough comprehensivemanualreviewoftheprehospitalelectronic healthrecord(EHR)fromadailyautomatedreportofthe twoperiods.TheEMSrecordswereelectronicallymatched usingnameanddateofbirth(DOB)forrepeatEMS activationwithin72hours.Atourpediatrichospitalhealth system,whichistheprimarytertiarycarechildren’shospital fortheEMScapturearea,wequeriedtheEHRforEDvisits andhospitaladmissionswithin72hoursofEMSactivation usingthesamenameandDOB.Ifconcernsaroseforaname spellingerror,weusedDOBandaddresstoconfirman identitymatch.Demographicdata,chief/dispatch complaint,EMSvitals,non-transportvolume,andnontransportreasonweremanuallyabstractedfromourEMS electronicpatientcaredatabase/automatedreport(byeither theprincipalinvestigatorPIorasingle,trainedresearch assistant[RA]).Race/ethnicitywasEMSidentifiedusinga

Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID

drop-downmenuintheelectronicpatientcarerecord;the categoriesareperNEMSIS(NationalEMSInformation System).BecausepriorversionsofNEMSIScombinedrace andethnicitytherearenotseparate fields.In-personED visitsandhospitaladmissions(includinginpatient observationandintensivecareunit[ICU]admission)within 72hoursofEMSactivationand finaldisposition(discharge vsdeath)weremanuallyabstractedfromthehospitalhealth systemEHR(byeitherthePIorasingletrainedRA).This includedhospitalpresentationsafterrefusalthatcameby repeatEMSactivationandothermeans(eg,privatevehicle).

Outcomes

WedefinedprimaryoutcomesasrepeatEMSactivation, in-personEDvisits,and/orhospitaladmissions,allwithin 72hoursofinitialEMSactivation.Weusedthe72-hour follow-upwindowbasedonotherpublishedpapersinthis area. 12–16 PediatricEMSprotocolsdidnotchangebetween thesetwostudyperiods.Thepopulationwasstratifiedbyage group(similartopreviouslypublishedstudies2,17,18)and chiefcomplainttodeterminewhethertherewasahigher proportionofnon-transportbasedonageandthemost commonnon-transportdiagnosis.WeclassifiedEMSchief complaint/diagnosisintothefollowingcategories:fever; gastrointestinal;respiratory;trauma;neurological;pain; mentalhealth;andother.Reasonforparentalrefusalof transportwasdescribed(EMSdocumented).

Analysis

Weanalyzedcategoricaldatausingthechi-squaredtest. TheFisherexacttestwasusedforsmallersamplesizes(ie, hospitalandICUadmissiondata).Weusedthe t -testand Wilcoxonrank-sumtestforEMSvitalsigns.InitialEMS vitalsignsoftemperature,heartrate,respiratoryrateand

oxygensaturationwereabstractedforeachsubjectand definedasabnormalbasedonnormalage-relatedranges withinthePediatricAdvancedLifeSupportguidelines.19 We madecorrectionformultipletestingandusedonlycaseswith completedatainthe finalanalysis.Covariatesforanalysis wereaprioribasedonpreviousliterature.Toidentify covariates’ associationwiththeoutcomeweperformeda stepwiselogisticregression.However,theanalysisidentified onlyonecovariate,andweusedaunivariatelogistic regressiontotestforassociationofthatcovariatewithinnontransportoutcomes.Wedidnotcalculateapredetermined samplesize,asthiswasapilotstudy.Resultsarepresentedas oddsratioswith95%confidenceintervals(CI),taking P -valuesof <0.05assignificant.Statisticalanalyseswere performedusingSASforWindowsrelease9.4 (SASInstitute,Inc.,Cary,NC).

RESULTS

AnnualpediatricEMSvolumeswere12,663(2019)and 10,429(2020).Therewere1,089pediatricEMSactivationsin September2019vs780activationsinSeptember2020 (Figure).Non-transportoccurredin633(58%)activationsin September2019vs412(53%)inSeptember2020(Table1). PerourEMSprotocol,weobtainedOLMCand/oraudio recordingin84%ofnon-transports.Demographicsarelisted in Table2

Table1. Volumeofnon-transportedpediatricpatientsbefore(2019) andduring(2020)theCOVID-19pandemic.

Sept2019Sept2020 EMSactivations1089780 Non-transport(%)633(58)412(53)

Figure. STROBEdiagramillustratingpatientinclusion. EMS,emergencymedicalservices.

Pandyaetal. PediatricOutcomesofEMSNon-TransportBeforeandDuringCOVID

2019(N = 633)2020(N = 412)All(N = 1,045) P-value

PatientAge(mos)113(38.0–182.0)111.5(33.4–181.5)

PatientAge(yrs) 0.9

0–2121(19.2%)76(18.6%)197(19.0%)

2–5107(17.0%)63(15.4%)170(16.4%)

5–12158(25.1%)105(25.7%)263(25.3%)

>=12244(38.7%)164(40.2%)408(39.3%)

Gender 0.7

Female312(49.5%)208(50.6%)520(50.0%)

Male318(50.5%)203(49.4%)521(50.0%)

Race 0.7

BlackorAfricanAmerican337(53.9%)207(50.9%)544(52.7%)

White63(10.1%)38(9.3%)101(9.8%)

HispanicorLatino205(32.8%)149(36.9%)354(34.3%)

Other20(3.2%)13(3.2%)33(3.2%)

Patientagenotedasmeanwithinterquartilerange.

PrimaryOutcomes

DuringSeptember2019EMSwasreactivatedwithin72 hoursin9/633(1.4%)activations,EDvisitsoccurredwithin 72hoursin57/633cases(9%),andhospitaladmissions occurredin10/633(1.5%).DuringSeptember2020,EMS wasreactivatedwithin72hoursin5/412(1.2%)activations, EDvisitsoccurredwithin72hoursin42/412cases(10%),and hospitaladmissionsoccurredin4/412(1%).OnenontransportedpatientwassubsequentlyadmittedtotheICUin September2020(<1%)andsurvivedtodischarge.Therewere nostatisticaldifferencesinoutcomesofnon-transportpreandduringthepandemic(Table3).

SecondaryOutcomes

Wefurtheranalyzednon-transportoutcomesin September2020todeterminewhethertherewasahigher proportionofnon-transportrelatedtospecificvariables (gender,race/ethnicity,age,EMSdiagnosis,andvitalsigns). Ourpercentageofmissingvariablesrangedfrom1–15%; however,intheanalysisweusedonlycaseswithcomplete data.InthosechildrenwhohadrepeatEMSactivation

within72hours,Hispanicethnicity,age >12years,andfever onEMSvitalswerestatisticallysignificantfactorsforrepeat activations.Therewasnodifferenceingender,EMS diagnosis,heartrate,respiratoryrate,oroxygensaturation (Table4).ForthosechildrenwithanEMSreactivation resultingintransporttotheED,adiagnosisoftrauma, Hispanicethnicity,age >12years,andfeverweresignificant (Table5).OfthosechildrenwithanEDvisitwithin72hours ofEMSnon-transport,malegenderwastheonlysignificant variable.Therewasnodifferenceinrace/ethnicity,age, diagnosis,orvitalsigns,includingtemperature(Table6).

Race/Ethnicity

Inourlarge,urbancountyin2020,Blackresidentsmade up22.8%ofthetotalpopulation20 andaccountedfor approximately53%ofallpediatricEMSactivationsduring ourstudymonth.Ofallnon-transportedchildreninour studymonth50%wereBlack.Hispanic/Latinoaccountedfor 41%ofthetotalpopulationand34%ofallpediatric activations;36%ofpediatricnon-transportswereidentified asHispanic/Latino.Inoururbancounty,27%ofthetotal

Outcomesofnon-transport(Within72hrs)2019(N = 633)2020(N = 412)All(N = 1,045) P-value RepeatEMSactivation9(1.4%)5(1.2%)14(1.3%)0.8 TransporttoEDonrepeatactivation9(1.4%)4(1.0%)13(1.2%)0.5 EDvisit57(9.0%)42(10.2%)99(9.5%)0.5 Inpatienthospitaladmission10(1.6%)4(1%)14(1.3%)0.2 ICUduringhospitaladmission0(0.0)1(0.2)1(0.1)0.4

Table2. Demographicsofnon-transportedpediatricpatientsbefore(2019)andduring(2020)theCOVID-19pandemic.

Table3. Outcomesofnon-transportedpediatricpatientsbefore(2019)andduring(2020)theCOVID-19pandemic.

populationidentifiedasWhiteandmadeup10%percentof totalpediatricactivations.Ofthosenon-transportedchildren 9%wereWhite.

ReasonsforNon-transport

Inthepre-pandemicperiod(September2019),thereason fornon-transportwas filedfor354(55%)ofactivationsas follows:parentfeltambulancenotnecessary(47.7%);chief complaintresolved(24.9%);transportbyprivatevehicle (20%);andother(3.1%).InSeptember2020,thereasonfor non-transportwasdocumentedin207(49%)cases,withthe mostcommonreasonbeingparentfeltambulancewasnot necessary(58%);followedbytransportbyprivatevehicle (22.2%);chiefcomplaintresolved(15.5%);andother(4.8%).

DISCUSSION

Wefoundourratesofpediatricnon-transport(both pre-andduringpandemic)tobehigherthanthepreviously reported16.3%–30.1%.2–6 Despitethehigherrateof non-transport,ourpediatricoutcomeswerefavorable.The EMSreactivationandhospitaladmissionsoccurredinless

than1.5%ofthosechildrennottransportedtoahealthcare facility.Duringourselectedmonthinthepandemic,onlyone patient(<1%)requiredICUcareandsurvivedtohospital discharge.VisitstotheEDwithin72hoursoccurredin approximately10%ofchildrennottransported;furtherstudy isneededtoevaluatethissubsetofpatients.

ArecentpublishedstudyfromtheUnitedKingdom showedasimilarrateofpediatricEMSreactivation(2%) afterambulancenon-transport.SubsequentEDvisitswere higherthaninour findings(upto24%),andhospital admissionswerealsohigher(ashighas6%comparedtoour 1.5%).Asinourstudy,nodeathsoccurredinpediatricnontransport.16 Anotherstudyshowedapproximately14%ED visitsafternon-transport, <1%hospitaladmission,andagain nodeaths.21 AScandinavianstudyreported17.4%ofnontransportedchildrenvisitedtheED,althoughthiswaswithin 96hourscomparedtoour72-hourtimeframe.Twopatients wereadmittedtotheICU(comparedtooneinourstudy), andagainnodeathsoccurred.22

Allprimaryoutcomeswerenotsignificantlydifferent whencomparedtopre-pandemicdata.Ofnote,weusedthe

Table4. Non-transportedoutcomeduringtheCOVID-19pandemic-repeatEMSactivation.

Temp

72-hourfollow-upwindowbasedonotherpublishedpapers inthisarea,12–16 whileacknowledgingthebalancebetweena longerwindowcatchingmorecasesbutincreasingtherisk thatthosearenotrelatedtotheindexvisit.

Themajorityofnon-transportedchildrenwereBlack (50%);however,thiswasexpectedbasedonour demographics(themajorityofallpediatricEMSactivations duringourstudymonthwereBlack).Similarly,Hispanic/ Latinochildrenaccountedfor34%ofpediatricEMS activationsand36%ofnon-transports.This findingdiffers frompriorstudiesthatshowalowerrateofnon-transportfor Black3 andHispanic6 children.Ourstudyissimilartoa recent,largenationalstudybyWardetal,whichshowedno associationofrace/ethnicitywithnon-transport.2

Althoughwefoundnoassociationwithrace/ethnicityfor non-transport,Hispanicchildreninourstudyweremore likelytohaverepeatEMSactivationswithin72hours.Age >12yearsoldanddocumentedfeverwerealsoassociated withrepeatEMSactivations.Thisageassociationwith repeatactivationsmaybeduetoaloweroverallrateof non-transportinyoungerkids,bothinourstudyandothers6

andthepostulatedlackofEMSresponders’ comfortlevel assessingyoungchildren.14 Wealsofoundthatchief complaint/diagnosiswasnotsignificantlyrelatedtoEMS non-transportduringthepandemic,althoughchildrenwith traumawerenotsurprisinglytransportedmoreoftentothe EDifEMSwasreactivatedwithin72hours.Interestingly, EMSvitals(exceptfever)didnotseemtoplayaroleinour primaryoutcomes.

Inourstudyweobservednosignificantdifferenceinthe percentageoroutcomesofpediatricnon-transportduringthe COVID-19pandemiccomparedtopre-pandemic.Itis importanttonotethatEMSprotocolsdidnotchange betweenthesetwostudyperiods.WhilemanyEMSagencies adoptedmorepermissive “non-transport” policiesin anticipationofhigherEMScallvolumesand9-1-1overuse forminor, flu-likeillnesssymptoms,oursystemdidnotadopt anysuchpolicy;thus,itisatruercomparison.

Ourreasonsfornon-transportaresimilartothose previouslyreportedintheliterature.5–7,23 Duringthe pandemic,therewasapproximatelya10%increasein “parentsfeelanambulanceisnotnecessary.” Itisunclear

Table5. Non-transportedoutcomeduringtheCOVID-19pandemic-transportedtoEDonrepeatEMSactivation.

whetherthiswasdirectlyrelatedtothepandemicandfearof COVID-19exposureortomissingdata.

LIMITATIONS

Therearecertainlimitationsofthisstudy.Althoughwe aretheprimarychildren’shospitalandurgentcarewithinthe jurisdictionservedbytheEMSsystem,therewasthe potentialtomissrepeatEDvisitsatanon-affiliatedadult ED/urgentcare.Futurestudieswillincludeaphonecall follow-upwiththepatient/family.Weselectedasingle month,duetoourhighvolumes,forthispilotstudy, assumingitwouldberepresentativeofothermonths.Data wasobtainedthroughmanualreviewofprehospital electronicpatientcarerecordsobtainedfromanautomated report,resultinginsomeoccasionalincompletedata. Hospitalrecords(EDandinpatient)werematchedusing nameandDOB,potentiallymissingsubjectsiftherewasan errorinnamespellingorprovidedDOB.Ifconcernsarose foramismatch,theprovidedaddresswasusedtoconfirman identity,butthisdatawasnotalwaysavailable. Furthermore,theEMSsystem’sclinicalpractice

guidelines(protocols)requireconsultationwithonline medicalcontrolforpatients <18yearsoldandforspecific conditionsandvitalsignparameters.Inthisstudywedidnot examinetheproportionofnon-transportedpatientswith onlinemedicalcontrolactuallycontacted.Itisnotknown whetherthisinfluencedthesafetyofnon-transports.Lastly, reasonfornon-transportwasmissinginupto50%ofdata, andthereasonwasasdocumentedbytheEMSclinician.

CONCLUSION

Inoursystem,non-transportofpediatricpatients occurredinover50%ofEMSactivationwithnosignificant adverseoutcome.Themostcommonreasonfornontransportwasparentsfeelingitwasnotnecessary.Age >12years,presenceoffever,andHispanicethnicitywere morecommoninrepeatedEMSactivations.Chief complaint/diagnosisdidnotseemtoplayaroleinrepeat EMSactivationsorsubsequentEDvisitsafternontransport.Weobservednosignificantdifferenceinthe percentageoroutcomesofpediatricnon-transportduringthe COVID-19pandemiccomparedtopre-pandemic.

Table6. Non-transportedoutcomeduringtheCOVID-19pandemic-EDvisitwithin72hours.

AdditionalstudiesareneededtodevelopreliableEMS guidelinesforpediatricnon-transport.

AddressforCorrespondence:LoriPandya,MD,Children’s HealthcareofAtlantaatScottishRite,DivisionofEmergency Medicine,1001JohnsonFerryRoad,Atlanta,GA30342.

E-mail: lori.pandya@pemaweb.com

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Pandyaetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.DiggsLA,Sheth-ChandraM,DeLeoG.Epidemiologyofpediatric prehospitalBasicLifeSupportcareintheUnitedStates. PrehospEmerg Care.2016;20(2):230–8.

2.WardC,ZhangA,BrownK,etal.NationalcharacteristicsofnontransportedchildrenbyemergencymedicalservicesintheUnited States. PrehospEmergCare.2022;26(4):537–46.

3.RamgopalS,Owusu-AnsahS,Martin-GillC.Factorsassociatedwith pediatricnontransportinalargeemergencymedicalservicessystem. AcadEmergMed.2018;25(12):1433–41.

4.RichardJ,OsmondMH,NesbittL,etal.Managementandoutcomesof pediatricpatientstransportedbyemergencymedicalservicesina Canadianprehospitalsystem. CJEM,2006;8(1):6–12.

5.KannikeswaranN,MahajanPV,DunneRB,etal.Epidemiologyof pediatrictransportsandnon-transportsinanurbanemergencymedical servicessystem. PrehospEmergCare.2007;11(4):403–7.

6.GerlacherGR,SirbaughPE,MaciasCG.Prehospitalevaluationofnontransportedpediatricpatientsbyalargeemergencymedicalservices system. PediatrEmergCare.2001;17(6):421–4.

7.ConeDC,KimDT,DavidsonSJ.Patient-initiatedrefusalsofprehospital care:ambulancecallreportdocumentation,patientoutcome,andonlinemedicalcommand. PrehospDisasterMed.1995;10(1):3–9.

8.ZachariahB,BryanD,PepeP,etal.Follow-upandoutcomeofpatients whodeclineoraredeniedtransportbyEMS. PrehospDisasterMed. 1992;7(4):359–64.

9.KnightS,OlsonLM,CookLJ,etal.Againstalladvice:ananalysisofoutof-hospitalrefusalsofcare. AnnEmergMed.2003;42(5):689–96.

10.SeltzerAG,VilkeGM,ChanTC,etal.Outcomestudyofminorsafter parentalrefusalofparamedictransport. PrehospEmergCare 2001;5(3):278–83.

11.SattyT,RamgopalS,ElmerJ,etal.EMSresponsesandnon-transports duringtheCOVID-19pandemic. AmJEmergMed.2021;42:1–8.

12.JoyT,BossonN,ChangA,etal.86short-termoutcomesandpatient perceptionsafterEMSnon-transportduringtheCOVID-19pandemic. AnnEmergMed.2022;80(4):S43–4.

13.GloberN,HamiltonJ,MontelauroN,etal.Safetyofanalternativecare protocolforEMSnon-transportintheCOVID-19pandemic. Prehosp EmergCare.2023;27(3):315–20.

14.HainesCJ,LutesRE,BlaserM,etal.Paramedicinitiatednon-transport ofpediatricpatients. PrehospEmergCare.2006;10(2):213–9.

15.SinclairJE,AustinM,FroatsM,etal.Characteristics,prehospital management,andoutcomesinpatientsassessedforhypoglycemia: repeataccesstoprehospitaloremergencycare. PrehospEmergCare 2019;23(3):364–76.

16.CosterJ,O’CathainA,JacquesR,etal.Outcomesforpatientswho contacttheemergencyambulanceserviceandarenottransportedto theemergencydepartment:adatalinkagestudy. PrehospEmergCare 2019;23(4):566–77.

17.LernerEB,DayanPS,BrownK,etal.Characteristicsofthepediatric patientstreatedbythePediatricEmergencyCareAppliedResearch Network’saffiliatedEMSagencies. PrehospEmergCare 2014;18(1):52–9.

18.HewesHA,DaiM,MannNC,etal.Prehospitalpainmanagement: disparitybyageandrace. PrehospEmergCare.2018;22(2):189–97.

19.TopjianAA,RaymondTT,AtkinsD,etal.Part4:PediatricBasicand AdvancedLifeSupport:2020AmericanHeartAssociationGuidelines forCardiopulmonaryResuscitationandEmergencyCardiovascular Care. Circulation.2020;142(16_suppl_2):S469–523.

20.USAFactsOrganization.OurChangingPopulation:Dallas,County, Texas.Availableat: https://usafacts.org/data/topics/people-society/ population-and-demographics/our-changing-population/state/texas/ county/dallas-county?endDate=2020-01-01&startDate=2019-01-01 AccessedJanuary12,2023.

21.KahaléJ,OsmondMH,NesbittL,etal.Whatarethecharacteristicsand outcomesofnontransportedpediatricpatients? PrehospEmergCare 2006;10(1):28–34.

22.OulasvirtaJ,SalmiH,KuismaM,etal.Outcomesinchildrenevaluated butnottransportedbyambulancepersonnel:retrospectivecohortstudy. BMJPaediatrOpen.2019;3(1):e000523.

23.HartkaTandVacaFE.FactorsassociatedwithEMStransportdecisions forpediatricpatientsaftermotorvehiclecollisions. TrafficInjPrev 2020;21(sup1):S60–5.

EvaluationofPeer-reviewQuality:APilotStudy

JeffreyN.Love,MD,MHPE,MSc*

AnneM.Messman,MD,MHPE†

JonathanS.Ilgen,MD,PhD‡ ChrisMerritt,MD,MPH,MHPE§∥ WendyC.Coates,MD¶ DouglasS.Ander,MD# DavidP.Way,MEd**

*GeorgetownUniversitySchoolofMedicine,DepartmentofEmergency Medicine,Washington,DistrictofColumbia

† WayneStateUniversity,DepartmentofEmergencyMedicine, Detroit,Michigan

‡ UniversityofWashington,DepartmentofEmergencyMedicine, Seattle,Washington

§ AlpertMedicalSchoolofBrownUniversity,DepartmentofEmergency Medicine,Providence,RhodeIsland

∥ AlpertMedicalSchoolofBrownUniversity,DepartmentofPediatrics& EmergencyMedicine,Providence,RhodeIsland

¶ DavidGeffenSchoolofMedicineatUCLA,DepartmentofEmergency Medicine,LosAngeles,California

# EmoryUniversity,DepartmentofEmergencyMedicine,Atlanta,Georgia

**OhioStateUniversityCollegeofMedicine,DepartmentofEmergency Medicine,Columbus,Ohio

SectionEditor:MarkLangdorf,MD,MHPE

Submissionhistory:SubmittedAugust16,2023;RevisionreceivedDecember1,2023;AcceptedDecember7,2023

ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18432

Introduction: Despitetheimportanceofpeerreviewtopublications,thereisnogenerallyaccepted approachforeditorialevaluationofapeerreview’svaluetoajournaleditor’sdecision-making.The graduatemedicaleducationeditorsofthe WesternJournalofEmergencyMedicine SpecialIssuein EducationalResearch&Practice(SpecialIssue)developedandstudiedtheholisticeditor’sscoring rubric(HESR)withtheobjectiveofassessingthequalityofareviewandanemphasisonthedegreeto whichitinformsaholisticappreciationforthesubmissionunderconsideration.

Methods: Usingpeer-reviewguidelinesfromseveraljournals,theSpecialIssue’seditorsformulatedthe rubricasdescriptionsofpeerreviewsofvaryingdegreeofqualityfromtheidealtotheunacceptable. Onceareviewwasassessedbyeacheditorusingtherubric,thescorewassubmittedtoathirdpartyfor blindingpurposes.Wecomparedtheperformanceofthenewrubrictoapreviouslyusedsemantic differentialscaleinstrument.Kane’svalidityframeworkguidedtheevaluationofthenewscoringrubric aroundthreebasicassumptions:improveddistributionofscores;relativeconsistencyratherthan absoluteinter-raterreliabilityacrosseditors;andstatisticalevidencethateditorsvaluedpeerreviewsthat contributedmosttotheirdecision-making.

Results: Ninetypeerreviewswerethesubjectofthisstudy,allwereassessedbytwoeditors.Compared tothehighlyskeweddistributionofthepriorratingscale,thedistributionofthenewscoringrubricwasbell shapedanddemonstratedfulluseoftherubricscale.Absoluteagreementbetweeneditorswaslowto moderate,whilerelativeconsistencybetweeneditor’srubricratingswashigh.Finally,weshowed thatrecommendationsofhigherratedpeerreviewsweremorelikelytoconcurwiththeeditor’s formaldecision.

Conclusion: EarlyevidenceregardingtheHESRsupportstheuseofthisinstrumentindeterminingthe qualityofpeerreviewsaswellasitsrelativeimportanceininformingeditorialdecision-making.[WestJ EmergMed.2024;25(2)254–263.]

BACKGROUND

Peerreviewplaysacriticalroleinthetraditionalparadigm ofpublishedscholarship.Whilepeerreviewisthestandard forassessingscholarlysubmissionforpublication,themost appropriatemeansbywhichtoassessthequalityofpeer reviewremainsunclear.1–8 Thisissueisproblematicforall thestakeholdersofpublishedscholarship.Thedevelopment ofarigorousandvalidtoolforeditorstoassessthequalityof peerreviewscouldhelptoenhancethepeer-reviewprocess. Thiswouldimproveeditors’ abilitiestostratifythe contributionsoftheirreviewerpool,identifyingreviewers whodeserveoutstandingrecognitionaswellasthosewho couldbenefitfromdedicatedmentorship,andinform mechanismstoevaluatethedownstreamimpact ofinterventionstoimprovethequalityof peerreviews.

Effortstoassesspeer-reviewqualityhavebeen challenging.Priorstudieshavebeenbasedprimarilyonthe beliefthatreviewevaluationisanobjectiveprocess.9–12 Consequently,interventionshavebeenaimedatachieving ahighdegreeofabsolutereliabilityinscoringbetween editors.Theresultshavedemonstratedamodestdegreeof inter-raterreliability.9–12

Theinter-raterreliabilityofevaluationsofperformance byexpertsisconfoundedbyidiosyncraticperceptionsthat areshapedbyindividualexperiences,values,andpriorities. Indeed,thepreponderanceoftheliteraturearguesthat evaluationbyexpertsisoftensubjectiveand nuanced.2,4–6,8,10,13–17 Coleetalproposedthatthepotential divergentperspectivesamongpeerreviewersareoftenthe resultof “realandlegitimatedif ferencesofopinionamong expertsaboutwhatgoodscienceis, ”14 aconceptsupported byothers.15–17 Capturingthenuancedandpotentially divergentperspectivesofreviewersallowseditorstodevelop aholisticunderstandingofthevalueofamanuscript.15 This variabilityamongeditors’ perspectiveslimitsthedegree ofreliabilitythatcanbeachievedinassessing individualreviews.

TheSpecialIssue’seditorialevaluationofreviewshas traditionallydependeduponasingle,global five-pointscale withanchorsattheextremes(5 = highquality,1 = low quality).Anumberofissueshavebeenappreciatedbythe editorswiththisapproach:1)Thewebsitetemplateonly allowedforasingleeditor’sevaluationofareview;2)scores of1and2wereseldomused;and3)noguidancewas providedforeditorstodeterminehowtoscoreonthe fivepointscale,resultinginalackofvalidevaluationdataon whichtobasedecisionspertainingtothequalityofreviews.

Ourobjectiveinthisinitiativewastodevelopandstudya scoringrubricforeditorstoassessthequalityofareviewwith anemphasisonthedegreetowhichitinformsaholistic appreciationforthesubmissionunderconsideration.Herein wedescribethedevelopment,refinement,andpilot-testingof thisrubric.Additionally,ourreportingwasgroundedwithin

thevalidityevidenceframeworksuggestedbyKaneto informtheinterpretationsofscoresgeneratedby thistool.18

METHODS

HolisticEditor’sScoringRubricDevelopment

Thisstudyinvolvedgraduatemedicaleducation(GME) submissionstotheSpecialIssueandwasdeterminedtobe exemptbytheGeorgeWashingtonUniversityInstitutional ReviewBoard.

Thereareseveralrecurrentthemesidentifiedinthe literaturethatappearimportanttodevelopinganeffective peer-reviewevaluationsystem.Suchasystemshouldbe: 1)practicalandsimpletouse5,6,19,20;2)criterion referenced4,5,20;and3)beabletocapturedifferences inexpertreviewers’ perspectives.14,16,21–23 To successfullyoperationalizeanevaluationsystem,past worksalsosuggestthatratertrainingisnecessarytoensure properimplementation.3,10,13,20,24–28

Priortotheproductioncycleforthe2020–2021Special IssuetheCouncilofResidencyDirectorsinEmergency Medicine(CORD)guesteditorandthreeassociateeditors discussedtheneedforanimprovedsystemforevaluating peerreviews.Theuseofaglobal five-pointscorehasbeen showntobepracticalinassessingreviews.5,6 Byadding anchorstoeachpointonthe five-pointscalebasedonquality asthecriteriareference,Landkroonetalprovidedearly validityevidencesupportingitsuse.6 Todefine characteristicsimportanttohigh-qualitypeerreviewsin developinganchorsforthecurrentstudy’ s five-pointglobal scale,theeditorsreviewedthemissionandvisionstatements ofCORD,29 theliteraturerelevanttopeer-reviewscoring instrumentsandreviewerguidelinesfromfourmajormedical journals.30–33 Throughaniterativeprocess,theeditors definedqualitiesofanidealreviewasonethatprovidesthe following:1)–insightsthatreflectboththevaluetoreadership andalignmentwiththecurrentliterature;2)–consideration oftheappropriatenessofthestudymethod(s)andrelevant tenetsofeducationscholarship;and3)–feedbackthat providesmentorshiptoauthorsonhowtoimprovetheir manuscriptaswellastheirownskillset.

Throughthesameprocess,theeditorsdeterminedthatthe anchorsforthe five-pointscoringrubricshouldbebasedon thesethreeprovisionsofaqualityreviewaswellasthedegree towhichthereviewinformsthe finalevaluationofthe submissionunderreview(Figure1).Inotherwords,areview evaluationof fiveonthisholisticeditor’sscoringrubric (HESR)providesallthreeprovisionsofaqualityreviewand couldstandaloneasthe finalevaluationofthatsubmission (Goldstandardreview).

ValidityAssumptions

WeusedKane ’sframeworktogathervalidityevidence foruseofthisinstrument,whichinvolvedtesting

Gold Standard Reviews are those that provide:

1)Insightful review (detailed and global)that reflects upon how the work under consideration may be of value to the readership and informs the current literature.

2)Appropriateness of the study method(s) employed anda reflection of the relevant tenets of education scholarship.

3)Feedback that provides mentorship to authors on how to improve their manuscript and their own skill set.

5-Exceptional: A model review that reflects each of the tenetsof the Gold Standard and could stand alone as a summary to the authors. Recommendations to the authors are appropriate, actionable and supportive with a basis in educational scholarship.The review provides an in-depth perspective which may include relevant citations, resources or specific suggestions for improving the manuscript and/or professional growth. An additional contributing factor includes instances where the reviewer makes an important observation or recommendation not previously considered by the editors.

4-Very Good: An excellent review that reflects the time, effort and expertise necessary tocontribute substantially to the formal decision but falls short in one or more of the 3 key areas that define the “Gold Standard”. For example, an excellent overall review that (1) misses 1-2 substantive points, (2) provides only cursory mention of educational scholarship concepts or (3) falls short of providing mentoring support when critiquing the authors work.

3-Good: The review meets the standard of an acceptable review. The analysis adds to the broader perspective in a measured way but is not as complete, organized, documented or is lacking adequate explanations for the authors. As a result, additional reviews are required to provide more extensive/actionable feedback to the authors.

2-Below Average: Though there may be some insights included the review provides a superficial evaluation of the submission. This may include lack of reasoning for the decisions rendered, comments are not actionable or there may be a general lack of critique for improvement. In essence, insights provided may reinforce other reviewers’ comments but are not substantive enough to shape editorial decision-making pertaining to the manuscript. The majority of components of a “Gold Standard review” are missing.

1-Unacceptable: The review is sparse and may provide 1-2 insights but either (1) provides a decision without explanation(accept/reject, like/dislike, good/bad)(2) provide praise without critique (no substantive feedback for how to improve the manuscript) and concludes “accept as is” when revisions are needed or rejects with minimal justification, (3) lacks meaningful insights or (4) conclusions are based on faulty reasoning based on the literature, opinions of the other reviewer(s) and the editor. In short, the review provides little if any substantive critique that contributes to consensus decision making.

Figure1. Theholisticeditor’sscoringrubricusedforevaluationofpeerreview.Theinitialversionofthisrubriccanbeappreciatedasthe unshadedcontent.Subsequentadditionsmadebasedonapilotofthe14initialreviewsaredenotedbytheshadedareas(See “Preliminary CalibrationExercise.”)

CORD,CouncilofResidencyDirectorsinEmergencyMedicine.

assumptionsaboutscoring,generalization,extrapolation, andimplication. 18 Our fi rstassumptioninvolvedthe distributionofratingsorscores.Inreviewingscoresfrom thepastfewyearswiththetraditional fi ve-pointscale,we observedlimiteduseoftheevaluationscalewithskewing towardshigherscores( >3);editorswerehesitanttoassign scoresof1or2whenappropriate.Thewidevariabilityof experienceandexpertiseamongthereviewerssuggested thatgreatervariabilityinscoresshouldhavebeenpresent. Thisskeweddistributioncouldbeattributedtoaleniency bias,whichisnotuncommoninmedicaleducation evaluations. 23, 34–37 OurlogicfollowedthatfortheHESRto beavalidre fl ectionofpeer-reviewperformance,thepeerreviewevaluationscoresmustre fl ectthefullrangeofpeerreviewperformance.Ifsuccessfullydevelopedand implemented,theHESRpeer-reviewevaluationscores

wouldhaveadistributionwhereallratingoptions wereused.

Oursecondassumptioninvolvedinferencesaboutscoring, namelythattheassignedHESRscoreforapeerreviewwould beanaccuraterepresentationoftheeditor’sperspectiveof thequalityandvalueofapeerreviewtoeditorialdecisionmaking.Iftrueateachscorelevel,theassociateandsenior editorswouldbeconsistentwitheachotherinapplyingthe HESRforanygivenreview.

Ourthirdassumptionhadtodowithimplicationsthat HESRscoreswouldbeusedtoinformtheeditorialdecisionmakingprocess.Inotherwords,highlyratedreviewswould havemorevalueindecision-making,andasaresultthe reviewer’sdecisionrecommendation(i.e.accept,revise, reject)wouldmorecloselyalignwiththeeditor’ s formaldecision.

StudySettingandParticipants

TheSpecialIssuewasestablishedin2014asanannual publicationof WestJEM dedicatedtoeducationalresearch practice.38 SubmissionsrelatedtoGMEweremanagedbya singleguesteditorandthreeassociateeditors.Peerreviewers fortheSpecialIssuewererecruitedbythesenioreditorvia theCORDandClerkshipDirectorsofEmergencyMedicine listservs.Theprerequisiteforbecomingareviewerincluded recognitionasanexperiencededucatorandauthorshipofat leastonescholarlyeducationalstudypublishedinapeerreviewedjournal.

Onceamanuscriptwassubmittedviathe WestJEM submissionportal,screeningeditorseitherapprovedthe manuscriptforpeerrevieworchoseto “desk-reject” themanuscriptwithoutreview.Manuscriptsthatpassedthe

screeningprocesswerethenassignedtotwoexternalpeer reviewers.Inaniterativeprocess,reviewersconcludedtheir reviewswitharecommendationtotheassociateeditorwhoin turnmadearecommendationtothesenioreditorfora formaldecision.Ateachstepinthisprocessthechoicewasto reject,revise,oracceptthemanuscript.Inthoseinstances whererevisionswererequestedandsubmitted,the final decisionwaseithertoacceptorrejectforpublication (Figure2).Reviewerassignmentwasrandomwithoutregard todefinedexpertise(eg,statistics,specificmethodology,topic underconsideration).

Throughthe first fiveeditionsoftheSpecialIssue,peer reviewswere “rated” byassociateandsenioreditorsusingthe methodsandinstrumentationadoptedfromtheparent WestJEM editorialboard:aclosed,internalevaluation

Figure2. Flowchartshowingthenumberofmanuscriptssubmittedandprocessedduringthe2020–21submissionscycleforthe Western JournalofEmergencyMedicine SpecialIssueinEducationalResearch&Practice. UGME,undergraduatemedicaleducation; CDEM,ClerkshipDirectorsofEmergencyMedicine; GME,graduatemedicaleducation; CORD, CouncilofResidencyDirectorsinEmergencyMedicine.

Loveetal. ScoringRubricforEvaluationofPeer-reviewQuality

systemthatusedaglobal five-pointsemanticdifferential scalewithlabelsattheextremes:5 = highquality and1 = low.

ImplementationoftheHolisticEditor’sScoringRubric

TheHESRwaspilotedduringproductionofthe 2020–2021SpecialIssue.Manuscriptsreviewedbythefour CORDeditorsoftheSpecialIssuewereincludedinthepilot. Tominimizeissuesofrecallandmaintainconsistency,the editorsallagreedtoreflectupontheHESRjustpriorto scoringeachpeerreview.Subsequently,eachpeerreview receivedanindependentscorefromanassociateeditorand thesenioreditor.Blindingwasaccomplishedbyhaving eacheditorreporttheirreviewscoretoathirdparty. Thethirdparty(AM),whowasnotinvolvedintheformal reviewprocess,maintainedthesecureddatabasethatlinked theeditor’sandassociateeditor’sHESRratingsfor eachreview.

PreliminaryCalibrationExercise(Pilot)

Theeditorspausedtoreviewtheirexperiencewiththe originalHESRafterthe first14peerreviewsofmanuscripts hadbeenscored.Comparisonsbetweenscoresassignedby theseniorandassociateeditorsraisedquestionsandconcerns abouttheclarityofthescoringrubric,whichwarranted anotherroundofrevisions.Changesweremadebasedon iterativediscussionsandconsensustoimprovetheclarityof therubricoptionsandratings.Inaddition,adjective descriptorsthatcharacterizedeachoptionwereaddedas follows:5-Excellent,4-Verygood,3-Good,2-Belowaverage, and1-Unacceptable(Figure1).Duringtheimplementation stage,the finalHESRwasusedtoscoretheremaining32 GMEsubmissionsduringthe2020-21SpecialIssue productioncycle.

DataAnalysis

Assumption1–DistributionofEvaluationScores

Our firstassumptionwasthatavalidscoringmechanism ofeditorialevaluationofpeerreviewsshouldreflectthe variabilityofqualityandvalueofthereviews.Theprevious semanticdifferentialratingsystemusedbytheSpecialIssue duringapriorproductioncycle(2019–2020)didnotreflecta highdegreeofvariabilityinpeer-reviewscores.Infact,the distributionofscoresfromthiscycleappearednegatively skewedwithscoresclusteredaround “4” onthe five-point semanticdifferentialscale.Accordingly,onegoalofthenew HESRwasforittomoreaccuratelyreflectthevariabilityof thereviewerpoolwithregardtoscholarlyexpertisethrough useoftheentireevaluationscale.Usingthe “Explore” featureinIBM-SPSSversion28(SPSS,Inc,Chicago,IL)to generatehistograms,frequencydistributions,andmeasures ofvariability,wecomparedthreesetsofpeer-reviewscores: semanticdifferentialratingsfromthe2019–2020Special Issue(baseline);thepilotCORDeditor’sevaluationofpeer

reviews;andthefullimplementationofCORDeditor’ s evaluationofpeerreviewsusingtherevisedHESRforthe 2020–2021edition.

Assumption2–Inter-raterReliabilityBetweenEvaluatorsof theSamePeerReviews

SincetheHESRprovidedclearcriteriafor fivedifferent levelsofpeer-reviewperformance,weexpectedtheHESRto generatereliablescoresacrosseditors.Accordingly,like Cicchetti,wecomparedinter-raterreliabilityamongeditors usingtheintraclasscorrelationcoefficient(ICC).39 To complementourreliabilityevaluation,wealsousedthree measuresofagreementbetweenassociateandsenioreditors’ ratingsofpeerreviewsfortheCORDeditorialteam:percent ofabsoluteandrelativeagreement,andtheSpearmanrho correlationforordinalleveldata.40 Thepercentageof identicalratingsisameasureofabsoluteagreementbetween raters,whilethepercentageofratingsincloseproximityof eachother(+1)isanindicatorofwithin-raterconsistency. TheSpearmanrhocorrelationprovidesanindicatorofthe strengthoftherelationshipbetweentheratingsacrossthe twotypesofraters.41 WeusedthecriteriafromSchoberetal. forinterpretingtheSpearmanrhocorrelation(rof

0–0.10 = negligible;rof0.10–0.39 = weak;rof0.40–0.69 = moderate;rof0.70–0.89 = strong,rof0.90–1-verystrong).40

TheICCmodelselectedforthisstudyisbasedonseveral assumptions.First,itisassumedthatassociateeditorswere randomlychosenfromalargerpoolandthatthesenior editorwas fixed.Second,thedesignwasnotfullycrossed, sincenoteveryreviewwasratedbythesameeditors.Third, sinceoneratingwasthefocus,ratherthanaseries,the absoluteagreementwasthoughttobethemostappropriate ICCmodel.A finalassumptionwasthatsincetheICCwas beingaskedtorepresenttheaverageofseveralcoders,the “ averagemeasures ” ICCwaschosen.Insummary,theICC formulachosenforthisstudyisaone-wayrandomeffects modelreflectingabsoluteagreementandtheunitofanalysis relatedtoaveragemeasures.42 Weappliedguidancefrom CicchettiforinterpretingtheresultingICCreliabilityindices (ICCof <.4 = poorreliability;ICCof.40–.59 = fair reliability;ICCof.60–.74 = goodreliability;ICCof .75–1.0 = excellentreliability).39 Unfortunately,wewerenot abletoperformcomparableinter-raterreliabilityanalyses forthepriorSpecialIssueproductioncycle(2019–2020)due tothetemplatedabilitytoprovideonlyoneeditor’ sscore permanuscript.

Assumption3–ImplicationsortheStatisticalRelationship BetweenPeer-ReviewRatingandtheEditorialDecision

Thecollectiveeditors’ evaluationsofthepeerreviewwere assumedtobeanindicatorofitsquality.Ifeditorsplaced valueonpeerreviewsduetotheirabilitytoinformthe decision-makingprocess,thenhigherqualitypeerreviews shouldhavebeenmorelikelytoagreewiththeeditorial

decisionsthanlowerqualitypeerreviews.Inthisanalysis,the categoriesofyes/norefertowhetherthereviewer’ s recommendationagreedwiththeformalmanuscript decision. “Yes” designationswereappliedifthereviewer recommendedthearticlebeaccepted,rejectedorrevised andtheeditorialdecisionmadeagreedwiththat recommendation.Ifthereviewer’srecommendationdidnot agreewiththeeditor’sdecision,thiswascategorizedasa “No.” Thisisknownasaparallellineofvalidityevidence accordingtoKane.18 Wetestedthishypothesisbyaveraging theseniorandassociateeditors’ peer-reviewratingsandthen categorizingtheseaverageratingsinto fivecategories: (1–1.5);(2–2.5);(3–3.5);(4–4.5);and(5).Next,usingachisquaretestofassociationwetestedtherelationshipbetween thesummaryratingcategoryandthereviewer’sagreement withthe finaldecision(Didthereviewer’srecommendation agreewiththe finaldecision,yesorno?).Iftrue,theauthors positedthatthehighertheratingsbytheeditorsonthe qualityofthepeerreview,themorelikelytheir recommendationsforthemanuscriptsubmissionwould agreewiththeactualformaldecision.Weappliedthecriteria fromHahs-Vaughnetalforinterpretingtheassociatedeffect sizesfromthechisquaretestofassociation(smalleffects = <0.10,mediumeffects = 0.30;andlargeeffectsare >0.50).43

WealsoevaluatedtherelationshipbetweentheHESR scoreandthereviewer’sagreementwiththe finaldecision usinglogisticregressionanalysis.Forthisanalysis,we attemptedtopredictwhetherthereviewerrecommendation wouldmatchthe finaleditorialdecision(yesorno)fromthe HESRscoresassignedbyeachtypeofeditor.Resultsofthis testshouldprovidearelativestrengthoftherelationship betweeneachtypeofeditor’sratingandthe editorialdecision.

RESULTS

Thetotalnumberofmanuscriptssubmittedforthe2021 SpecialIssuewas163.Ofthese,85weremanagedbythe

CORDeditors.Thirty-eightsubmissionsweredesk-rejected bytheeditorialstaff.Subsequently,47(55.3%)manuscripts wereapprovedforpeerreviewand16werepublished,foran acceptancerateof18.8%.These47peer-reviewed manuscriptswerethesubjectofthisstudy,14duringthe pilotperiodand33duringfullimplementationofthe HESR(Figure2).

Eighty-fourpeerreviewersreviewedanaverageof1.84 manuscriptseach(SD1.34).Abouttwo-thirdsofpeer reviewersperformedonlyonereview(52/85;61.2%),while anadditional34%(29/85)completed2-4reviews,andfour individuals(4.8%)completed5– 7reviews.Theeditors performed91evaluationsofpeerreviews,27atthepilot stageand64atthefullimplementationstage.Thethree associateeditorsperformed95peer-reviewevaluations,32 atthepilotstageand63atthefullimplementationstage (Table1).Therewere90matchedpairsofevaluations onthesamepeerreviewfromboththeseniorand associateeditor.

DistributionofScores

Duringthepriorproductioncycle(2019–2020),163peer reviewswereratedusinga five-pointsemanticdifferential scale.Thedistributionofeditors’ ratingsofthesereviewswas showntobenegativelyskewed( 0.371),whichwascaused bytheunderuseofthe “1” ratingandoveruseofthe “4” and “5” ratings(Figure3).Contrastedwiththesemantic differentialscale,theHESRdistributionatboththepilotand fullimplementationstagehadskewnessclosertozero(0.005 and0.078,respectively).Thedistributionduringthepilot stageisconsideredaparalleldistribution,sincealmostall responseoptionswerechosenequally(exceptforthe “1” HESRrating).Duringthefullimplementation, negativeskewness(0.078)almostdisappearedasthe distributionbecamemorebellshaped,andkurtosis continuedtosuggestadistributionwithsymmetry (kurtosis = 0.967)(Figure3).44

Table1. Numberandpercentagesofsenioreditor,associateeditors,andreviewersinvolvedintheproductionofthe2021SpecialIssueby group.Includedarethenumbersofreviewevaluationsperformedandmanuscriptsprocessedbyseniorandassociateeditorsandthe numbersofpeerreviewersandmanuscriptstheyreviewed.

CORDHESRStudyTotal

Calibration-pilotImplementation

Associate editors

Reviewers*32N/A1469N/A3384N/A47

*Therewere84totalpeer-reviewerswhoreviewedmanuscriptsduringeitherthepilotorfullimplementationphaseofthisstudy.Fifteenof32 reviewersparticipatedonlyduringthepilotwhiletheother17contributedtoreviewingatbothstagesoftheproject(pilotandfull).Reviewers werenotinvolvedwithusingtheHolisticEditor’sScoringRubrictoassesstheirownpeer-reviews. CORD,CouncilofResidencyDirectorsinEmergencyMedicine; HESR,holisticeditor’sscoringrubric; evals,evaluations.

Figure3. Comparisonofthepeer-reviewratingsdistributionsfortwomethodsofeditorevaluationsacrosstwoSpecialIssueproduction cycles.The firstmethodinvolveda5-pointsemanticdifferentialscalewithlabelsonlyattheendpoints,whichwasusedduringthe2020cycle. Thesecondmethodinvolvedtheholisticeditor’sscoringrubricusedbytheCORDeditorsatthepilotandfullimplementationstagesofthe 2021productioncycle.

CORD,CouncilofResidencyDirectorsinEmergencyMedicine. *Datasignificantlydeviatefromnormaldistribution(P ≤ 0.001).

Resultsoflogisticregressionusingtheoutcome,whetherthereviewerrecommendationmatchedwiththe finaldecision(yesorno), regressedonthepredictors:associateandsenioreditor’speer-reviewratings.

Equation1:Associateeditor’sscoreaspredictorforreviewrec/finaldecmatch[93]

95%CIforEXP (B)

Equation2:Senioreditorscoreaspredictorforreviewerrec/ finaldecmatch[N = 91]

95%CIforEXP (B)

Equation3:Associate&senioreditorscoresaspredictorforreviewerrec/finaldecmatch[N = 90]

95%CIforEXP

Inter-raterAgreement

NinetypeerreviewswereassessedwiththeHESRbyboth thesenioreditorandoneofthethreeassociateeditors.The

percentageofabsoluteagreementbetweenthetwotypesof editors’ ratingsofpeerreviewswas37.8%(Table2).Nearly half(47.8%)ofratingswereindisagreementbyonlyone

Table2.

point.TheassociatedSpearmanrhocorrelationand r-squaredfortheratingsfromthetwotypesofassessorswas 0.703(R2 = 0.49).ASpearmanrhoofthismagnitudeis interpretedasborderingbetweenamoderateandstrong positivecorrelationorstatisticalrelationship.40 Finally,the ICCbetweentheassociateandsenioreditorswas0.795.

AsinterpretedbyCicchetti,anICCofthismagnitudeis consideredexcellentintermsofclinicalsignificance (between0.75–1.00).39

ImplicationofHESRScoresasAssociatedwithand PredictorsofManuscriptOutcomes

Thechi-squaretestofassociationfortherelationship betweenaveragepeer-reviewratingsandthepeerreviewer’ s recommendationwiththe finalmanuscriptdecisionwas statisticallysignificant(chi-square = 17.4,df = 4, P < 0.01, effectsize = 0.44).Theassociatedeffectsizeof0.44is classifiedasamediumeffectsizeaccordingtoHahs-Vaughn etalwhosuggestthatsmalleffectsarethose ≤ 0.10,medium effects = 0.30,andlargeeffects >0.50(Table3).43

LogisticRegression

Foralllogisticregressionanalyses,thetestsformodel coefficientsweresignificant,suggestingthatanyoneofthe threeformulaswouldimproveourestimateoftheprobability thatthepeer-reviewrecommendationmatchedtheeditorial decision.TheHosmer-Lemeshowtestswerenotsignificant, indicatingthatthemodelscouldbeagood fit,andanalysesof thescatterplotsofpredictedscoresandresidualscontributed totheconclusionthattheanalysesmettheassumptionsof normalityandequalvariance(statisticsnotshown).

Table3. Reviewersummaryratinggroupedinto5categoriescross tabulatedwithwhetherthereviewer’srecommendationagreedwith manuscript finaldecision(expectedvaluesareinparentheses)with chisquaretestofassociation*betweenthesetwovariables.

Didreviewer’srecommendation agreewith finaldecision Reviewersummary

Logisticregressionanalysesdemonstratedthatthe associateeditor’sHESRratingswereasignificantpredictor ofthemanuscriptoutcome:asuccessfulmatchbetweenthe reviewer’srecommendationandthe finaldecisionforthe manuscript.Thiswasalsotrueofthesenioreditor’sHESR ratings.However,becausetheratingsoftheassociateeditor andsenioreditorweresohighlycorrelatedwitheachother (Spearmanrhocorrelation = 0.703),oncecombinedintoone logisticregressionmodel,onlythesenioreditor’sratings surfacedasasignificantpredictor.

Interpretationofthesenioreditor’sHESRratingsasa predictorsuggeststhattheratingscontributedtoimproving thecorrectclassificationofpredictedvsobservedoutcomes from51.6%withnopredictorto63.7%.Theadjustedodds ratioExp(B) = 2.548(95%confidenceinterval[CI]

1.570–4.133]canbeinterpretedasfollows: “Foreveryone stepincreaseinthesenioreditor’sevaluationratings,therisk oftheoutcomeofasuccessfulmatchbetweenreviewer recommendationandthe finaldecisionincreasesbyafactor of2.548” (Table2).

DISCUSSION

UsingKane’sframeworkforvalidityevidence,thiswork testedthreeassumptionsregardingtheHESRasanovel meansforeditorstoassessthequalityofpeerreviewsof educationalscholarship.The firstassumptioninvolvedthe distributionofeditors’ ratingsofreviews, findingthatthe HESRdemonstratedgreatersymmetryinscorescompared toapriorinstrumentusedduringthe2019cycle.We concludethatleniencybiaswaslikelylimitedbythecriteriareferencingbasisofthisintervention,whichmayhave resultedfromclearerbehavioralanchorsoftheinstrument itself,5,6,13 improvementsinratertraining,3,10,13,24–28 and/or moreintentionalqualitycontrolamongeditorsduringthe reviewprocess.Thischangeinbehaviormayalsoreflectthe practicalityoftheHESRsinceitwasclearlybeingusedin editorialevaluationofreviews.

ThesecondassumptionmadeinfersthattheassignedpeerreviewscoresbasedontheHESRareanaccurate representationoftheeditors’ perspectiveonqualityand informaholisticperspectiveonthesubmission.Whilethe editorsoftheSpecialIssuehadlowerabsoluteagreement betweentheseniorandassociateeditors(37.8%),they demonstratedexcellentrelativeconsistencyreliability (ICC = 0.795)andcorrelations(Spearmanrho = 0.703) betweeneditors’ scores.Inotherwords,theirevaluations, whilenotidentical,wereinternallyconsistent.This finding relatedtoreliabilitysupportsthehypothesisthattheHESR capturestheeditorialperceptionofqualityaswellasthe degreetowhichpeerreviewinformsaholisticunderstanding ofasubmissionsvalue.

*Theminimumexpectedcountsare3.42.Cramer’sphieffect sizesareinterpretedas ≤ 0.10 = small;0.30 = medium;and ≥ 0.50 = largeeffects.

Finally,thethirdassumptionsmadehastodowiththe implicationthatscoresareusedtoinformtheeditorial decision-makingprocess.Thisissubstantiatedbythe finding

thattheratingsontheHESRcorrespondedtothereviewer’ s agreementwiththemanuscript’ s finaldisposition.Thehigher thepeer-reviewevaluationofqualitybyHESRscoring,the higherthecorrelationbetweenthereviewerdisposition recommendationandthemanuscript’sformaloutcome. Althoughthisistobeexpected,thefactthatitholdstruein thisinstancedemonstratesthateditorsvalueandrate reviewshigherwhentheycontributesubstantiallytothe editorialdecision.18

LIMITATIONSANDFUTURESTUDIES

These findingsshouldbeinterpretedinthecontextof severallimitations.Traditionally,theSpecialIssuehasnot blindeditsseniorandassociateeditorstotheidentityofthe reviewers.Thisraisesthepotentialforbiasifeditors recognizedpeerreviewers’ names,whichcouldconceivably haveimpactedtheratingsofmorefamiliarpeerreviewers. Second,theeditorsinthisstudyhadregulardiscussions regardingtheuseandinterpretationofthisscoring instrument.Giventhecentralityofratertrainingintheuseof anyevaluationinstrument,futureworkwillhelpto determinewhethertheperformanceoftheHESRandlackof skewnessinscoringpersistsbeyondtheeditorialfocus associatedwiththisstudy.

Mostimportantly,thisstudyisbasedonasinglecycleofan annualspecialty-relatedpublicationfocusedonhealth professions-educationtopicswithasmallnumberofeditors andreviews.Futurestudiesshouldfocusonassessing additionalvalidityevidencesupportingtheHESR’ suse aswellasvaryingjournalenvironmentswithlargernumbers ofeditorsandreviews.Ourresultsaremostlikelytogeneralize tospecialty-specificeducationjournalswhoseapproachis similartothatpresentedinthisstudy.Our findingsareless likelytogeneralizetojournalsthattakeanalternative approachsuchasthosethatbringtogetheradiversesetof reviewersbasedonexpertise(eg,methodology, psychometrics,content,etc)toassessspecificcomponents ofthesubmission.

CONCLUSION

Aholisticunderstandingofthevalueofascholarly submissionrequiresaniterativeprocessinformedbythe expertperspectiveofreviewersthatisoftensubjectiveand nuanced.Theholisticeditor’sscoringrubricwasdevelopedas apracticalapproachtoeditorialevaluationofthequalityofa reviewandthedegreetowhichitinformstheformaleditorial decision.Bystudyingaprioriassumptionsrelatedtothe developmentanduseoftheHESR.includingdistributionsof evaluationscores,inter-raterreliabilitybetweenevaluatorsof thesamepeerreviewsandthestatisticalrelationshipbetween peer-reviewratingandtheeditorialdecision,thisstudy providesvalidityevidencesupportingtheuseoftheHESR. Futureworkshouldfocusonfurtherdefiningthevalueand limitationsoftheHESR.

AddressforCorrespondence:JeffreyN.Love,MD,4917HineDr., ShadySide,MD20764.Email: Jlove01@georgetown.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Loveetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.LockS. ADifficultBalance:EditorialPeerReviewinMedicine Philadelphia,PA:ISIPress,1986.1–38,122–132.

2.CicchettiDV.Thereliabilityofpeerreviewformanuscriptandgrant submissions:across-disciplinaryinvestigation. BehavBrainSci. 1991;14(1):119–35.

3.StrayhornJrJ,McDermottJrJF,TanguayP.Aninterventiontoimprove thereliabilityofmanuscriptreviewsfortheJournaloftheAmerican AcademyofChildandAdolescentPsychiatry. AmJofPsychiatry. 1993;150(6):947–52.

4.HowardLandWilkinsonG.Peerreviewandeditorialdecisionmaking. BrJPsychiatry. 1998;173:110–3.

5.CallahamML,BaxtWG,WaeckerleJF,etal.Reliabilityofeditors’ subjectivequalityratingsofpeerreviewsofmanuscripts. JAMA. 1998;280(3):229–31.

6.LandkroonAP,EuserAM,VeekenH,etal.Qualityassessmentof reviewers’ reportsusingasimpleinstrument. ObstetGynecol. 2006;108(4):979–85.

7.BormannL,MutzR,DanielHD.Areliability-generalizationstudyof journalpeerreviews:amultilevelmeta-analysisofinter-raterreliability anditsdeterminants. PLoSOne. 2010;5(12):e14331.

8.SmithR.PeerReview:A flawedprocessattheheartofscienceand journals. JRSocMed. 2006;99(4):179–82.

9.FeurerID,BeckerGJ,PicusD,etal.Evaluatingpeerreviews:pilot testingofagradinginstrument. JAMA. 1994;272(2):98–100.

10.VanRooyenS,BlackN,GodleeF.Developmentofthereview qualityinstrument(RQI)forassessingpeerreviewofmanuscripts. JClinEpidemiol. 1999;52(7):625–9.

11.OnitiloAA,EngelJM,Salzman-ScottSA,etal.Acore-itemreviewer evaluation(CoRE)systemformanuscriptpeerreview. AccountRes. 2014;21(2):109–21.

12.DoiSAR,Salzman-ScottSA,OnitiloAA.ValidationoftheCoRE questionnaireforamedicaljournalpeerreview. AcctResear. 2016;23(1):47–52.

13.GingerichA,KoganJ,YeatesP,etal.Seeingthe “blackbox” differently: assessorcognitionfromthreeresearchperspectives. MedEduc. 2014;48(11):1055–68.

14.ColeS,ColeJR,SimonGA.Chanceandconsensusinpeerreview. JSci. 1981;214(4523):881–6.

15.GarfunkelJM,UlshenMH,HamrickHJ,etal.Problemsidentified bysecondaryreviewofacceptedmanuscripts. JAMA. 1990;263(10):1369–71.

16.KassirerJPandCampionEW.Peerreview:crudeandunderstudied,but indispensable. JAMA. 1994;272(2):96–7.

17.DeMariaAN.Whatconstitutesagreatreview? JACC. 2003;42(7):314–5.

18.CookDA,BrydgesR,GinsburgS,etal.Acontemporaryapproachto validityarguments:apracticalguidetoKane’sframework. MedEd. 2015;49(6):560–75.

19.ThompsonSR,AgelJ,LosinaE.TheJBJSpeer-reviewscoringscale: avalid,reliableinstrumentformeasuringthequalityofpeerreview reports. LearnPubl. 2016;29(1):23–5.

20.RajeshA,CloudG,HarisinghaniMG.Improvingthequalityof manuscriptreviews:impactofintroducingastructuredelectronic templatetosubmitreviews. AJR. 2013;200(1):20–3.

21.CaelleighAS,SheaJA,PennG.Selectionandqualitiesofreviewers. AcadMed. 2001;76(9):914–5.

22.VarsGF,WentworthJ,SebbersonD.Educationalconnoisseurship, criticism,andtheassessmentofintegrativestudies. IssuesStud. 2002;20:65–76.

23.MillerKH,WestDC,KaraniR.Foreword:sticksandstones:moving towardamoreproductivepeer-reviewprocessforauthorsand reviewers. AcadMed. 2017;92(11S):Si–iii.

24.AlbaneseJ,PruchaC,BarnetJH.Labelingeachresponseoption andthedirectionofthepositiveoptionsimpactstudentcourseratings. AcadMed. 1997;72(10Suppl1):S4–6.

25.SattlerDN,McKnightPE,NaneyMR.Grantpeerreview:improvinginterraterreliabilitywithtraining. PLoSOne. 2015;10(6):e0130450.

26.JacksonJS,BondM,LoveJN,etal.Emergencymedicinestandardized letterofevaluation(SLOE):datafrom2016-2017eSLOEelectronic website. JGradMedEduc. 2019;11(2):182–6.

27.RobbinsJ,FirthA,EvansM.Improvingwork-basedassessment: addressinggradeinflationnumericallyorpedagogically. Practitioner ResearchinHigherEducation. 2018;11(1):80–6.

28.Gomez-GaribelloCandYoungM.Emotionsandassessment: considerationforrater-basedjudgementsofentrustment. JMedEd. 2018;52(3):254–62.

29.CouncilofResidencyDirectorsofEmergencyMedicine:Visionand MissionStatements.Availableat: https://www.cordem.org/about-cord/ mission–vision/mission–vision/.AccessedFebruary17,2020.

30.TaskForceofAcademicMedicineandtheGEA-RIMECommittee. Checklistofreviewcriteriaforresearchmanuscripts. Academic Medicine. 2001;76(9):958–9.

31.MedicalEducation:GuidelineforAuthors.Availableat: https:// onlinelibrary.wiley.com/page/journal/13652923/homepage/ guidelines_for_reviewers.htm.AccessedFebruary17,2020.

32.WileyAuthorServices.Howtopeerreview:stepbystepguideto reviewingamanuscript.Availableat: https://authorservices.wiley.com/ Reviewers/journal-reviewers/how-to-perform-a-peer-review/ step-by-step-guide-to-reviewing-a-manuscript.html

AccessedFebruary17,2020.

33. JGradMedEduc.Instructionsfor JournalofGraduateMedical Education (JGME)PeerReviewers.2018.Availableat: https://meridian.allenpress.com/DocumentLibrary/GMED/ JGME-Peer-Reviewer-Instructions_April%202018.pdf

AccessedFebruary17,2020.

34.BandieraGandLendrumD.Dailyencounteredcardsfacilitate competency-basedfeedbackwhileleniencybiaspersists. CJEM. 2008;10(1):44–50.

35.ArcherJCandMcAvoyP.Factorsthatmightunderminethe validityofpatientandmulti-sourcefeedback. MedEduc. 2011;45(9):886–93.

36.LoveJN,DeiorioNM,Ronan-BentleS,etal.Characterizationofthe CORDstandardizedletterofrecommendationin2011-2012. Acad EmergMed. 2013;20(9):926–32.

37.Bing-YouR,VaraklisK,HayesV,etal.Thefeedbacktango: anintegrativereviewandanalysisofthecontentofthe teacher-learnerfeedbackexchange. AcadMed. 2018;93(4):657–63.

38.LoveJNandAnderDS.Buildingaspecialtyspecificcommunity ofpracticeineducationscholarship. WestJEM. 2015;16(6):799–800.

39.CicchettiDV.Guidelines,criteria,andrulesofthumbforevaluating normedandstandardizedassessmentinstrumentsinpsychology. PsycholAssess. 1994;6(4):284–90.

40.SchoberP,BoerC,SchwarteL.Correlationcoefficients:appropriate useandinterpretation. AnesthAnalg. 2018;126(5):1763–8.

41.KottnerJ,AudigéL,BrorsonS,etal.Guidelinesforreportingreliability andagreementstudies(GRRAS)wereproposed. IntJNursStud. 2011;48(6):661–71.

42.HallgrenKA.Computinginter-raterreliabilityforobservationaldata: anoverviewandtutorial. TutorQuantMethodsPsychol. 2012;8(1):23.

43.Hahs-VaughnDLandLomaxRG.Chapter8.In: AnIntroductionto StatisticalConcepts(312).(4th Edition).NewYork,NY:Routledge,2020.

44.GlassGVandHopkinsKD. StatisticalMethodsinEducationand Psychology,2ndEd.EnglewoodCliffs,NJ:Prentice-HallInc; 1984:68–70.

NovelScoringScaleforQualityAssessmentofLungUltrasound intheEmergencyDepartment

JessicaR.Balderston,MD*

TaylorBrittan,MD*

BruceJ.Kimura,MD†

ChenWang,MS‡

JordanTozer,MD*

SectionEditor:RyanGibbons,MD

*VirginiaCommonwealthUniversityMedicalCenter,DepartmentofEmergency Medicine,Richmond,Virginia

† ScrippsMercyHospital,DepartmentofCardiology,SanDiego,California

‡ VirginiaCommonwealthUniversity,DepartmentofBiostatistics,Richmond,Virginia

Submissionhistory:SubmittedMay15,2023;RevisionreceivedNovember28,2023;AcceptedDecember5,2023

ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18225

Introduction: Theuseofareliablescoringsystemforqualityassessment(QA)isimperativetolimit inconsistenciesinmeasuringultrasoundacquisitionskills.ThecurrentgradingscaleusedforQA endorsedbytheAmericanCollegeofEmergencyPhysicians(ACEP)isnon-specific,applies irrespectiveofthetypeofstudyperformed,andhasnotbeenrigorouslyvalidated.Ourgoalinthisstudy wastodeterminewhetherasuccinct,organ-specificgradingscaledesignedforlung-specificQAwould bemoreprecisewithbetterinterobserveragreement.

Methods: ThiswasaprospectivevalidationstudyofanobjectiveQAscaleforlungultrasound(LUS)in theemergencydepartment.Weidentifiedthe first100LUSperformedinnormalclinicalpracticeinthe year2020.Fourreviewersatanurbanacademiccenterwhowereeitheremergencyultrasound fellowship-trainedorcurrentfellowswithatleastsixmonthsofQAexperiencescoredeachstudy, resultinginatotalof400.Theprimaryoutcomewasthelevelofagreementbetweenthereviewers.Our secondaryoutcomewasthevariabilityofthescoresgiventothestudies.Fortheagreementbetween reviewers,wecomputedtheintraclasscorrelationcoefficient(ICC)basedonatwo-wayrandom-effect modelwithasingleraterforeachgradingscale.Wegenerated10,000bootstrappedICCstoconstruct 95%confidenceintervals(CI)forbothgradingsystems.Atwo-sidedone-sample t-testwasusedto determinewhetherthereweredifferencesinthebootstrappedICCsbetweenthetwogradingsystems.

Results: TheICCbetweenreviewerswas0.552(95%CI0.40–0.68)fortheACEPgradingscaleand 0.703(95%CI0.59–0.79)forthenovelgradingscale(P < 0.001),indicatingsignificantlymore interobserveragreementusingthenovelscalecomparedtotheACEPscale.Thevarianceofscoreswas similar(0.93and0.92forthenovelandACEPscales,respectively).

Conclusion: Wefoundanincreasedinterobserveragreementbetweenreviewerswhenusingthenovel, organ-specificscalewhencomparedwiththeACEPgrading scale.Increasedconsistencyinfeedback basedonobjectivecriteriadirectedtothespecific,targetedorganprovidesanopportunitytoenhancelearner educationandsatisfactionwiththeirultrasoundeducation.[WestJEmergMed.2024;25(2)264–267.]

INTRODUCTION

Lungultrasound(LUS)isfrequentlyusedinthe emergencydepartment(ED)toassessbothmedicaland traumapatients.1,2 Qualityassessment(QA)ofultrasound imagesisoneofthesixrequiredelementsofdiagnostic

ultrasoundpertheAmericanCollegeofEmergency Physicians(ACEP)andisroutinelyperformedtoevaluate imagequality,ensuringappropriatepatientcare,and enablingreviewerstoassessuserperformance.2 The useofareliablescoringsystemforQAisimperative

tolimitinconsistenciesinmeasuringultrasound acquisitionskills.

ThecurrentQAgradingscaleendorsedbyACEPwas developedfromaconsensusreportofemergencyultrasound leaderstoprovideasystematicmethodtoreportand communicateultrasound findings.2 Itisanon-specificscale thatappliesirrespectiveofthetypeofstudyperformedand hasnotbeenrigorouslyvalidated.Similarlyformatted organ-specificQAgradingsystemsforcardiacandobstetric examshavebeendescribedbutarenotyetendorsedby ACEPandarenotwidelyused.2–5 AlternativeLUS assessmenttoolshavebeendeveloped;however,theyare extensiveandassuchimpracticalforroutineQAuseorare focusedonimageacquisitionskillsandnottailoredto anatomicfeedback.6,7 Ourgoalinthisstudywasto determinewhetherasuccinct,organ-specificgradingscale designedforQAwouldbemoreprecisewithbetter interobserveragreement.

METHODS

ThiswasaprospectivevalidationstudyofanobjectiveQA scaleforLUS.Wedevelopedanovel,lung-speci ficgrading scalebyarigorousreviewofexpert,publishedexperienceat anoutside,unaffiliatedinstitution(ScrippsMercyHospital, SanDiego,CA).Thisinstitutionroutinelyperformslung imagingandhaspublishedanassessmenttoolforthe evaluationofresident-performedbedsideultrasoundB-line interpretationinthoracicultrasound,aswellasananalogous cardiacqualityassessmentscale.3,7–13 Intheexpertreview, thecurrentavailable,organ-specificgradingscalefoundin theliteraturewasmodifiedtotheanatomyofthechest wall.3,5 Thegradationsofthescalewereempiricallyderived fromtheexperienceatthisinstitutioninadditiontoa rigorousreviewoftheliterature. 3,5,7–13 Theuseoffour criticallandmarks ribshadows,pleuralline,A/Blines,and technical flaws wererecognizedascommonalitiesinall publishedimagesinLUSstudies,includingexpert consensus.14,15 We,therefore,dividedtheselandmarksintoa pointscalethatprogressivelydefinesthepatternof acquisitionrequiredtoobtainanimage(ie,bones first, pleuralline,followedbyartifacts).Wedescribedtechnical flawsasnon-optimizeddepth/gain,distractingadjacent structures,inadequateaxis,orhandmovement.Wedeemed flawstobemajoriftheywerepresenttoadegreesignificant enoughtodecreasediagnosticcapabilities,orifmultiple flawswerepresent.

Thescalewasthenvalidatedatanurbanacademictertiary carecenterinRichmond,Virginia.Weidentifiedthe first100 LUSstudiescompletedaspartofregularclinicalpracticein theEDbyemergencyphysicianswithtwoormoreLUS videosperformedintheyear2020.Dedicatedthoracic ultrasoundexaminationsareingeneralperformedby residentphysicianswithattendingoversight.Studieswere obtainedusingSonositeXPorteultrasoundmachine

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Areliablemethodofqualityassessment(QA) ofultrasoundimagesisimperativetoassess userperformanceandlimitinconsistenciesin measuringultrasoundacquisitionskills.

Whatwastheresearchquestion?

IsthereaQAscoringscaleforlungultrasound (LUS)thatismoreprecisethanthe commonlyusedACEPscoringscale?

Whatwasthemajor findingofthestudy?

IntheQAofLUS,anovelscoringscale showedsigni fi cantlymoreinterobserver agreementcomparedtothe ACEPscale.

Howdoesthisimprovepopulationhealth?

AmoreindividualizedscoringscaleforQAof LUSresultsinlessgradingvarianceandmore objectivefeedbackwhencomparedtothe ACEPscale.

(FujifilmSonosite,Bethell,WA)usingeithertheC60XP5-2MHzcurvilineartransducer,L2513-6-MHzlineararray transducerortheP195-1-MHzphasedarrayprobe.Four reviewerswhowereeitheremergencyultrasoundfellowshiptrainedorcurrentfellowswithatleastsixmonthsofQA experiencescoredeachofthe100studiesresultinginatotal of400.TwoblindedreviewersusedthecurrentACEP gradingscale,2 andtwousedanovellung-specificgrading scale;therewasonefellowandoneultrasound-trained physicianineachgroup(Figure).Theprimaryoutcomewas thelevelofagreementbetweenthereviewers,indicatingthe reliabilityofthescoringsystem.Oursecondaryoutcomewas thevariabilityofthescoresgiventothestudies.Forthe agreementbetweenreviewers,wecomputedtheintraclass correlationcoefficient(ICC)basedontwo-wayrandomeffectmodelwithasingleraterforeachgradingscale.Ten thousandbootstrappedICCsweregeneratedtoconstruct 95%confidenceintervals(CI)forbothgradingsystems.We usedatwo-sidedone-sample t -testtodeterminewhether thereweredifferencesinthebootstrappedICCsbetweenthe twogradingsystems.

RESULTS

The first100LUSstudiescompletedintheEDby emergencymedicineresidents(postgraduateyear[PGY]-1,

1 No recognizable features of ribs, pleural line, or A/B line artifacts No recognizable structures

2 1 or 2 rib shadows seen with minimal identifiable pleural line (off axis)

3 1 or 2 rib shadows and pleural line seen, with major technical flaws

4 2 rib shadows seen, framing either A- or Blines, but with minor technical flaws

5 2 rib shadows seen, framing either A- or Blines, with no technical flaws

Minimally recognizable structures but insufficient for diagnosis

Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws

Minimal criteria met for diagnosis; all structures imaged well

Minimal criteria met for diagnosis; all structures imaged with excellent image quality

Figure. Comparisonofthenovel,lungultrasoundquality assessmentscalewiththetraditionalAmericanCollegeof EmergencyPhysiciansscale.

ACEP,AmericanCollegeofEmergencyPhysicians; LUS, lungultrasound.

42%;PGY-2,14%;PGY-3,22%)andEDfaculty(22%)were reviewedbyfourblindedreviewers.Imageswereobtained usingthelinearprobe(27%),curvilinearprobe(32%),phased arrayprobe(28%),oracombinationofprobes(13%).Studies hadamedianofsixclips(IQR4–9).Thescoresgivenusing theACEPscaleandthenovelscalearesummarizedinthe Table.TheICCbetweenreviewerswas0.552(95%CI 0.4–0.68)fortheACEPgradingscaleand0.703(95%CI0.59,0.79)forthenovelgradingscale(P < 0.001),indicating significantlymoreinterobserveragreementusingthe novelscalecomparedtotheACEPscale.Thevarianceof scoreswassimilar(0.93and0.92forthenovelandACEP scales,respectively).

Summarytableofscoringsystems.

StatisticsNovelACEP

N*200200

Min,max1,51,5 Mean(SD)3.70(0.96)3.32(0.96)

Median43 Q1,Q33,43,4 ICC(95%CI)0.703(0.59,0.79)0.552(0.40,0.68)

Variance0.930.92

Varianceratio (Novel:ACEP) 1.01

*N = numberofscoresgiven.

ACEP,AmericanCollegeofEmergencyPhysicians; ICC,intraclass correlationcoefficient; CI,confidenceinterval.

DISCUSSION

ThecurrentACEPgradingscaleusedforQAwas developedfromaconsensusreportofemergencyultrasound leadersbuthasnotbeensystematicallyvalidated.2 Theuseof areliable,validatedscoringsystemforQAisimperativeto limitinconsistenciesandensureobjectivityinmeasuring ultrasoundacquisitionskill.Thevaguelanguageusedinthe ACEPscalemaycontributetovariableinterpretationby thoseassessingstudies,leadingtodiscrepanciesingrading ultrasoundskill.Inconsistentfeedbackmayconfusethe learnerandhindergrowthoftechnicalskill.Inourstudy,we foundthattherewasanincreasedinterobserveragreement betweenreviewerswhenusingthenovel,organ-specificscale whencomparedwiththeACEPgradingscale.Increased consistencyinfeedback,combinedwithdirectedfeedbackto thespecifictargetedorgan,providesanopportunityto enhancelearnereducationandsatisfactionwiththeir ultrasoundeducation.

Organ-specificcardiacandobstetricQAgradingsystems havebeendescribed,althoughtheyhavenotyetbeenwidely adoptedinclinicalpractice.3–5 Thisisthoughttobeduein parttothecomplexityofthesescalesand/orthattheywere validatedoutsidetheED,limitingtheexternalvalidity.3,4,6,7 Wesoughttodevelopascalethatwasconcise,organ-specific, andapplicabletothemostcommonsettinginwhichLUSis performed.Toimprovesuchvaguelanguageas “all structuresimagedwell,” wefoundbenefitinspecifically statingtheanatomiclandmarksneededtomaximize diagnosticimagingineachview.Byemphasizingproper imagingtechniquebeforediagnosticinterpretation,our assessmenttoolmayimproveerrorsinimagegradingand reducelearnerfeedbackvariability.

LIMITATIONS

OurstudywaslimitedbyitsevaluationofaQAexperience atasingle,academictertiary-carecenterinwhichthe validationtookplace.Patientdemographicswerenot collected.Theblindedreviewersalltrained(orcurrent trainees)atthesameclinicalultrasoundfellowshipand, therefore,weretaughttoperformQAusingtheACEP gradingscaleinasimilarmanner.Interestingly,thisperhaps mayhavecontributedtoahigheragreementwiththeACEP scalethanif,alternatively,reviewershadtrainedatdifferent institutions.Further,thescaleitselfwasdevelopedafteran extensivereviewoftheliterature,customizedintoafeasible scalethatisdirectlyapplicabletolearnerobjectives.Assuch, thisscalelackstherigorofalternativemethodological methodssuchasmodifiedDelphianalysis.Importantly,this scaledidnotvalidatewhetherthescorewasrelatedtothe diagnosisoroutcome,orwhetheritimprovedQAefficiency oreducationalfeedback,butratherthedegreeofagreement. Additionally,ourscalefocusesonpathologyrelatedtothe pleurallineitselfanddoesnotincludelanguagetoassessthe abilitytodiagnoseapleuraleffusion.Finally,ourstudy

Score Novel- LUS scale ACEP scale2

Table.

involvedreviewerswithsixmonthsexperienceinQAand includedasmall(100)numberofstudies;consequently,our resultsmaybeunderstated.Furtherresearchiswarrantedto validatethisnovelscale,investigatelearnersatisfaction,and assessitsimpactoneducationalenhancement.

CONCLUSION

Wefoundthatamoreindividualizedqualityassessment scaleofultrasoundimagingtargetedtoaspecificorgan in thiscasethelung resultsinlessgradingvarianceandmore consistent,objectivefeedback.This findingmayhave implicationsonknowledgegainedandlearnersatisfaction. Futurestudiesarewarrantedpriortotheadoptionofthis novelscaleinclinicalpractice.

AddressforCorrespondence:JessicaR.Balderston,MD,VCU MedicalCenter,DepartmentofEmergencyMedicine, 1250E.MarshallSt,Box980401,Richmond,VA23298-0401. Email: Jessica.balderston@vcuhealth.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Baldersonetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.AmericanCollegeofSurgeons’ CommitteeonTrauma Advanced TraumaLifeSupport (ATLS);10th ed.Chicago,Illinois:American CollegeofSurgeons;2018.

2.Ultrasoundguidelines:emergency,point-of-careandclinicalultrasound guidelinesinmedicine. AnnEmergMed.2017;69(5):e27–54.

3.KimuraBJ,GilcreaseGW,ShowalterBK,etal.Diagnostic performanceofapocket-sizedultrasounddeviceforquick-lookcardiac imaging. AmJEmergMed.2012;30(1):32–6.

4.SalomonLJ,NassarM,BernardJP,etal.Ascore-basedmethodto improvethequalityofemergencygynaecologicalultrasound examinations. EurJObstetGynecolReprodBiol 2009;143(2):116–20.

5.DeMasiS,TaylorLA,WeltlerA,etal.Novelqualityassessment methodologyinfocusedcardiacultrasound. AcadEmergMed 2022;29(10):1261–3.

6.RavishankarSM,TsumuraR,HardinJW,etal.Anatomicalfeaturebasedlungultrasoundimagequalityassessmentusingdeep convolutionalneuralnetwork. IEEEIntUltrasonSymp.2021;2021: 10.1109/ius52206.2021.9593662

7.DessieAS,CalhounAW,KanjanauptomP,etal.Developmentand validationofapoint-of-careultrasoundimagequalityassessmenttool: thePOCUSIQscale. AnnEmergMed.2019;74(4):S45–6.

8.GaribyanVN,AmundsonSA,ShawDJ,etal.Theprognosticvalueof lungultrasound findingsinhospitalizedpatientsundergoing echocardiography. JUltrasoundMed.2018;37(7):1641–8.

9.KimuraBJ,LouMM,DahmsEB,etal.Prognosticimplicationsofa point-of-careultrasoundexaminationuponhospitaladmission. JUltrasoundMed.2020;39(2):289–97.

10.KimuraBJandNayakKR. “Asymptomatic” flashpulmonaryedemaby point-of-careultrasound:anovelbedside findingoftransientglobal ischemia. JACCCaseRep.2020;2(10):1545–9.

11.KimuraBJ,ShiR,TranEM,etal.Outcomesofsimplifiedlung ultrasoundexaminCOVID-19:Implicationsforself-imaging. JUltrasoundMed.2022;41(6):1377–84.

12.KimuraBJ,ResnikoffPM,TranEM,etal.Simplifiedlungultrasound examinationandtelehealthfeasibilityinearlySARS-CoV-2Infection. JAmSocEchocardiogr.2022;35(10):1047–54.

13.MaiTV,ShawDJ,AmundsonSA,etal.Learningtoapplythepocket ultrasounddeviceonthecriticallyill:comparingsix “quicklook” signsfor qualityandprognosticvaluesduringinitialusebynovices. CriticalCare 2013;17(5):448.

14.TaylorA,AnjumF,O’RourkeMC.Thoracicandlungultrasound. In:StatPearls[Internet].TreasureIsland,FL:StatPearlsPublishing; 2022.

15.MayoPH,CopettiR,Feller-KopmanD,etal.Thoracicultrasonography: anarrativereview. IntensiveCareMed.2019;45(9):1200–11.

ORIGINAL RESEARCH

DiagnosticAccuracyofaHandheldUltrasoundvsaCart-based Model:ARandomizedClinicalTrial

RyanC.Gibbons,MD*

DanielJ.Jaeger,MD*

MatthewBerger,MD†

MarkMagee,MD*

ClaireShaffer,MD‡

ThomasG.Costantino,MD*

SectionEditor:MarkLangdorf,MD,MHPE

*LewisKatzSchoolofMedicineatTempleUniversity,DepartmentofEmergency Medicine,Philadelphia,Pennsylvania

† CapitalHealth,DepartmentofEmergencyMedicine,Pennington,NewJersey ‡ UniversityofPittsburghMedicalCenterHarrisburg,DepartmentofEmergency Medicine,Harrisburg,Pennsylvania

Submissionhistory:SubmittedJanuary1,2023;RevisionreceivedNovember16,2023;AcceptedNovember22,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.17822

Introduction: Numerousstudieshavedemonstratedtheaccuracyofpoint-of-careultrasound(POCUS). Portable,handhelddeviceshaveexpandedtheclinicalscopeofPOCUSatafractionofthecostof traditional,cart-basedmodels.Thereisapaucityofdataassessingthediagnosticaccuracyofportable devices.Ourobjectiveinthisstudywastocomparethediagnosticaccuracyofaportabledevicewitha cart-basedmodel.

Methods: Thiswasaninstitutionalreviewboard-approved,observational,prospective,randomized clinicaltrial(NCT05196776)ofaconveniencesampleofadultpatientswhopresentedtoauniversitybasedhealthsystem.Patientswhorequiredacardiac,lung,renal,aorta,orbiliaryPOCUSwere randomizedtoaportabledeviceortoacart-basedmodel.Wehypothesizedthatthecart-basedmodel wouldhavea90%diagnosticaccuracyvs70%forthehandhelddevice.Todetecta20%difference,the samplesizewascalculatedtobe98,with49patientsrandomizedtoeacharm.Weusedstandard2x2 tablestocalculatetestcharacteristicswith95%confidenceintervals(CI).

Results: Atotalof110patientswereenrolled,with56patientsrandomizedtothecart-basedmodeland 54tothehandhelddevice.Thesensitivity,specificity,anddiagnosticaccuracyofthecart-basedvs handheldwere77.8%(40–97.2)vs92.9%(66.1–99.8),91.5%(79.6–97.6)vs92.3%(79.1–98.4%),and 89.3%(78.1–96)vs92.5%(81.8–97.9),respectively.

Conclusion: Thediagnosticaccuracyofaportable,handhelddeviceissimilartothatofacart-based model.[WestJEmergMed.2024;25(2)268–274.]

INTRODUCTION

Numerousstudieshavedemonstratedtheaccuracyof point-of-careultrasound(POCUS)todiagnosispathology andtoaugmentproceduralguidance.1–10 Portable,handheld deviceshaveexpandedtheclinicalscopeofPOCUSacross diversesettings,includingprehospital,resource-limited,and outpatientclinics.11–13 Themajorityofexistingliteraturehas assessedthetimelinessandimagequalityofhandhelddevices only.13–15 Todate,thereisapaucityofdataassessingthe

diagnosticaccuracyoftheseportabledevices.16–24 Toour knowledge,therearenorandomizedstudiescomparingthe diagnosticaccuracyofaportable,handhelddevice withatraditionalcart-basedmodel.Ourprimaryobjectivein thisstudywastocomparethediagnosticaccuracy ofthesetwodiagnosticimagingmodalities,specificallyfor cardiac,lung,biliary,renalandabdominalaorta imaging,Secondaryanalysisincludedassessment ofimagequality.

METHODS

StudyDesign

Thiswasaninstitutionalreviewboard-approved, observational,prospective,randomizedclinicaltrial (NCT05196776)withparallelassignmentandanallocation ratioof1∶1.WefollowedtheCONSORTguidelinesand checklistsforclinicaltrials.ButterflyNetwork,Inc.provided fundingforthisstudy.

StudySettingandPopulation

BetweenOctober1–December31,2021weincludeda conveniencesampleofpatients ≥18yearsold,whopresented tooneofthreeclinicallydistinctemergencydepartments (ED)affiliatedwithanurban,LevelI,university-based healthsystemwith >200,000adultandpediatricvisits annually,andwhorequiredacardiac,lung,biliary,renal,or abdominalaortaPOCUSbasedonthediscretionofthe emergencyattendingphysician(EP).Studyinvestigators enrolledpatientscapableofprovidingwritteninformed consent.OurdepartmentcredentialsallEPsinthecore POCUSapplicationsasdefinedbytheAmericanCollegeof EmergencyPhysicians(ACEP).25 AllEnglish-andSpanishspeakingpatientsrequiringaPOCUSevaluationwere eligibleforenrollment.Weexcludedpatients unabletoconsent.

StudyProtocol

Weusedpermuted-blockrandomizationwithan allocationratioof1∶1.Allocationconcealmentincluded sequentiallynumbered,opaque,sealedenvelopes.Upon enrollment,blindedstudyinvestigatorsselectedanenvelope containingstudymaterialsandpre-randomizedselection intothehandhelddevice(HH)orcart-basedmodel(CB) usingResearchRandomizerversion4.0(www.randomizer. org).26 Patients,whorequiredacardiac,lung,renal,aorta,or biliaryPOCUS,wererandomizedtoaportabledevice,the ButterflyiQ(ButterflyNetwork,Inc,Guilford,CT) transducerconnectedtoa fifthgenerationAppleiPadMini

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue?

Point-of-careultrasound(POCUS) enhancesourabilitytosafely,ef fi ciently,and accuratelydiagnoseandmanage ourpatients.

Whatwastheresearchquestion?

DoesahandheldPOCUSdevicehavesimilar diagnosticaccuracyasatraditional, cart-basedmodel?

Whatwasthemajor findingof thestudy?

AhandheldPOCUSdevicehassimilar diagnosticaccuracyasatraditional,cartbasedmodel(sensitivity77.8%vs.92.9%, speci fi city91.5%vs.92.3%andaccuracy 89.3%vs.92.5%).

Howdoesthisimprovepopulationhealth?

Giventhesimilardiagnosticaccuracies, handhelddevicesbroadentheavailabilityof POCUSandenhancepatientcareinresourcelimitedsettings.

(AppleInc,Cupertino,CA),ortoacart-basedmodel,theGE VenueGoorGELogiqE(GEHealthCare,Wauwatosa, WI).(Referto Image.)Westudiedthe fivemostcommonly performedPOCUSscansinourdepartment.

Usingthephasedarraytransducer(2–5mHz)forcardiac imagingorthecurvilineartransducer(1-mHz)forthelung, renal,aortic,andbiliaryscans,postgraduateyear1–3 emergencymedicine(EM)residentsperformedeachPOCUS priortoadvancedimaging.Performingphysiciansusedthe

Image. HandheldButterflyiQdeviceandcart-basedGEVenueGomodeldemonstratingparasternallongaxisview.

Gibbonsetal. HandheldUltrasoundsvsCart-basedModel

correspondingsettingsfortheHHdevice.AnattendingEP, credentialedinthecoreACEPPOCUSapplications, reviewedeachstudyconcurrently.Studyinvestigators blindedallresidentsperformingthescansandtheattending EPsreviewingthemtothestudyobjectiveanditsfunding.

Acardiologist-interpretedechocardiogram,performed within24hoursofpresentationtotheED,servedasthe referencestandardforcardiacimages.Forbiliarytract images,thereferencestandardwasaradiology-interpreted ultrasoundperformedduringtheEDvisit.Forlung,renal andaorticscans,thereferencestandardwascomputed tomographyimages(whenavailableandperformedduring theEDvisit),orPOCUSqualityassurance(QA)reviewby twoultrasoundfellowship-trainedphysicians(whennoCT wasavailable).Iftherewasdisagreement,athirdultrasound fellowship-trainedphysicianprovidedaninterpretation.The cardiologist,radiologist,andtheultrasoundfellowshiptrainedEPswereblindedtothereal-timePOCUSreads. However,theEPsperformingQAknewaboutthestudyand itsfunding.

Priortostartingtheirinternship,ourEMresidents participateinanintroductory five-hourIntroductionto POCUScoursetaughtbyouremergencyultrasoundfaculty. Additionally,eachresidentcompletesathree-week emergencyultrasoundrotationduringtheirinternshipin accordancewithAccreditationCouncilforGraduate MedicalEducation(ACGME)andACEPguidelines.24,27 Residentsreceivednoadditionaltrainingpriortotheirstudy participation.Nonetheless,eachparticipantcompletedmore than25ofeachscanpriortoparticipatinginthestudyto achievecompetencyperACEPandAGMEguidelines.24,27

Measurements

Priortostudycommencement,wedefinedthefollowing diagnosticendpoints:ejectionfraction(EF)(good >50%, moderate30–50%,poor <30%)andthepresenceorabsence ofthefollowing:gallstones;hydronephrosis(mild,moderate, orsevere);abdominalaorticaneurysm(>3centimeters),and B-lines(≥3inasinglelung fieldorasingle,confluentB-line occupying >1/3oftheintercostalwindow).28 Thepresenceof B-linesindicatesaninterstitialprocess,whetherlocalizedor diffuse,reflectsitsetiology.Wecomparedthistointerstitial findingsonCT(ifavailable)ofthecorrespondinglobe.We didnotcompareadditionalmeasurements(ie,gallbladder wallthickness,orassessM-modeorDoppler findings).The studyincludedB-mode findingsonly.Usingtheelectronic healthrecord(EpicSystemsCorp,Verona,WI),we performedchartabstractiononallpatientstoobtainresults ofcardiology-interpretedechocardiogramsandradiologyinterpretedultrasoundandCTstudies.

Diagnosticaccuracyofeachimagingmodalitycompared totheaforementionedgoldstandardsservedastheprimary endpoint.Imagequalityservedasthesecondaryendpoint. Threeultrasoundfellowship-trainedphysiciansuseda

previouslyvalidatedLikertscaletoassessimagequality.29 Ascoreof1indicatedunabletointerpret,andascoreof7 specifiedsuperiorimagingquality.

StatisticalAnalysis

PriorstudiesassessingPOCUSperformedusing traditionalCBtechnologyhavedemonstratedthefollowing sensitivitiesforrespectivepathologies:EF(89%); cholelithiasis(94%);abdominalaorticaneurysms(97%);Blines(92%);andhydronephrosis(75%),providinganaverage sensitivityof90%.28,30–37 Giventhelackofpre-existingdata comparingthemodalities,wehypothesizedthattheHH devicewouldhaveanoverallsensitivityof70%.We postulatedthattheHHwouldbeinferiorgiventhesmaller screensize,noveltechnologytogeneratesonographic images,andlimitedclinicianexperiencewiththedevice. Basedonapowerof80%andanalphaof0.05,wecalculated asamplesizeof98,with49patientsrandomizedtoeacharm, todetecta20%difference.Wereportcontinuousand categoricaldataasmedianswithinterquartileranges(IQR) orproportionswith95%confidenceintervals(CI),andwe usedstandard2 × 2tablestocalculatetestcharacteristics with95%CIsusingMedCalcversion19.1.6(MedCalc SoftwareLtd,Ostend,Belgium).Intraclasscorrelation coefficientassessedinter-raterreliabilitybetweenblinded expertreviewers,andweusedthe t -testtocomparemedian Likertscores.

RESULTS

Weenrolled110patientswith56patientsrandomizedto theCBmodeland54totheHHdevice(Figure1).Authors excludedoneHHpatientgiventherewerenosonographic imagesavailabletoreview. Table1 illustratesthesimilarity ofpatientcharacteristicsandthenumberofeachPOCUS typeacrossbothcohorts(Table1). Table2 portrays testcharacteristicsforeachdiagnosticmodality,while

Figure1. Patient flowchart. CB,cart-basedultrasoundmodel; HH,handhelddevice; POCUS, point-of-careultrasound; QA,qualityassurancereview; NCCT,noncontrastcomputedtomographyoftheabdomenandpelvis.

HandheldUltrasoundsvsCart-basedModel Gibbonsetal.

Gibbonsetal. HandheldUltrasoundsvsCart-basedModel

CharacteristicCart-basedmodel(n = 56)Handhelddevice(n = 53)

Age,median(IQR),years57(18–90)60(18–89)

Gender,N(%)

Female60.751

Male39.349

Bodymassindex,median(IQR)30(22–64.9)27.9(15–42.2)

Point-of-careultrasoundscansAorta(9)Aorta(4)

Cardiac(11)Cardiac(17)

Gallbladder(10)Gallbladder(14)

Lung(11)Lung(7)

Renal(15)Renal(11) IQR,interquartilerange.

Cart-basedmodel(n = 56;95%CI)Handhelddevice(n = 53;95%CI)

Sensitivity77.8(40–97.2)92.9(66.1–99.8)

Specificity91.5(79.6–97.6)92.3(79.1–98.4)

Positivelikelihoodratio9.2(3.4–24.9)12.1(4.0–36.2)

Negativelikelihoodratio0.2(0.1–0.8)0.1(0–0.5)

Diseaseprevalence0.20.3

Positivepredictivevalue63.6(39.2–82.6)81.3(59.1–92.9)

Negativepredictivevalue95.6(86.3–98.7)97.3(84.5–99.5)

Accuracy89.3(78.1–96)92.5(79.3–96.9)

CI,confidenceinterval.

Tables3 and 4 depictthediagnosticcriterionreferenceused andthediagnosticinaccuracies,respectively.

Overall,therewere10incorrectdiagnoses,fourforthe HHdeviceandsixfortheCBmodel. Table4 highlightsthe diagnosticinaccuraciesbyscantype,diagnosticmodality, andcriterionreference.TheHHcorrectlyidentifiedthe following:sixinstancesofcholelithiasis;onecaseofmildand oneofmoderatehydronephrosis;fourindividualswith pulmonaryedema;andonepatientwithamoderateEF.The CBmodalitycorrectlyidentifiedthefollowing:twoinstances ofcholelithiasis;onecaseofseverehydronephrosis;two individualswithpulmonaryedema;andtwopatientswith poorEFs.ThemedianLikertscoreforCBwas5,and4 fortheHH.Intraclasscorrelationcoefficientsforthe HHandCBwere0.5(95%CI0.2–0.7)and 0.8(95%CI0.7–0.8),respectively.

DISCUSSION

Tothebestofourknowledge,oursisthe firstpublished randomizedtrialcomparingaportableHHdevicewitha

traditionalCBmodelinEDpatients.Giventhelackofpreexistingdata,wehypothesizedthatthetraditionalCBmodel wouldbesuperiorwithrespecttodiagnosticaccuracyand imagequality.Handhelddevicesarestillnovelandhavenot beenadoptedbroadly,limitingclinicianexperience. Moreover,noveltechnologytogeneratesonographic images,comparedtothetraditionalpiezoelectriccrystals, mayaffectimagequalityaswell.Similarly,weassumed screenresolutionandsizewouldlimitimagequalityand, subsequently,accuracy.However,asmallpilotstudyby MageeetaldemonstratedsimilarresultsbetweenHHand CBdeviceswheninterpretatingpre-recordedvideos assessingforfree fluidintherightupperquadrant.13

Wechose fivebasicPOCUSexaminationsthatourEPs haveconsiderableexperienceperformingwithappropriate diagnosticaccuracy.OurEPscurrentlyhavelessexperience withotherPOCUSindications,suchasregionalanesthesia andfractureassessment.Moreover,wedidnothaveaccessto aHHendocavitarytransducertoassessforpregnancyrelatedissues.Theseareasareripeforfutureresearch.

Table1. Patientcharacteristics.

Table2. Testcharacteristics.

Cart-based model(n = 56)

Handhelddevice (n = 53)

QAEchoQAEcho

Cardiac(n = 27)5688

QARUQUSQARUQUS

Biliary(n = 24)100104

QACTQACT

Lung(n = 18)11052

Renal(n = 27)13293

Aorta(n = 13)6304

QA,qualityassurancereview; Echo,cardiology-performedand interpretedechocardiography; RUQUS,radiology-performed andinterpretedrightupperquadrantultrasound; CT,computed tomographyofthechestorabdomenandpelviswithor withoutcontrast.

Overall,wefoundnosignificantdifferenceinsensitivityor specificitybetweenCBandHHultrasoundimages. However,thislimitedoursamplesizeforeachmodality.

Althoughthestudytypesandindicationsvaried,theidea ofdiagnosticaccuracyshouldapplytoallPOCUSstudies.It isprobablyexpectedthatwhenthediagnosiswasthe objectivepresenceorabsenceofa finding,(ie,gallstones) therewerenomisdiagnoses.29,30 However,whenthe diagnosiswasmoresubjective(ie,estimatingEForthe degreeofhydronephrosis)thereweremoreinaccuracies acrossbothmodalities.Thisisconsistentwithprevious studiesshowingmoreoverlapofgoodandmoderateEFsand betweenpoorandmoderate.31 Inourstudy,therewasa tendencytooverestimatethepresenceordegreeof hydronephrosis,whichislikelyconfirmationalbiasinthe settingofapresumednephrolithiasisdiagnoses.

Asexpected,theCBdevicehadbetteroverallimage qualitythantheHH.However,thisdidnotaffectdiagnostic accuracy,asourresultssuggestthatitissimilarbetweenHH andCBmodalitiesinanacademicEMresidency.Superior imagequalitymaydetectmoresubtlepathology,suchas signsofcholecystitis.32 Eachdiagnosticmodalityservesa clinicalrole.Thisdatacanbeextrapolatedtothebroader EMcommunitywiththeincreasingprevalenceofultrasound competencyinpracticingEPsandavailabilityofportable devices.Furthermore,itsupportstheutilityofHHdevicesin resource-limitedsettings,outpatientclinics,andinpatient locationswithlimitedaccesstotraditionalsonographic machines,nottomentionpandemicsettingswhere disinfectionisparamount.2

LIMITATIONS

Thisstudysuffersfromthelimitationsofanobservational designwithconveniencesamplingatasinglehealthsystem resultinginaselectionbiasaswellasasmallersamplesize, whichlimitsthelevelofprecisiontoexcludeatypeIIerror. UsingthediscretionoftheattendingEPtodetermine whetherapatientneededaspecificPOCUSexamination createdaselectionbiasaswell.Wedidnotdefinespecific indicationstoperformoneoftheaforementionedPOCUS scans.Moreover,wehypothesizedthediagnosticaccuracyof theHHdevicegiventhelackofpre-existingdata.Thislimits thevalidityofourpoweranalysis.

ButterflyNetwork,Inc.fundedthestudy,whichmayhave introducedbias.However,physiciansperformingthe ultrasoundswereunawareofthisfunding.Furthermore, physiciansperformingtheultrasoundhadsignificantlymore experienceusingtheCBmodelcomparedtotheHHdevice, whichmayhaveintroducedbiasinfavorofthetraditional modality.Furthermore,wedidnotaccountforthe

Cart-basedmodel(n = 6)Handhelddevice(n = 4)

Cardiac(n = 5)2interpretedasnormalEF,readasmoderateduringQA1interpretedasnormalEF,readasmoderateduringQA 1interpretedasmoderateEF,readasnormalduringQA 1interpretedaspoorEF,readasnormalonecho

Biliary(n = 0)00

Lung(n = 0)00

Renal(n = 4)2interpretedasmildhydronephrosis,readasnormal duringQA

1interpretedasmoderatehydronephrosis,readas normalduringQA

1interpretedasmoderatehydronephrosis,readas normalonNCCT 1interpretedasmildhydronephrosis,readasnormalon NCCT

Aorta(n = 0)00

EF,ejectionfraction; QA,qualityassurancereview; Echo,cardiologyperformedandinterpretedechocardiography; RUQUS,radiology performed-andinterpretedrightupperquadrantultrasound; NCCT,non-contrastcomputedtomographyoftheabdomenandpelvis; CB, cart-basedmodel; HH,handhelddevice.

Table3. Diagnosticcriterionreferenceusedforcomparison.

Table4. Diagnosticinaccuraciesbyimagingmodality.

experienceleveloftheresidentsperformingtheultrasound, whichcouldhaveimpactedqualityandaccuracy.

Presumably,seniorresidentshadmoreproficiency.

WedidnotcompareadditionaltypesofHHdevices.

Therefore,itisunclearwhetherourdataisapplicableto otherdevicesusingdifferenttechnology.Specifically,the ButterflyiQdeviceuseschiptechnologycomparedto traditionalpiezoelectriccrystals.Thismayimpactimage qualityanddiagnosticaccuracy.Presumably,theHH frequenciessettingsforeachstudyreflectthoseofthe traditionalCBmodalities.However,wedidnotaccountfor softwarefeatures,screensize,orresolutioninourstudy. Futurestudiesneedtovalidateour findingsacrossthearray ofHHdevicesandemergingtechnology.Furthermore,we limitedourstudytoonly fiveoftheACGMEcoreultrasound competencies.Therefore,additionalstudiesareneededto validateour findingstobroaderPOCUSapplications, includingvarioussettingssuchasM-modeandDoppler.

Usingthesubjectiveinterpretationofultrasound fellowship-trainedfacultyasthecriterionreferencewhen otherstandarddiagnosticsimagingmodalitieswerenotdone limitsthevalidityoftheresultsandcausesamisclassification bias.Specifically,wedidnotaccountforthepotentialfor inferiortechnology.Forexample,iftheHHorCBmodel providesinferiorimaging,notonlymaytheperforming physicianmisspathology,buttheEPsconductingQAmay overlookitaswell.Thisfalsenegativemaynotbemissedbya radiology-performedandinterpretedultrasound.Moreover, reviewerswerenotblindedtotheimagesource,HHvsCB, giventhateachmodalityusesuniquestoragemeans. Nonetheless,qualityassurancereviewiscommonpracticein academicEDswithanultrasounddivision,andconfirmatory studiesaretypicallyunnecessary.

Additionally,usingcardiologist-obtained echocardiogramsasareferencestandardintroducesthe potentialfortreatmenteffectsbetweenwhenthePOCUS imageswereobtainedandwhenthecardiologyimageswere obtained.Whileeachpatientreceivedacardiology echocardiogramwithin24hoursoftheEDvisittolimitsuch effects,thisisnonethelessalimitationtoourstudy.Finally, ourEDisnotrepresentativeofthebroaderEMcommunity. Wehaveanactiveultrasounddivisionwithnumerousfaculty andfellows.AllEPsarecredentialedinPOCUS.Inour department,residentsarethetreatingclinicians,who typicallyhavemorePOCUSexperiencecomparedtomost practicingEPs.Furthermore,ourdepartmenthasregular accesstoandexperiencewithportabledevices.

CONCLUSION

Thediagnosticaccuracyofaportable,handheld ultrasounddeviceissimilartotheaccuracyofatraditional, cart-basedmodelwhenperformingcardiac,lung,biliary, renal,orabdominalaortastudies.Futurelarger,multicenter studiesarerequiredtovalidatethese findings.

AddressforCorrespondence:RyanC.Gibbons,MD,LewisKatz SchoolofMedicineatTempleUniversity,DepartmentofEmergency Medicine,1017JonesHall,1316WestOntarioStreet,Philadelphia, PA19140.Email: ryan.gibbons@tuhs.temple.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.ThisstudywasfundedbyButterfly Network,Inc.Thecompanyhadnoroleinconductionofthestudy, themethodofreportingordecisiontopublishthestudy.

Copyright:©2024Gibbonsetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.GibbonsRC,ZanaboniA,GenningerJ,etal.Ultrasound-versus landmark-guidedmedium-sizedjointarthrocentesis:arandomized clinicaltrial. AcadEmergMed. 2022;29(2):159–63.

2.GibbonsRC,MageeM,GoettH,etal.Lungultrasoundvs.chestX-ray studyfortheradiographicdiagnosisofCOVID-19pneumoniainahighprevalencepopulation. JEmergMed. 2021;60(5):615–25.

3.GibbonsRC,ZanaboniA,SaravitzSM,etal.Ultrasoundguidance versuslandmark-guidedpalpationforradialarteriallineplacementby noviceemergencymedicineinterns:arandomizedcontrolledtrial. J EmergMed. 2020;59(6):911–7.

4.BeckerBA,LahhamS,GonzalesMA,etal.Aprospective,multicenter evaluationofpoint-of-careultrasoundforsmall-bowelobstructioninthe emergencydepartment. AcadEmergMed. 2019;26(8):921–30.

5.Smith-BindmanR,AubinC,BailitzJ,etal.Ultrasonographyversus computedtomographyforsuspectednephrolithiasis. NEnglJMed. 2014;371(12):1100–10.

6.CostantinoTG,SatzWA,DehnkampW,etal.Randomizedtrial comparingintraoralultrasoundtolandmark-basedneedleaspirationin patientswithsuspectedperitonsillarabscess. AcadEmergMed. 2012;19(6):626–31.

7.BernardiE,CamporeseG,BüllerHR,etal.Serial2-point ultrasonographyplusD-dimervswhole-legcolor-codedDoppler ultrasonographyfordiagnosingsuspectedsymptomaticdeepvein thrombosis:arandomizedcontrolledtrial. JAMA. 2008;300(14):1653–9.

8.CostantinoTG,ParikhAK,SatzWA,etal.Ultrasonography-guided peripheralintravenousaccessversustraditionalapproachesinpatients withdifficultintravenousaccess. AnnEmergMed. 2005;46(5):456–61.

9.CostantinoTG,BrunoEC,HandlyN,etal.Accuracyofemergency medicineultrasoundintheevaluationofabdominalaorticaneurysm. J EmergMed. 2005;29(4):455–60.

10.BlaivasM,TheodoroD,SierzenskiPR.Astudyofbedsideocular ultrasonographyintheemergencydepartment. AcadEmergMed. 2002;9(8):791–9.

11.ColliA,PratiD,FraquelliM,etal.Theuseofapocket-sizedultrasound deviceimprovesphysicalexamination:resultsofanin-andoutpatient cohortstudy. PLoSOne. 2015;10(3):e0122181.

12.ShorterMandMaciasDJ.Portablehandheldultrasoundinaustere environments:useintheHaitidisaster. Prehosp&DisastMed. 2012Apr;27(2):172–7.

13.MageeMA,AuAK,FlashnerL,etal.Theeffectofdisplaysizeon ultrasoundinterpretation. AmJEmergMed. 2022;51:285–9.

14.SabbadiniL,GermanoR,HopkinsE,etal.Ultrasoundhypotension protocoltime-motionstudyusingthemultifrequencysingletransducer versusamultipletransducerultrasounddevice. WestJEmergMed. 2021;22(3):775–81.

15.DewarZE,WuJ,HughesH,etal.Acomparisonofhandheldultrasound versustraditionalultrasoundforacquisitionofRUSHviewsinhealthy volunteers. JAmColEmergPhysOpen. 2020;1(6):1320–5.

16.RykkjeA,CarlsenJF,NielsenMB.Hand-heldultrasounddevices comparedwithhigh-endultrasoundsystems:asystematicreview. Diagnostics. 2019;9(2):61.

17.LeggettCB,NaqviM,EsakoffTF,etal.Incorporatingpersonal-devicebasedpoint-of-careultrasoundintoobstetriccare:avalidationstudy. AmJObsGyn. 2022;226(4):552.e1–e6.

18.VourvouriEC,PoldermansD,SchinkelAF,etal.Abdominalaortic aneurysmscreeningusingahand-heldultrasounddevice. “apilot study” EurJVascEndovascSurg. 2001;22(4):352–4.

19.Keil-RíosD,Terrazas-SolísH,González-GarayA,etal.Pocket ultrasounddeviceasacomplementtophysicalexaminationforascites evaluationandguidedparacentesis. InternEmergMed. 2016;11(3):461–6.

20.KimuraBJ,GilcreaseGW,ShowalterBK,etal.Diagnosticperformance ofapocket-sizedultrasounddeviceforquick-lookcardiacimaging. AmJ EmergMed. 2012;30(1):32–6.

21.BobbiaX,ChabannonM,ChevallierT,etal.Assessmentof fivedifferent probesforlungultrasoundincriticallyillpatients:apilotstudy. JEmerg Med. 2018;36(7):1265–9.

22.AmedaT,UebayashiK,WagaiK,etal.Assessmentoftherenal collectingsystemusingapocketsizedultrasounddevice. JMed Ultrasonics. 2018;45(4):577–81.

23.LauBC,MotamediD,LukeA.Useofpocket-sizedultrasounddevicein thediagnosisofshoulderpathology. ClinJSportsMed. 2020;30(1):20–4.

24.PujolS,LaurentJ,MarkarianT,etal.Compressionwithapocket-sized ultrasounddevicetodiagnoseproximaldeepveinthrombosis. AmJ EmergMed. 2018;36(7):1262–4.

25.AmericanCollegeofEmergencyPhysiciansPolicyStatement. UltrasoundGuidelines:Emergency,Point-of-care,and ClinicalUltrasoundGuidelinesinMedicine.2001.Availableat:

https://www.acep.org/patient-care/policy-statements/ultrasoundguidelines-emergency-point-of–care-and-clinical-ultrasoundguidelines-in-medicine/.AccessedDecember1,2022.

26.UrbaniakGCandPlousS.ResearchRandomizer(Version4.0) [Computersoftware].Availableat: http://www.randomizer.org/ AccessedOctober1,2021.

27.AccreditationCouncilofGraduateMedicalEducationEmergency MedicineDefinedKeyIndexProcedureMinimums.Availableat: https:// www.acgme.org/Portals/0/PFAssets/ProgramResources/ EM_Key_Index_Procedure_Minimums_103117.pdf?

ver=2017-11-10-130003-693.AccessedDecember1,2022.

28.ChiuL,JairamMP,ChowR,etal.Meta-analysisofpoint-of-carelung ultrasonographyversuschestradiographyinadultswithsymptomsof acutedecompensatedheartfailure. AmJCardiol. 2022;174:89–95.

29.BlaivasM,DeBehnkeD,SierzenskiPR,etal.Tissueharmonicimaging improvesorganvisualizationintraumaultrasoundwhencomparedwith standardultrasoundmode. AcadEmergMed. 2002;9(1):48–53.

30.KendallJLandShimpRJ.Performanceandinterpretationoffocused rightupperquadrantultrasoundbyemergencyphysicians. JEmerg Med. 2001;21(1):7–13.

31.MillerAH,PepePE,BrockmanCR,etal.EDultrasoundinhepatobiliary disease. JEmergMed. 2006;30(1):69–74.

32.MooreCL,RoseGA,TayalVS,etal.Determinationofleftventricular functionbyemergencyphysicianechocardiographyofhypotensive patients. AcadEmergMed. 2002;9(3):186–93.

33.SummersSM,ScruggsW,MenchineMD,etal.Aprospectiveevaluation ofemergencydepartmentbedsideultrasonographyforthedetectionof acutecholecystitis. AnnEmergMed. 2010;56(2):114–22.

34.RossM,BrownM,McLaughlinK,etal.Emergency physician–performedultrasoundtodiagnosecholelithiasis:a systematicreview. AcadEmergMed. 2011;18(3):227–35.

35.MarbachJA,AlmuflehA,DiSantoP,etal.Comparativeaccuracyof focusedcardiacultrasonographyandclinicalexaminationforleft ventriculardysfunctionandvalvularheartdisease:asystematicreview andmeta-analysis. AnnInternMed. 2019;171(4):264–72.

36.FernandoSM,TranA,ChengW,etal.Accuracyofpresenting symptoms,physicalexamination,andimagingfordiagnosisofruptured abdominalaorticaneurysm:systematicreviewandmeta-analysis. Acad EmergMed. 2022;29(4):486–96.

37.RiddellJ,CaseA,WopatR,etal.Sensitivityofemergencybedside ultrasoundtodetecthydronephrosisinpatientswithcomputed tomography-provenstones. WestJEmergMed. 2014;15(1):96–100.

EDUCATIONAL ADVANCES

SpaceUltrasound:AProposalforCompetency-basedUltrasound TrainingforIn-flightSpaceMedicine

ChanelFischetti,MD*

EmilyFrisch,MD†

MichaelLoesche,MD,PhD*

AndrewGoldsmith,MD,MBA*

BenMormann,MD*

JosephS.Savage,MD‡ RogerDias,MD,PhD*

NicoleDuggan,MD*

SectionEditor:MarkLangdorf,MD,MHPE

*BrighamandWomen’sHospital,HarvardMedicalSchool,Departmentof EmergencyMedicine,Boston,Massachusetts

† ClevelandClinic,DepartmentofObstetricsandGynecology,Cleveland,Ohio ‡ YaleBridgeportHospital,DepartmentofEmergencyMedicine, Bridgeport,Connecticut

Submissionhistory:SubmittedAugust8,2023;RevisionreceivedNovember7,2023;AcceptedDecember6,2023

ElectronicallypublishedFebruary13,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18422

Spacetravelhastransformedinthepastseveralyears.Giventheburgeoningmarketforspacetourism, in-flightmedicalemergenciesarelikelytobeexpected.Ultrasoundisoneofthefewdiagnosticand therapeuticmodalitiesavailableforastronautsinspace.However,whilepoint-of-careultrasound (POCUS)isavailable,thereisnocurrentstandardoftrainingforastronautpreparation.Wesuggestan organizedandstructuredmethodologybywhichastronautsshouldbestprepareforspacewiththe medicalequipmentavailableonboard.Astechnologycontinuestoevolve,theassistanceofother artificialintelligenceandaugmentedrealitysystemsarelikelytofacilitatetraininganddynamicreal-time needsduringspaceemergencies.

Summary: Asspacetourismcontinuestoevolve,anorganizedmethodologyforPOCUSuseisadvised tobestprepareastronautsforspace.[WestJEmergMed.2024;25(2)275–281.]

BACKGROUND

Overthelastdecade,impressivetechnologicaladvances surroundingspacetravelhavemadespacetourismareality forthenot-too-distantfuture.Ascommercialindustry increasinglylaysdownastakeinthisnascentmarket,prior barrierstoprivate-passengerspacetravel,suchascostand accessibility,arerapidlyfalling.1,2 Proposedopportunities forprivatepassengersrangefromlong-haulglobaltravel throughspace,tobrieforbit,toprolongedstaywithhotel accommodations.3 Asoflate2021,lessthanahandfulofthe nearly600individualswhohavegonetospacewerecivilian passengers. 4 Despitesmallnumberscurrently,itisprojected thatby2030bothspacetourismandlong-haultravel byspacewillcapturenearly$20billionofthelarger spaceeconomy. 4

Professionalastronautsareoftenscreenedforbaseline healthconditionsthatcouldleadtoin-flightmedical emergenciesandpotentiallyjeopardizepersonnelsafetyor

themission.5 Therefore,truein-flightmedicalemergenciesto datehavebeenrare.However,withthegreaterdiversityof traveleranatomy,physiology,andmedicalhistory,which willinevitablyresultfromexpansionofprivate-passenger spacetravel,asignificantincreaseinin-flightmedical emergenciesisexpected.1,2,5 Forlongerdurationmissions,it isprojectedthatatleastonemedicalemergencywilloccur percrewofsixtravelers.5 Unlikeformedicalemergencies duringairtravel,emergencylandingsandreal-time conversationswithgroundcontrolarenotreliableoptionsin space.6,7 Additionally,asmore flightsdepartitwillbe increasinglyunlikelythatatrainedmedicaldoctor willbeavailableorpresentoneach flight.Infact,SpaceX justrecentlylaunchedanall-civilianmissioncrewwith onlyatrainedphysicianassistant.8 Thus,withthe expansionoftheprivatespace flightindustry,innovative medicalprotocolsandapproachesmust bedeveloped.9

OBJECTIVES

Point-of-careUltrasoundTrainingforSpaceMedicine CurrentTrainingStandards

Priortocurrentspacetravel, flightcrewsarerequiredto trainforanticipatedmechanical,mission,andengineering challenges.3 Medicalcareistheresponsibilityofthecrew medicalofficer(CMO)whotypicallyhaslimitedprior medicalknowledge.10 TheCMOtraininginvolves40–80 hoursofhands-ontrainingwithremediationandcontinuing virtualtrainingsasneeded.10 Someofthispreparatory trainingincludesrudimentarymedicaleducation (phlebotomy,vitalsignmeasurement,tonopenuse,panoptic use,andultrasound)11 designedinanticipationof coordinatedcarewithMissionControlfortelehealth interpretations.11 Thisoftenconsistsof “just-in-time” diagnosticalgorithmstofacilitateultrasoundinterpretation withtheaidoflivetelehealthguidance.10 Flightsurgeonsare frequentlyandregularlyonconsoleatFlightControlRoom 1-MissionControlandactivelyparticipateinmedical monitoringandguidance.

Point-of-careultrasound(POCUS)imagescanbe downloadedinrealtimeforevaluation,andbyusinga privatemedicalconferencechannelloop,onlytheultrasound operator(trainedandundernon-disclosureagreement),the physician,andthepatient/subjectareinvolved.Eventhe mission’ s flightdirectorwouldnothaveaccess.Inherently, telemedicinehasbeenapartoftheInternationalSpace Station(ISS)sinceitlaunched.Aslongerspaceduration missionsandinterplanetarytravelprogress,timelapsesof40 minutesorlongerareanticipatedforgroundcrewvirtual contact.6,7,12,13 Thesecommunicationdelayscouldleadto severemedicalconsequencesformissionswith flightcrew trainedaccordingtothecurrentstandardofcare.6,7,12

Becauseofitsportability,low-cost,andradiation-free, real-timeimagingforanimpressivearrayofmedical conditions,POCUShasademonstratedutilityinspace medicine,Incaseswhereultrasoundtrainingiscurrently provided,amaximumof2-3hoursisallottedthroughoutthe entirepre-flighttrainingcurriculum.11,14 ThisAdvanced DiagnosticUltrasoundinMicrogravity(ADUM) educationalprogramisusedontheISSwhere “cuecards” areusedtorapidlyguidenon-expertuserstoperform ultrasoundsonpatients,withmorethan90%accuracyafter justminutesoftraining.14 While “cuecards” canbeused, ADUMhasfoundthatnon-medicaloperatorscanobtain qualitydatawiththerightamountoftrainingand direction.14 Forthisreason,anon-boardproficiency enhancementhasalsobeencreatedbothinEnglishand Russian.14 Thecombinationofthisseveral-hourtraining coursewiththe “remoteexpertguidance” (availableby MissionControl)isthemosteffectivemeansbywhich in-flightultrasoundguidanceiscurrentlyconducted withattentiontolimitationsofultrasoundinspace(geluse, devicebatterylife,etc).14

Incontrast,truemasteryofPOCUSforhealthcare professionalstypicallyrequiresyearsofpracticeduring medicalresidencyandoftenanadditionalyearofdedicated trainingthroughanultrasoundfellowship.Whilemasteryof POCUSatthesamelevelofamedicalprofessionalisnot realisticformost flightsurgeontraining,introducingPOCUS tocrewmembersanda flightsurgeon’srepertoirethrougha structuredandsystematiccurriculumhasthepotentialto yieldsignificantbenefittobothprivatepassengersand potentiallytheentiretyofthemission.Additionally,inlonger duration flightmissionswhenemergencydecisionsneedtobe madeusingADUM’sproposedtelecommunicationand ultrasoundvideotransmission,timeandvideodelays haverealandsignificantlimitationsforastronautcare andoutcomes.

CURRICULARDESIGN

ProposedPoint-of-careUltrasoundTrainingSolutions

PriordataonPOCUSeducationsuggeststhatevennovice POCUSlearnerscanretainthebasicsofimageacquisition andinterpretationwithaminimumamountoffocused training.14–18 Corecompetencyinscanningeachorgan systemcanbeachievedwithatwo-hoursessionofcombined didacticsandhands-onscanning.19–22 Thus,asaconsortium ofmedicaldoctorsandexperts,weproposeastructured, competency-basedPOCUScurriculumforcommercialspace travelthatincludeswell-definedaimstargetingimage acquisitionandinterpretationforthemostcommonorgan systemsinvolvedinin-flightmedicalemergencies (Figure1A).Astronautstrainedforspaceshouldbe consideredtechniciansinthesescenarios,withphysicians supplementingthereal-timediagnosesandtreatments.

Thesevenmosthigh-yieldprocedureororgan-based systemsareidentifiedwithanadvised1–2hourstrainingper topic.Realistically,aone-daycourseofabout6–8hours wouldbesufficienttosatisfyafoundationforcompetency. However,aspre-missionastronautpreparationtimeisbusy and filledwithrequirements,thesepreparatorycoursescan beadjustedandelongatedastoleratedbyindividualmission schedulesandneeds.Thisstructuredformatwouldensure consistentandhomogenoustrainingforallastronauts anticipatedinspace.Eachaspectofthemissionisrehearsed, andeachastronaut(andback-upastronaut)isalsocrosstrainedforactivitiesoutsidetheirprimarymission designationscope,intheeventofastronautdrop-out. Trainingwouldbemission-specificandexpectedtobe intensiveandstartaboutthreeyearsbeforelaunch.Realtimeupdatescanbemadeifmissionrequirementschange atanypointwithinthethreeyearstolaunch,sothatthe mostup-to-dateequipmentandproceduresareused priortolaunch.

Corecompetencyisanappropriategoalformost flight surgeonsintrainingandshouldincludethebasicskills neededforPOCUSimageacquisitionandaproficientlevel

ofindependentinterpretation.Althoughfarfrommastery level,corecompetencyallowsforanappropriatebalanceof limitedinputoftrainingtimerequired.TheskillsofPOCUS acquisitionandinterpretationcanalwaysbesupplemented withadjuvanttoolssuchaslivetelehealthwithground

control,or(artificialreality/artificialintelligence[AI])tools duringlivemissions(Figure1B).Fortravelatloweraltitudes oforbit,lowertieredcompetencycoupledwithavailable telehealthguidancemaybesufficient.23–25 Forlong-haulor deeperspacetravel,however,completingtheentire

Figure1A. Corepoint-of-careultrasoundcompetenciesinastructuredultrasoundtrainingprogram.

Figure1B. Suggestedtieredcompetency-basedultrasoundtraining.ImagesfromtheSTRATUSSpaceSimulationtraining.

Fischettietal.

Physiological systemMedicalevents

Ocularand sensory organs

• Ocularforeignbody

• Increasedintracranial pressure

• Disequilibrium

Incidencein space (%reported)Pathophysiology

Upto42%32 Foreignbodiesfromexposureswithin thespacecapsuleororbit

CardiacArrhythmias0.2–9.55%Shifting fluidsanddynamicchangesin gravitationalmovementscancause compensatorychangesinboth pulmonaryandcardiacvolumesand potentiallyprovokingcardiac arrhythmiasandcardiacirritability33,34

Pulmonaryembolism***Lackofgravityandvenousstasisin spacecanpromotethromboticevents

Ultrasoundindication/ POCUS finding

• Identifyocularforeignbodies

• Measureopticnervesheath diameter

• Measureopticnervesheath diameter

Transthoracicechofor arrhythmias,wallmotion abnormalities,orcardiac standstill

Transthoracicechoforright ventricularstrain

Pulmonary

• Pneumothorax

• Respiratoryinfections

VascularVenous thromboembolism

GastrointestinalBowelobstructionand constipation

Genitourinary

• Acuteurinaryretention

• Renalstones

7.6–64%35 BarotraumaConfirmlungsliding Dysregulationoftheimmunesystem withpossibleconcurrentviral reactivation36

***37,38 Lackofgravityandvenousstasisin spacecanpromotethrombotic events31

***39 Constipationiscommoninspacebut symptomscanmimicbowel obstruction

~1.20%32

Oftenmultifactorial,pharmacologic, lossofgravitationalforces,and demandingscheduleswithlimited accesstovoidingareconsidered contributorstourinaryretention40

0–5%Bonelossandmusclewastingcan leadtoincreasedcalciumexcretion thatcanprecipitaterenalstones41

DermatologicalSofttissueinfections8–10%32 Inagravity-lessenvironment,bacteria andotherpathologic floracan potentiallylingerlongerontheskin’s surface

Traumatic injuries

• Intra-abdominal bleeding

• Fractures

• Jointinjuries

• Softtissueinjuries

11–26%32 Trauma27

IdentifypulmonaryB-linesor consolidations

Identifydeepveinthromboses

AbdominalPOCUSforbowel obstruction

Measurepost-voidbladder volume

Identifyhydronephrosis

Confirmabscessvscellulitis

• eFASTforintraperitonealfree fluid

• Identifybonyabnormalities

• Identifyjointeffusions

• Identifyhematomas,etc

***Indicatesdescribedreportsofpathophysiologyinspacewithoutdisclosednumericalvaluesinspaceorwithlittletonoepisodesinspace. Terrestrialincidencesareoftenusedforriskstratificationmodeling.

POCUS,point-of-careultrasound; eFAST,extendedfocusedassessmentwithsonographyfortrauma.

curriculum,perhapswithprogressiontoadvanced-level training,ishighlyrecommended.Foranyleveloftraining, learningcanbeconsolidatedbyremediationandspaced repetitionoftrainingthroughaugmentedrealityand recordedlectures.

IMPACTANDEFFECTIVENESS

Point-of-careUltrasoundandSpaceMedicine

WhilePOCUShasbeenusedbytheNationaland AeronauticsSpaceAdministration(NASA)astheprimary formofimagingaboardtheISSsince1982,originaldevices

Table1. Medicalemergencieswithrespectiveincidencesinspaceandrelatedutilityofpoint-of-careultrasound.

offeredonlyrudimentaryimagingcapabilities.12,27–29 Today POCUSdevicesarecapableofadvancedimagingwith multiplefrequenciesandmodalitiesforbothdiagnosticand therapeuticapplications.30 ManyPOCUSdevicesarenow hand-held,whichoffersauniqueadvantageoveralternative imagingmodalitiesinsettingswhereweightandvolume restrictionsarecritical,suchasinspacetravel.6–8 Consequently,POCUSrepresentsanidealimagingmodality forthegrowingspacemedicineindustry.8

Previouslydescribedmedicalemergenciesinspacespan nearlyallorgansystemsandreflecttheuniquephysiological stressplacedonthehumanbodybymicrogravityandother naturalrisksinspacesuchasdehydration(Table1).5,9,31 Similarly,therearescenariosinzerogravity,suchas scanningforfree fluidfortrauma(asincaseswithfocused assessmentwithsonographyfortraumaexams)orforpleural edema(pneumoniaorotherinfectiousorcardiacconditions), thathavealternativeinterpretationsgiventhegravity-less conditions.OnEarth,bloodor fluidwouldpoolincertain areasofthebody(thebladderrecessorinferioraspectsofthe lungs),butinzerogravity,thereisnoproclivityforpoolingin anyonespecificarea;hence,acompleteandthoroughexam isimportanttotrainfor.Asdemonstratedin Table1, POCUShasapotentialroleinassessingmedicalconditions associatedwithnearlyeveryorgansysteminspacetravel includingcardiac,pulmonary,genitourinary,andocular complaints.Althoughexpansive,thislistdoesnotaddressthe arrayofpotentialPOCUS-guidedcriticalprocedures.

Anticipatedin-flightproceduresincludethefollowing: establishingvascularaccess;regionalanesthesiaforacute paincontrol(orrarebutlife-savingprocedures);and pericardiocentesisandneedledecompressionfortension pneumothorax.Additionally,therearemanyother importantimplicationsforthepreparationofPOCUSusein spaceandadjustmentsthatmustbemadeandtrainedforin zero-gravityconditions.Forexample,ultrasoundgelisnot usedinspace,inpartbecausewaterisequallyaseffective andbecauseduplicateuseofitemsiscriticalforthecost andweightrestrictionsimposedforeachlaunch.

CONCLUSION

FutureofPoint-of-careUltrasoundinSpaceMedicine

Whilethereareavarietyofultrasoundapplicationsnot describedhere,theidentifiedorgansystemslistedwerechosen basedonfrequencyofemergenciesandtheanticipationofinflightmedicalneeds.42 Whiletheserecommendationshaveyet tobetestedandappliedforspacemedicinepractices,basedon similarPOCUSeducationmodels,skillretentionislikelytobe highamongastronauts.11,23 Spacedrepetitionand remediationwillhelpconsolidateskillsandcanbe instrumentalinmaintaining fluencylongterm.23,26 Alternativelearningmodalitiessuchasvirtualrealityand mixedin-persontrainingmodulescanassistwithskill retentioninrealtimewhenlivetele-consultsareunavailable.

Similarly,AIalgorithmshavethepotentialtoofferautomated imageinterpretationandclinical-decisionassistancewithout theneedforlivetele-support.

ImplementingastructuredPOCUScurriculumhasthe potentialtomaketangiblechangestoin-flighthealthcareand emergencyprocedures,whichwillbecrucialasthespace flightindustrycontinuestoevolve.Tomaximizetheutilityof thisdiagnosticandtherapeuticdevice,weproposethat POCUSeducationshouldbeaprerequisiteoftrainingfor space flightforbothnear-futureandfuturemissionsandcan beachievedthroughastructuredcurriculumtomakethe mostefficientuseofastronauttrainingandtime.11,29,43,44 Whiletimeallocationisanexceptionallyvaluableresourcein astronauttrainingandeducation,integratingPOCUS educationintothemandatoryspace-flighttraining curriculumwilllikelypayoffindividendsforfuture passengersandmissions.

AddressforCorrespondence:ChanelFischetti,MD,Brighamand Women’sHospital,DepartmentofEmergencyMedicine,75Francis St,Boston,MA02115.Email: cfischetti@bwh.harvard.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Drs.EmilyFrisch,AndrewGoldsmith, andNicoleDugganallhavereceivedconsultingoradvisingfeesfrom CentaurLabs.Dr.ChanelFischettiwasformerlyemployedby CentaurLabsandownsequityinthecompany.Shealsoisa consultantforPhilipsUltrasoundandLevelEx.Selectauthors (Drs.Dias,Savage,LoescheandMormann)havereceivedfunding from,arecurrentlyemployedby,orreceiverelatedfunding/grant supportfromNASAorSpaceX.

Copyright:©2024Fischettietal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.ReddyMV,NicaM,WilkesK.Spacetourism:research recommendationsforthefutureoftheindustryandperspectivesof potentialparticipants. TourismManagement. 2012;33(5):1093–102.

2.ZubrinRandSz´okaBFollettC. TheCaseforSpace.Amherst,NY: PrometheusBooks;2019.

3.KingAK.(2020). TheFutureofSpaceTourism.Congressional ResearchService; reports.congress.gov;2020.

AccessedOctober1,2021.

4.FrielM.Tourismasadriverinthespaceeconomy. CurrIssuesTour. 2020;23(13):1581–6.

5.LandonLB,RokholtC,SlackKJ,etal.Selectingastronautsfor long-durationexplorationmissions. Reach. 2017;5:33–56.

6.FrankJ,SpirkovskaL,McCannR,etal.Autonomousmission operations. IEEE.2013:1–20.

7.JanuaryO.Humanhealthandperformanceforlong-duration spaceflight. Future. 2008;79:629–35.

8.ChowD.SpaceXmakeshistorywith firstall-civilianspaceflight.NBC News.2021.Availableat: https://www.nbcnews.com/science/space/ spacex-makes-history-first-civilian-spaceflight-rcna2027 AccessedOctober1,2021.

9.RobertsonJM,DiasRD,GuptaA,etal.Medicaleventmanagement forfuturedeepspaceexplorationmissionstoMars. JSurgRes. 2020;246:305–14.

10.McFarlinK,SargsyanAE,MeltonS,etal.Asurgeon’sguidetothe universe. Surgery. 2006;139(5):587–90.

11.SargsyanAE,HamiltonDR,JonesJA,etal.FASTatMACH20:clinical ultrasoundaboardtheInternationalSpaceStation. JTrauma. 2005;58(1):35–9.

12.StewartLH,TrunkeyD,RebagliatiGS.Emergencymedicineinspace. JEmergMed. 2007;32(1):45–54.

13.KirkpatrickAW,McKeeJL,MoeiniS,etal.Pioneeringremotelypiloted aerialsystems(drone)deliveryofaremotelytelementoredultrasound capabilityforselfdiagnosisandassessmentofvulnerablepopulations. JDigitImaging. 2021;34(4):841–5.

14.DulchavskySA,GarciaK,Hamilton,etal.Advancedultrasoundforthe spaceprogramandonearth.In: InternationalSpaceStationBenefitsfor Humanity,3rd ed.Bookdevelopedcollaborativelybythemembersofthe InternationalSpaceStationProgramScienceForum.CreateSpace Publishing.2021;27–31.

15.NobleVE,NelsonBP,SutingcoAN,etal.Assessmentofknowledge retentionandthevalueofproctoredultrasoundexamsafterthe introductionofanemergencyultrasoundcurriculum. BMCMedEduc. 2007;7(1):1–5.

16.Osei-AmpofoM,TafoyaMJ,TafoyaCA,etal.Skillandknowledge retentionaftertrainingincardiopulmonaryultrasoundinGhana. EmergMedJ. 2018;35(11):704–7.

17.McKayGFMandWeerasingheA.Canwesuccessfullyteachnovice juniordoctorsbasicinterventionalultrasoundinasinglefocusedtraining session? PostgraduateMedJ. 2018;94(1111):259–62.

18.SteinmetzP,OleskevichS,LewisJ.Acquisitionandlong-termretention ofbedsideultrasoundskillsin first-yearmedicalstudents. JUltrasound Med. 2016;35(9):1967–75.

19.RussellFM,FerreR,EhrmanRR,etal.Whataretheminimum requirementstoestablishproficiencyinlungultrasoundtrainingfor quantifyingB-lines? ESCHeartFail. 2020;7(5):2941–7.

20.WatermanB,VanAarsenK,LewellM,etal.Qualityofabdominal ultrasoundimageacquisitionbynovicepractitionersfollowingaminimal trainingsessiononhealthyvolunteers. CJEM. 2020;22(S2):S74–8.

21.Situ-LaCasseE,AcunaJ,HuynhD,etal.Canultrasoundnovices developimageacquisitionskillsafterreviewingonlineultrasound modules? BMCMedEduc. 2021;21(1):175.

22.JensenJ,DyreL,JørgensenM,etal.Simulation-based point-of-careultrasoundtraining. ActaAnaesthesoilScand. 2018;62(6):811–9.

23.FinckeEM,PadalkaG,LeeD,etal.Evaluationofshoulderintegrityin space. Radiology. 2005;234(2):319–22.

24.ShackfordSR,RogersFB,OslerTM,etal.Focusedabdominal sonogramfortrauma. JTrauma. 1999;46(4):553–62.

25.ChiaoL,SharipovS,SargsyanAE,etal.Ocularexaminationfortrauma. JTrauma. 2005;58(5):885–9.

26.FischettiC,BhatterP,FrischE,etal.Theevolvingimportanceofartificial intelligenceandradiologyinmedicaltraineeeducation. AcadRadiol. 2021;29(Suppl5):S70–5.

27.WagnerMS,GarciaK,MartinDS.Point-of-careultrasoundinaerospace medicine. AviatSpaceEnvironMed. 2014;85(7):730–9.

28.PianL,GillmanLM,McBethPB,etal.Potentialuseofremote telesonographyasatransformationaltechnologyinunderresourced and/orremotesettings. EmergMedInt. 2013;2013:986160.

29.MartinDS,SouthDA,GarciaKM,etal.Ultrasoundinspace. Ultrasound MedBiol. 2003;29(1):1–12.

30.TheButterflyiQcatchesarideontheSpaceXdragontospace. BusinessWire.2021.Availableat: https://www.businesswire.com/ news/home/20210603005345/en/The-Butterfly-iQ-Catches-aRide-on-the-SpaceX-Dragon-to-Space.AccessedJuly1,2021.

31.SummersRL,JohnstonSL,MarshburnTH,etal.Emergenciesinspace. AnnEmergMed. 2005;46(2):177–84.

32.McPheeJCandCharlesJB.(2009). HumanHealthandPerformance RisksofSpaceExplorationMissions:EvidenceReviewedbytheNASA HumanResearchProgram.vol3405.USNationalAeronautics&Space Administration;2009.

33.Fritsch-YelleJM,LeuenbergerUA,D’AunnoDS,etal.Anepisodeof ventriculartachycardiaduringlong-durationspaceflight. AmJCardiol. 1998;81(11):1391–2.

34.D’AunnoDS,DoughertyAH,DeBlockHF,etal.Effectofshort-and long-durationspaceflightonQTcintervalsinhealthyastronauts. AmJCardiol. 2003;91(4):494–7.

35.ScheuringRAandJonesJ.Spacemedicineissuesandhealthcare systemsforspaceexplorationmedicine;NASATechnicalReport,20-7.

36.CrucianBE,ChoukèrA,SimpsonRJ,etal.Immunesystem dysregulationduringspaceflight:potentialcountermeasuresfordeep spaceexplorationmissions. FrontImmunology. 2018;9:1437.

37.Aun´on-ChancellorSM,PattariniJM,MollS,etal.Venousthrombosis duringspaceflight. NEnglJMed. 2020;382(1):89–90.

38.Marshall-GoebelK,LaurieSS,AlferovaIV,etal.Assessmentofjugular venousblood flowstasisandthrombosisduringspaceflight. JAMA NetworkOpen. 2019;2(11):e1915011.

39.ReyesDP,CarrollDJ,WaltonME,etal.Probabilisticriskassessmentof prophylacticsurgerybeforeextended-durationspaceflight. SurgInnov. 2021;28(5):573–81.

40.LiakopoulosV,LeivaditisK,EleftheriadisT,etal.Thekidneyinspace. IntUrolNephrol. 2012;44(6):1893–901.

41.CrucianB,Babiak-VazquezA,JohnstonS,etal.Incidenceofclinical symptomsduringlong-durationorbitalspaceflight. IntJGenMed. 2016;9:383–91.

Fischettietal. Competency-basedUltrasoundTrainingforIn-flightSpaceMedicine

42.HodkinsonP,AndertonR,PosseltB,etal.Anoverviewofspace medicine. BrJAnaesth. 2017;119(suppl_1):i143–53.

43.ModiP,Glavis-BloomJ,NasrinS,etal.Accuracyofinferior venacavaultrasoundforpredictingdehydrationinchildren

withacutediarrheainresource-limitedsettings. PLoSOne. 2016;11(1):e0146859.

44.ChisholmCB,DodgeWR,BaliseRR,etal.Focusedcardiacultrasound training:howmuchisenough? JEmergMed. 2013;44(4):818–22.

UltrasoundPerformedbyEmergencyPhysiciansforDeep VeinThrombosis:ASystematicReview

DanielHercz,MD*†

OrenJ.Mechanic,MD‡§

MarciaVarella,MD,PhD,MHS§

FranciscoFajardo,PhD,MLIS,MPA,MA§

RobertL.Levine,MD∥

*JacksonMemorialHospital,DepartmentofEmergencyMedicine, Miami,Florida

† UniversityofMiami,DepartmentofSurgery,Miami,Florida

‡ MountSinaiMedicalCenter,DepartmentofEmergencyMedicine, MiamiBeach,Florida

§ FloridaInternationalUniversity,HerbertWertheimCollegeof Medicine,DepartmentofMedicalEducation,Miami,Florida

∥ FloridaInternationalUniversity,HerbertWertheimCollegeof Medicine,DepartmentofEmergencyMedicineandCriticalCare, Miami,Florida

SectionEditors:JasonFields,MD,andPatrickMaher,MD,MS

Submissionhistory:SubmittedMay17,2023;RevisionreceivedOctober24,2023;AcceptedNovember22,2023

ElectronicallypublishedFebruary9,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18125

Introduction: Point-of-careultrasound(POCUS)performedbyemergencyphysicians(EP)has emergedasaneffectivealternativetoradiologydepartmentultrasoundsforthediagnosisoflower extremitydeepveinthrombosis(DVT).Systematicreviewssuggestedgoodsensitivityandspecificity overallforEP-performedPOCUSforDVTdiagnosis,yethighlevelsofheterogeneitywerereported.

Methods: Inthissystematicreviewandmeta-analysis,weaimedtoprovidethemostup-to-date estimatesoftheaccuracyofEP-performedPOCUSfordiagnosisofDVTandtoexplorepotential correlationswithtestperformance.WeperformedsystematicsearchesinMEDLINEandEmbasefor original,primarydataarticlesfromJanuary2012–June2021comparingtheef ficacyofPOCUS performedbyEPstothelocalstandard.QualityAssessmentofDiagnosticAccuracyStudies-2for individualarticlesarereported.Weobtainedsummarymeasuresofsensitivity,specificity,andtheir corresponding95%confidenceintervals(CI)usingbivariatemixed-effectsregressionmodels.We performedmeta-regression,subgroup,andsensitivityanalysesasplannedintheprotocol CRD42021268799submittedtoPROSPERO.

Results: Fifteenpublications fittheinclusioncriteria,totaling2,511examinations.Pooledsensitivityand specificitywere90%(95%CI82%–95%)and95%(CI91%–97%),respectively.Subgroupanalysesby EPexperiencefoundsignificantlybetteraccuracyforexamsperformedbyEPspecialists(93%,CI 88%–97%)vstrainees(77%,CI60%–94%).SpecificityforEPspecialists(97%,CI94%–99%)was higherthanfortrainees(87%,CI76%–99%, P = 0.01).Three-pointcompressionultrasound(CUS)was moresensitivethantwo-pointCUSbutwasonlystatisticallysignificantwhenlimitedtoEPspecialists (92%vs88%, P = 0.07,and95%vs88%, P = 0.02,respectively).

Conclusion: Point-of-careultrasoundperformedbyemergencyphysiciansissensitive andspecificforthediagnosisofsuspectedDVTwhenperformedbytrainedattendingEPs. Three-pointcompressionultrasoundexaminationmaybemoresensitivethantwo-point CUS.[WestJEmergMed.2024;25(2)282–290.]

INTRODUCTION

Lowerextremitydeepvenousthrombosis(DVT)isan acutemedicalconditionthat,ifnoturgentlydiagnosedand treated,canresultinseveremorbidityandmortality.Left untreated,theassociatedone-monthmortalityofacuteDVT is10–15%.1 Postphlebiticsyndromeisseenin23–67%of patientsafterresolutionoftheinitialthrombosis.2 Further, DVTisacommonproblemrepresentingupto2%of diagnosesmadeintheemergencydepartment(ED),3,4 makingitacompelling “ can ’t-miss” urgentdiagnosis. Compressionultrasonography(CUS)hasbecomea widespreadtoolthatmakestheevaluationofDVTrapidand precise.Compressionultrasonographyisrecognizedbythe AmericanCollegeofEmergencyPhysiciansandthe AmericanCollegeofRadiologistsasthestandardofcarefor thediagnosisofDVT,supplantingoldertechniques.5 In additiontoradiologydepartment-performedCUS,point-ofcareultrasound(POCUS)performedintheEDhasemerged asaneffectivediagnosticmodality.6

TheregionofinterestformostED-basedDVTPOCUS protocolsextendsfromthecommonfemoralveintothe poplitealvein.MostDVTPOCUSprotocolsincludeCUSof thecommonfemoralvein,poplitealvein,andpossiblythe femoralvein.7 Thesearereferredtoastwo-pointorthreepointCUS,respectively,dependingonthenumberofsites interrogated.Theclinicalsignificanceofisolatedvenous thrombosisofthecalfiscontroversial;however,non-urgent outpatientsurveillanceisanacceptedtreatment.8 Finally, whileisolatedthrombosisoftheiliacveinisapotentiallylifethreateningcondition,itisrareanddifficulttodetectwith existingsonographictechniques.9 Thus,distalDVTand isolatediliacveinthrombosisarenotaddressedin thisreview.

WhileED-performedPOCUSisacceptedbyemergency physicians(EP)andradiologistsforthediagnosisof DVT,thereexistssubstantialvariabilityinthediagnostic accuracyofPOCUS.7 Factorsthatmayaffectdiagnostic accuracyincludetheexperienceandabilityofthe ultrasoundoperator,thenumberofanatomicalsites ofthelowerextremityscanned,10 whetheraugmentation techniquesareused(suchasDoppler)andimage interpretation(suchasvesselidentificationand partialcompressibility).11,12

StudiesandreviewscomparingtheaccuracyofEDperformedPOCUSforthediagnosisofDVTtoaradiology department-performedultrasoundspanmorethan20years. Earlierstudiesweresmall,morelikelybasedintheUnited States,andcomplicatedbyheterogeneousmethodsand results.13 Currently,toourknowledge,thereexistno guidelinesorbestpracticesforED-basedDVTPOCUS. Withthelastsystematicreviewpublishedalmostadecade ago,weperformedanupdatedsystematicreviewtoexplore thediagnosticaccuracyofED-basedPOCUScomparedto radiologydepartment-performedultrasound.Wealso

exploredfactorsaffectingthediagnosticaccuracyforthe diagnosisofDVTthroughsubgroupanalysisandmetaregressionofrecentstudies.

METHODS

Inthissystematicreviewweaimedtoassesstheaccuracy ofbedsidevenousultrasonographyasperformedbyEPs whencomparedtothoseperformedbytheradiology departmentforthediagnosisofDVToflowerextremitiesin adultpatients.Theprotocolforthisreviewwasaccepted andregisteredontheInternationalProspectiveRegister ofSystematicReviews(PROSPERO)underthe numberCRD42021268799.

SearchStrategy

WeconductedaliteraturesearchinMEDLINE(viaOvid MEDLINE)andEmbase(viaElsevier)forrelevant,original studiespublishedfromJanuary2012–June2021toupdate fromthelatestpublishedsystematicreviewonthetopic.13 Thedetailedlistofsearchtermsusedislistedinthe Appendix (supplementalmaterial).Weconsultedwithdomainexperts forunpublishedstudiesandconductedamanualsearchof publishedliteraturefromthereferenceslistedontheincluded articles.ThelanguagewasrestrictedtoEnglish.

StudySelection

Eligiblestudieswereoriginal,primarydata,collected usingcross-sectionalandlongitudinalstudydesigns(cohort orrandomizedcontrolledtrials),thatincludedadultpatients (age >18years)presentingtotheEDforwhichDVTwas listedasadifferentialdiagnosisandforwhich,aspart ofthediagnosticworkup,anultrasonographicexamwas performedbyanEPandanultrasoundwasperformed bytheradiologydepartment.Acontrastvenogram (angiography)wasanacceptablealternativetoaradiology department-performedultrasound.Weusedthesystematic reviewmanagementtool,Covidence,forthescreening oftitles/abstractsandqualityassessmentofstudies.At leasttwoinvestigators(DHandMVorOHandMV) independentlyreviewedthetitlesandabstractsofthe studiesforeligibility.

Discrepanciesintheeligibilitydecisionwereresolvedbya thirdinvestigator(RL)afterreviewingthefullarticle. Reasonsforexclusionwererecorded.Weexcludedreview articles,editorialsorletters,expertopinions,comments,and animalexperiments.Lastly,weexcludedarticlesforwhich noinformationwasavailableonthetotalnumberoftrue positives,truenegatives,falsepositives,orfalsenegatives.

DataExtraction

Atleasttworeviewersindependentlyextracteddataonthe selectedstudies(DHandMV,orOMandMV).Collected informationincludedthefollowing:countrywherethestudy wasperformed;thetypeofUSexamusedfortheindextest

(two-pointorthree-point);clinicalexperience(attending and/ortrainee)anddescriptionoftheformaltrainingof physiciansperformingtheindextest;whethertheoriginal studyhadperformedriskstratificationofparticipantsprior totheuseoftheindextest;numbersoftruepositives,true negatives,falsepositives,andfalsenegatives,sensitivityand specificityasreported;andcorrespondingmeasuresof precision(confidenceintervals[CI]).Toassesspotential biasesinindividualstudies,weusedtheQualityAssessment ofDiagnosticAccuracyStudies2(QUADAS-2)checklist. Disagreementswereresolvedbyconsensus,orbya thirdreviewer.

StatisticalAnalyses

Summarymeasuresofsensitivity,specificity,andtheir corresponding95%CIswereobtainedusingbivariatemixedeffectsregressionmodels.WeestimatedI2 statisticassessing forstudyheterogeneity.Inaddition,inconsistencieswere furtherexploredthroughvisualinspectionofforestplots(for overlappingofsensitivityandspecificitypointestimatesand corresponding95%CI)andbysubgroupanalyses.Subgroup analyses,definedapriori,includedstratificationbythetype ofUSstudyperformed(two-pointCUSvsthree-pointCUS); experienceofphysiciansperformingtheindexPOCUS (completedspecialtyEMtrainingorspecialist/attendingvs EMtraineeorresidentstatus);prevalenceofDVT;sample size;riskofbias;andoutlierstatus.Weperformedall

Recordsidentified from Medline and EMBASE: Databases (n =230)

analyseswithSTATAv16(StataCorpLLC,College Station,TX).14

RESULTS

Weidentified230studiesinEMBASEandMedlinethat fit oursearchstrategy(Figure1).Afterremovingduplicates, titles,andabstracts,weretrieved38studiesforfurther evaluationbasedoninclusioncriteriaandabstractreview. Fifteenpublications15–29 remainedafterfulltextreviewwith reasonsforexclusionlistedin Figure1.Fourteenwerefulllengtharticleswithonemanuscriptreportingtwotrials.One additionalstudyreportingsufficientdataforinclusioninthe analysiswaspublishedasanabstract.Intwoinstances, weobtainedadditionalstudycharacteristicsviadirect authorcorrespondence.

CharacteristicsofStudies

Studiesvariedgreatlyingeographiclocations;three studiesweredoneintheUnitedStatesandCanada,twoin Australia,andthreeinIran,amongotherlocations(Table1). Thenumberofdiagnostictestscomparedrangedfrom 56–385.Moststudiesreporteddataperpatient,withtwo studiesthatreportedresultsbylimb.15,18 PrevalenceofDVT inthesamplesvariedfrom10%–79%.16,19,27 About50%of thestudiesusedtwo-pointultrasound,and50%usedthreepoint.Onepublicationtestedbothtwo-pointandthree-point UStothereferencestandard.22 Moststudiesusedthelocally

Recordsremoved before screening: Duplicate records removed. (n =6)

Recordstitles and abstract screened (n =224)

Reportssought for retrieval and assessed for eligibility. Full article screened: (n = 38)

Recordsexcluded(n =184)

Reason for exclusion(n=23):

Different index test (n=3)

Different outcome (n=2)

Different Reference (n=4)

Patient population (n=2)

Study design (n=2)

Review article (n=4)

Not English (n=1)

Duplicate data (n=2)

No data to construct 2x2 table (n=3)

Studies included in review. (n=15)

Figure1. PRISMA flowdiagramofthesearchandselectionprocessforstudiesincludedinthemeta-analysis.

Author,year

Experienceof physician performingthe indextest Torres-Macho,2012Spain76*34Two-pointUSUSdonebyradiologistAttending Abbasi,2012Iran8179Three-pointUS (withDoppler)

Patient’s country Number oftests*

DVT prevalence %IndextestReference

DuplexUSdonebya2nd-year radiologypostgrad EMresident Crowhurst,2013Australia178*13Three-pointUSDuplexUSdonebyradiologist (Dopplerusedifobesepatient)

Attending Poley,2014Canada22712Two-pointUSLCUSdonebyradiologistor medicalrecordreviewat6 monthsinthosewhohadno comprehensiveLCUS

LCUSdonebyradiologistAttending + EM resident

Attending + EM resident Zitek,2016UnitedStates385*10Two-pointUSUSdonebyradiologistEMresident Kim,2016UnitedStates29619Three-pointUS (withDoppler)

Pedraza-Garcia, 2017 Spain10954Three-pointUSUSdonebyradiologist (withDoppler) Attending

Zuker-Herman, 2018 Israel19526Two-andthreepointUS

DuplexUSdonebyradiologistAttending + EM resident Pujol,2018France5620Two-pointUSDuplexultrasounddonebya vascularcertifiedpractitioner. Attending Dehbozorgi,2019Iran24044Three-pointUSDuplexUSdonebyradiologistAttending + EM resident

Basaure,2019Chile10117Three-pointUSUSdonebyradiologistwith Doppler Attending + EM resident

Jahanian,2019Iran7238Three-pointUS (withDoppler)

USdonebyradiologist withDoppler EMresident

Howland,2019Australia10010Three-pointUSUnclearAttending Elsenga,2020Netherlands13821Two-pointUS (withDoppler) rCUSdonebyradiologistAttending + EM resident

Canakci,2020Turkey26626Two-pointUSUSdonebyradiologist orvenography EMresident

Diagnosticassessmentcouldbedoneperpatientsorperlimb(*markstudiesdoneperlimb).

DVT,deepveinthrombosis; US,ultrasound; ED,emergencydepartment; LCUS,limitedcompressionultrasound; rCUS,regional compressionultrasound; EM,emergencymedicine.

availableradiologydepartment-performedDVTUSasthe referencestandard.Lastly,pre-interventiontraining requirementsfortheEPoperatorsvariedgreatlybetween studies,rangingfrombriefdidacticstomulti-daypractical courses.Pre-existingexperiencewascategorizedaseither completionofanemergencymedicine(EM)postgraduate trainingprogramorbytraineestatus.

PrimaryOutcomes

Boththestudy-specificandpooledsensitivities, specificities,andrespective95%CIsareshownin Figure2. Comparedtothereferencestandard,thepooledsensitivity andspecificityoftheEP-performedUSfordiagnosisofDVT ofthelowerlimbwas90%(95%,CI82%–95%)and95% (95%,CI91%–97%),respectively.I2 andQ-teststatistics

suggestedsignificantheterogeneitybetweenstudies (Figure2).Thepooledpositiveandnegativelikelihoodratio forthesamecomparisonswere,respectively,19.1(95%,CI 10.2–35.8)and0.10(95%,CI0.06–0.19)(datanotshown).

SubgroupAnalysesandMeta-Regression

Weperformedexploratorymeta-regressionanalyseswith onlyoneexplanatoryvariableaddedtothemodel, consideringthelimitednumberofstudiesincluded.We assessedpresenceofbias,two-pointvsthree-pointCUS, priorexperienceoftheEP,prevalenceofDVTreported(less thanorgreaterthan30%),andsamplesize.Theexperienceof theEPandincreasedprevalenceofDVTinthesamplewere foundtobesignificantlyassociatedwithimprovedsensitivity andspecificity(meta-regressionjointmodel P = 0.01and

Table1. Characteristicsofstudiesselectedfordataabstraction.

Herczetal. UltrasoundforDeepVeinThrombosis

Zuker-Herman

Zuker-Herman 2018*

Kim 2016

Abbasi 2012

Torres-Macho 2012

0.05,respectively)(Figure3).Traineesensitivitywas77%vs 93%withintheattendinggroup.Specificitywas87%and 97%,respectively.Thesensitivityoftwo-pointandthreepointCUSwere88%and92%,respectively.Whenassessing foraccuracythiswasanon-statisticallysignificant improvement(P = 0.07).

Heterogeneitywassubstantiallyreducedwithrespectto thepooledsensitivityandspecificityforthestudiesincluding onlyspecialistEPs.Giventhese findings,weperformed furthersubgroupanalysisonspecialistEP-performed studies.Two-pointCUSstudiesperformedbyspecialist EPshadapooledsensitivityof88%comparedtothe 95%foundforthree-pointCUSalsoperformedby specialistEPs(P = 0.02).SpecificityofUSperformed byEPspecialistwasnotdifferentwhencomparingtwotothree-pointsUS.

QualityAssessment

BasedontheQUADAS-2toolforassessmentofthe qualityoftheindividualstudies,therewereconcerns regardingtheriskofbias(Figure4).Theaggregateriskof

SPECIFICITY

Canakci 2020

Elsenga 2020

Howland 2019

Jahanian 2019

Dahbozorgi 2019

Basaure 2019

Zuker-Herman 2018**

Zuker-Herman 2018*

Pujol 2018

Garcia 2017

Kim 2016

Zitek 2016

Poley 2014

Crowhurst 2013

Abbasi 2012

Torres-Macho 2012

StudyId COMBINED

0.93 [0.89 - 0.96]

0.97 [0.92 - 0.99]

1.00 [0.96 - 1.00]

0.85 [0.71 - 0.94]

0.93 [0.88 - 0.97]

0.90 [0.82 - 0.96]

biasidentifiedthat40%ofstudieswereconsideredhighor unclearriskofbiasofpatientselectionduetotheuseof convenience,non-consecutivesampling.Concerns regardinghighorunclearriskofbiasesrelatedtotheindex test,thereferencestandard,blinding,orthe flowand timing(oftheindexprocedurerelativetothereferencetest) werefoundinfewerthan30%ofthestudiesincluded (Figure4A).Theratingofeachindividualstudy regardingtheQUADAS-2biasesassessedisshownin SupplementalTable1.

SensitivityAnalyses

Weperformedsensitivityanalysesexcluding studies16,19,24–27 thatwereoutliersbasedonmodel fittingand outliers’ assessment.Pooledaccuracyfortheremaining11 resultswasslightlylower,andheterogeneityreduced substantially(sensitivity89%,95%CI85%–92%,andI2 = 27.8;specificity96%,95%,CI93%–97%,I2 = 60.3).Lastly, analysesrestrictedtostudiesforwhichtheriskofbiaswas consideredlowforalldomainsyieldedsimilarpooled sensitivityandspecificity(datanotshown).

SENSITIVITY (95%

Figure2. Forestplotsofsensitivityandspecificityofultrasoundperformedbyemergencyphysicianforthediagnosisoflowerextremities deepveinthrombosis(DVT).

Figure3. Subgroupanalysesforsensitivityandspecificityaccordingtoselectedstudycharacteristics.I2 toassessheterogeneityandmetaregression P-valuesfordifferencesintheaccuracywithinsubgroups.Thedottedlinerepresentsreferencevaluesobtainedinthepooled sensitivityandspecificityofallstudies.

US,ultrasound; EP,emergencyphysician; DVT,deepveinthrombosis; CI,confidenceinterval.

UpdatedSearch

Weperformedanewliteraturesearchinlate2022.Only onenewrelevantstudyof100patientshadbeenpublished sinceJune2021.30 Anexploratoryanalysisaddingthisstudy tothepoolof16studiespreviouslyassessedshowedno differencesinthepooledresultsreported.

DISCUSSION

ThediagnosisofDVTintheEDevolvedfrom cumbersometestsperformedoutsidetheED,suchas impedanceplethysmographyandvenography,toeasily implementedPOCUSthatismainstaytrainingofcurrent EMcurriculumintheUnitedStatesandsomeother countries.31,32 DespitewidespreaduseofPOCUS,concerns persistregardingtheaccuracyoftestsdoneinwidely disparateEDs.Anearlierquantitativesystematicreviewof studiesperformedintheUSyieldedsensitivitiesgreaterthan 95%.33 However,asmorediversestudieswerepublished,a subsequentreviewdemonstratedamoremoderatepooled sensitivitycloseto90%.13 Bothreviewsdemonstratedvery highspecificity.Unfortunately,todateallmeta-analyses addressingthistopichavebeenplaguedbyhighlevelsof heterogeneity,aproblemidentifiedinarecentreviewby Lee.12 Nomodelhasbeenproposedtoreduceheterogeneity.

Figure4. Aggregateassessmentofindividualstudyquality accordingtoQUADAS-2tool.

Toourknowledgethisisthe firstsystematicreviewwitha focustomaximizeperformanceofED-basedDVTPOCUS withrecommendationsonoperatorandtechnique.We identifiedtrendsexplainingstudyvariabilityaswellaskey biaseswithintheliterature.Inthismeta-analysis,usingthe mostrecentstudiesontheuseofPOCUSinEDsfrom multiplecountries,wedemonstratedapooledsensitivityand specificityof90%and95%,respectively.Theseresultsare somewhatsimilartopriorsystematicreviewsonED-based DVTPOCUS.However,clinicallysignificantvariationin operatorandscanningprotocolexistedinthe subgroupsexamined.

Generaloperatorleveloftraining(trainee/residentvs attending/fellow/specialiststatus)wasanimportant predictorofperformancewith77%sensitivitynotedinthe traineegroupvs93%inthespecialistgroup.Specificityin thesegroupswas87%and97%,respectively.Thisisinsharp contrastwithtrainingprovidedaspartoftheincluded studies.Aquantitativeanalysisoftrainingimmediatelypreinterventionwasnotpossibleduetolackofdetailed information.Withwhathasbeenreported,itseffecton accuracyappearstobefarlessthangeneralleveloftraining/ specialization.CompletionofformalEMtrainingpathway appearstohaveastrongeffectonPOCUSDVT USperformance.

Thisreviewspans10nationsfrom2012-2022,representing differentapproachestoEMandultrasoundtrainingandis, therefore,broadlyapplicabletocontemporarypractice.

Whilespecialtytrainingisoftencountry-specific, 34–36 most ofthesecountriesnowincludededicatedPOCUStrainingas amandatoryrequirementforEMspecialistqualification withsubspecialistUScertificationavailableaswell. Ultrasoundtechniqueacrossallincludedcountriestendedto besimilar,witharelianceonCUSoftheproximallegveins, inaccordancewithinternationallypublishedguidelineson thediagnosisofDVT.32

AnotherunsettledquestionfortheEMapplicationof POCUSforthediagnosisofDVTiswhetherthree-pointUS issuperiorcomparedtothecommonlyimplementedtwopointexamination.A2018radiologistconsensusreport recommendsthree-pointratherthantwo-pointCUSasa baserequirementfordiagnosisofDVTbecausethree-point CUSdetectsisolatedfemoralveinthrombosesthatwould otherwisebemissedin5%–8%ofthosewithlowerextremity DVT.7 ThestudybyAdhikarietal,10 analyzingthreeyearsof radiology-performedCUSintheEDalsofoundthatthreepointCUSdetectedanadditional6%oflowerextremity DVTisolatedtothefemoralvein,withoutinvolvementofthe commonfemoralveinorpoplitealvein.Lastly,thestudyof Tabbutetalfoundasimilarrateofisolatedthrombifroma mixofPOCUSandradiology-performedstudies.37

Oneofthisreview’sstudiesexploredthesensitivityoftwo vsthree-pointUSexamsperformedbytraineesand specialistsasa within-patient analysis.Thesensitivityforthe

diagnosisofDVTincreasedby7%byincludingthethirdsite. Theseresultsareintuitiveevenincasesofnon-isolated femoralveinthrombi.Scanningmultiplesitesreducesthe probabilityoffalsenegativescansasjustasinglepositive findingisarequirementfordiagnosis.Ourpooledanalysisof two-pointvsthree-pointscanningyieldeda5%higherpoint estimateofsensitivityforthemorecomprehensivescan withoutlossinspecificity,whichiscongruentwithprior literature.Thedifferencewasnotstatisticallysignificant witha P -valueof0.07.Whenlimitedtoonlyspecialistperformedexams,thedifferencewasstatistically significant(P = 0.02).

Wefoundlargereductionsinheterogeneityinmultiple subgroupswhenlookingatstudiesofattendingphysicianperformedPOCUS.TheseincludespecialtytrainedEPperformedtwo-pointandthree-pointscansandstudies withouthighlevelsofbias.Thisimpliesahigherdegreeof confidenceintheconsistencyoftheintervention’ s performanceinqualifiedhands.Subgroupanalyseswith prevalencebeloworabove30%yieldedincreasedspecificity forstudieswithprevalenceabove30%.However,a30% prevalenceofDVTintheEDisunusuallyhighand unexpected.Differencesinpatientinclusioncriteria (Wellsscoringand/orD-dimer)mayhavecontributedtothis effect.Thepotentialeffectofhighprevalenceof DVTonthediagnosticaccuracystudiesisyetto beconfirmed.

LIMITATIONS

Thismeta-analysishassomelimitations.First,because only15studieswereidentified,morecomplexanalysescould notbeperformed.Furthermore,moststudiescontained elementsofbias,especiallyrelatedtopatientselection; recruitmentoftenoccurredasaconveniencesample, presumablyselectedbytheultrasoundoperator/clinician. Additionally,threestudiesincludedinconclusive results.18,20,28 Wefollowedbestpracticesandmadethe decisiontoclassifyinconclusivecasesasallpositiveorall negativedependingontheclinicalcontext.38 Basedonthe studydesignreportedbytheauthors,wecategorizedthe inconclusiveresultsasnegative.Asensitivityanalysiswas conducted,andthelimitednumberofinconclusiveresultsare unlikelytoaffectthepooledresultsherebyreported.

Anotherlimitationrelatestotheinabilitytobetter characterizethelevelofexperienceoftheUSoperatordueto limiteddetailedinformationonoperatortraining (SupplementalTable2).Lastly,restrictingpublicationsfrom 2012tothepresentlimitedthenumberofstudiesandthe powertoassesspotentialsubgroupdifferences.However, since2012formalizedtraininginPOCUShasbeen adoptedaspartofspecialisttraininginmostcountries includedinthisreview.Thus,webelievethatthis review’sresultsaremoregeneralizabletothebroad EMpopulation.

CONCLUSION

Thismeta-analysisofstudiesreportedsince2012 demonstratedexcellentperformanceofEMspecialistperformedthree-pointpoint-of-careultrasoundforthe diagnosisofdeepveinthrombosis.Boththepooled sensitivityandspecificitywere95%.Werecommendthat POCUS-trainedattendingEPsperformathree-point examinationintheEDtoeffectivelyandaccuratelydiagnose DVT.FuturegeneralstudiesonED-basedDVTPOCUSare unlikelytomodifythese findingsgiventhenumerousexisting studiesofatleastmoderatequality.Futurestudiesof rigorousmethodologyfurtheraddressingcertainsubgroups arerecommended.

AddressforCorrespondence:Robert,L.Levine,MD,Florida InternationalUniversity,HerbertWertheimCollegeofMedicine, DepartmentofEmergencyMedicineandCriticalCare,11200SW8th St.,AHC4suite261,Miami,FL33199.Email: rlevine@fiu.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financialrelationshipswithanycompaniesthatarerelevanttothis study.Therearenoconflictsofinterestorsourcesoffunding todeclare.

Copyright:©2024Herczetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.GalanaudJP,MonrealM,KahnSR.Epidemiologyofthepostthromboticsyndrome. ThrombRes.2018;164:100–9.

2.WhiteRH.Theepidemiologyofvenousthromboembolism. Circulation 2003;107(23_suppl_1):I.4–8.

3.Carter-StorchR,OlsenUF,MogensenCB.Admissionstoemergency departmentmaybeclassifiedintospecificcomplaintcategories. Dan MedJ.2014;61(3):A4802.

4.KellyJandHuntBJ.Doanticoagulantsimprovesurvivalin patientspresentingwithvenousthromboembolism? JInternMed 2003;254(6):527–39.

5.ExpertPanelonVascularImaging,HanleyM,SteignerML,etal. ACRAppropriatenessCriteria® suspectedlowerextremitydeepvein thrombosis. JAmCollRadiol.2018;15(11S):S413–7.

6.UltrasoundGuidelines:Emergency,point-of-care,andclinical ultrasoundguidelinesinmedicine. AnnEmergMed 2023;82(3):e115–55.

7.NeedlemanL,CronanJJ,LillyMP,etal.Ultrasoundforlowerextremity deepvenousthrombosis:multidisciplinaryrecommendationsfromthe SocietyofRadiologistsinUltrasoundConsensusConference. Circulation.2018;137(14):1505–15.

8.GarryJ,DukeA,LabropoulosN.Systematicreviewofthecomplications followingisolatedcalfdeepveinthrombosis. BrJSurg 2016;103(7):789–96.

9.BarrellierMT,LezinB,MonsallierJM.Isolatediliacdeepvenous thrombosis.Studyof48casesseenin7yearsamong 18,297echo-Dopplerevaluationsofthelowerlimbs. JMalVasc 2001;26(5):290–8.

10.AdhikariS,ZegerW,ThomC,etal.Isolateddeepvenousthrombosis: implicationsfor2-pointcompressionultrasonographyofthelower extremity. AnnEmergMed.2015;66(3):262–6.

11.VarriasD,PalaiodimosL,BalasubramanianP,etal.Theuseofpoint-ofcareultrasound(POCUS)inthediagnosisofdeepveinthrombosis. JClinMed.2021;10(17):3903.

12.LeeJH,LeeSH,YunSJ.Comparisonof2-pointand3-pointpoint-ofcareultrasoundtechniquesfordeepveinthrombosisatthe emergencydepartment:ameta-analysis. Medicine(Baltimore) 2019;98(22):e15791.

13.PomeroF,DentaliF,BorrettaV,etal.Accuracyofemergencyphysicianperformedultrasonographyinthediagnosisofdeep-veinthrombosis:a systematicreviewandmeta-analysis. ThrombHaemost 2013;109(1):137–45.

14.StataCorp. StataStatisticalSoftware, 16thed.CollegeStation,Texas: StataCorp,LLC,2019.

15.Torres-MachoJ,Ant´on-SantosJM,García-GutierrezI,etal.Initial accuracyofbedsideultrasoundperformedbyemergencyphysiciansfor multipleindicationsafterashorttrainingperiod. AmJEmergMed 2012;30(9):1943–9.

16.AbbasiS,BolverdiE,ZareMA,etal.Comparisonofdiagnosticvalueof conventionalultrasonographybyemergencyphysicianswithDoppler ultrasonographybyradiologyphysiciansfordiagnosisofdeepvein thrombosis. JPakMedAssoc.2012;62(5):461–5.

17.CrowhurstTDandDunnRJ.Sensitivityandspecificityofthree-point compressionultrasonographyperformedbyemergencyphysiciansfor proximallowerextremitydeepvenousthrombosis. EmergMed Australas.2013;25(6):588–96.

18.PoleyRA,NewbiggingJL,SivilottiML.Estimatedeffectofanintegrated approachtosuspecteddeepvenousthrombosisusinglimitedcompressionultrasound. AcadEmergMed.2014;21(9):971–80.

19.ZitekT,BaydounJ,YepezS,etal.Mistakesandpitfallsassociatedwith two-pointcompressionultrasoundfordeepveinthrombosis. WestJ EmergMed.2016;17(2):201–8.

20.KimDJ,ByynyRL,RiceCA,etal.Testcharacteristicsofemergency physician-performedlimitedcompressionultrasoundforlower-extremity deepveinthrombosis. JEmergMed.2016;51(6):684–90.

21.PedrazaGarcíaJ,ValleAlonsoJ,CeballosGarcíaP,etal.Comparison oftheaccuracyofemergencydepartment-performedpoint-of-careultrasound(POCUS)inthediagnosisoflower-extremitydeepvein thrombosis. JEmergMed.2018;54(5):656–64.

22.Zuker-HermanR,AyalonDangurI,BerantR,etal.Comparisonbetween two-pointandthree-pointcompressionultrasoundforthediagnosisof deepveinthrombosis. JThrombThrombolysis.2018;45(1):99–105.

23.PujolS,LaurentJ,MarkarianT,etal.Compressionwithapocket-sized ultrasounddevicetodiagnoseproximaldeepveinthrombosis. AmJ EmergMed.2018;36(7):1262–4.

24.DehbozorgiA,DamghaniF,Mousavi-RoknabadiRS,etal.Accuracyof three-pointcompressionultrasoundforthediagnosisofproximaldeepveinthrombosisinemergencydepartment. JResMedSci.2019;24:80.

25.BasaureCE,RojasN,ClausdorffH,etal.Shorttrainingofcompressible ultrasoundtechniquefordiagnosisofdeepveinthrombosisby emergencyphysicians. AcadEmergMed.2019;26:S264.

26.JahanianF,KhatirIG,Bani-MostafaviES,etal.Diagnosticaccuracyofa three-pointcompressionultrasonographyperformedbyemergency medicineresidentforthediagnosisofdeepveinthrombosis:a prospectivediagnosticstudy. ActaInformMed.2019;27(2):119–22.

27.HowlandCandMurrayA.Canemergencyphysicianperformed threepointcompressionultrasonographyforlowerlimbdeepvein thrombosisreducelengthofstay? EmergMedAustralas.2019;31:48–9.

28.ElsengaHE,ColleeA,RosendaalAV.Agreementbetweenemergency physiciansandradiologistsforthediagnosisofdeepvenousthrombosis withcompressionultrasound:aprospectivestudy. EurJEmergMed 2021;28(1):25–8.

29.CanakciME,AcarN,BilginM,etal.Diagnosticvalueofpoint-of-care ultrasoundindeepveinthrombosisintheemergencydepartment. JClin Ultrasound.2020;48(9):527–31.

30.AhmedEl-GazzarEH,AlkafafyAM,El-SalamFathiHA,etal.Sensitivity andspecificityofthree-pointcompressionultrasonographytest performedbyemergencyphysiciansfordiagnosisoflowerlimbsdeep venousthrombosis. EgyptJAnaesth.2021;37(1):517–22.

31.AtkinsonP,BowraJ,LambertM,etal.InternationalFederationfor EmergencyMedicinepointofcareultrasoundcurriculum. CJEM 2015;17(2):161–70.

32.FischerLM,WooMY,LeeAC,etal.Emergencymedicinepoint-of-care ultrasonography:anationalneedsassessmentofcompetenciesfor generalandexpertpractice. CJEM.2015;17(1):74–88.

33.BurnsidePR,BrownMD,KlineJA.Systematicreviewofemergency physician-performedultrasonographyforlower-extremitydeepvein thrombosis. AcadEmergMed.2008;15(6):493–8.

34.DuchenneJ,MartinezM,RothmannC,etal.Premierniveaude compétencepourl’échographiecliniqueenmédecined’urgence. RecommandationsdelaSociétéfrançaisedemédecined’urgencepar consensusformalisé. Ann.Fr.Med.Urgence.2016;6(4):284–95.

35.LewisD,RangL,KimD,etal.Recommendationsfortheuseofpointof careultrasound(POCUS)byemergencyphysiciansinCanada. CJEM 2019;21(6):721–726.

36.AustralasianCollegeforEmergencyMedicine.TheUseofFocused UltrasoundinEmergencyMedicine.2022.Availableat: https://acem. org.au/getmedia/000b84ee-378f-4b65-a9a7-c174651c2542/

2023_2-P21-Policy-for-Focused-Ultrasound-in-Emergency-Medicine AccessedOctober10,2023.

37.TabbutM,EbersoleN,IckenL,etal.Two-pointcompressionultrasound techniquerisksmissingisolatedfemoralveinDVTs. WestJEmergMed 2022;23(4):597–600.

38.CohenJF,KorevaarDA,AltmanDG,etal.STARD2015guidelinesfor reportingdiagnosticaccuracystudies:explanationandelaboration. BMJOpen.2016;6(11):e012799.

UltrasoundforDeepVeinThrombosis Herczetal.

ORIGINAL RESEARCH

andEmergencyContraceptioninEmergencyDepartments

intheUnitedStates:ANationalSurvey

ColleenCowdery,MD*

DianaHalloran,MD*

RebeccaHenderson,PhD*

MAKathleenM.Allen,MPH†

KellyO’Shea,MD*

KristenWoodward,MD*

SusanRifai,MD*

ScottA.Cohen,MPH†‡

MuhammadAbdulBakerChowdhury,MPH‡

CristinaZeretzke-Bien,MD‡

LaurenA.Walter,MD§

Marie-CarmelleElie-Turenne,MD‡§

SectionEditor:LauraWalker,MD

*UniversityofFloridaCollegeofMedicine,Gainesville,Florida

† UniversityofFloridaCollegeofPublicHealthandHealth Professions,Gainesville,Florida

‡ DepartmentofEmergencyMedicineUniversityofFlorida, Gainesville,Florida

§ DepartmentofEmergencyMedicine,SchoolofMedicine, UniversityofAlabamaatBirmingham,Birmingham,Alabama

Submissionhistory:SubmittedJuly19,2023;RevisionreceivedNovember28,2023;AcceptedDecember5,2023

ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem

DOI: 10.5811/westjem.18405

Background: Despitetheprevalenceofsexualassaultpresentationstoemergencydepartments(ED) intheUnitedStates,currentaccesstosexualassaultnurseexaminers(SANE)andemergency contraception(EC)inEDsisunknown.

Methods: Inthisstudyweemployeda “secretshopper,” cross-sectionaltelephonicsurvey.Ateam attemptedphonecontactwitharepresentativesampleofEDsandaskedrespondentsaboutthe availabilityofSANEsandECintheirED.Reportedavailabilitywascorrelatedwithvariablesincluding region,urban/ruralstatus,hospitalsize,faithaffiliation,academicaffiliation,andexistenceoflegislative requirementstoofferEC.

Results: Overatwo-monthperiodin2019,1,046callstohospitalswereattemptedand960were completed(91.7%responserate).Ofthe4,360eligiblehospitalslistedinafederaldatabase,960 (22.0%)werecontacted.AccesstoSANEsandECwerereportedtobeavailablein48.9%(95% confidenceinterval[CI]45.5–52.0)and42.5%(95%CI39.4–45.7)ofhospitals,respectively.Accessto ECwaspositivelycorrelatedwithSANEavailability.TheEDsreportingSANEandECavailabilitywere morelikelytobelarge,rural,andaffiliatedwithanacademicinstitution.ThosereportingaccesstoEC weremorelikelytobeintheNortheastandinstateswithlegislativerequirementstoofferEC.

Conclusion: Ourresultssuggestthatperceivedaccesstosexualassaultservicesandemergency contraceptioninEDsintheUnitedStatesremainspoorwithregionalandlegislativedisparities.Results suggestdisparitiesinperceivedaccesstoECandSANEintheED,whichhaveimplicationsforimproving EDpracticesregardingcareofsexualassaultvictims.[WestJEmergMed.2024;25(2)291–300.]

Keywords: emergencycontraception;sexualassaultnurseexaminer;sexualassault.

INTRODUCTION

Theemergencydepartment(ED)isanimportantpointof entryforvictimsofrape,trafficking,andotherformsof sexualanddomesticviolence.IntheUnitedStates,sexual assaultpresentationstoEDSincreasedby1,533%from2006 to2019.1 Thecurrentstateofaccesstohigh-quality emergencysexualassaultcareintheUSisunclear.

SexualassaultcareinEDsintheUSincludestheneedfor forensicevidencecollection.Adirectedapproachtoprovide thisspecializedcareisthroughtheuseofsexualassault nurse(orforensic)examiners(SANE) 2;SANEsare registerednursesorclinicianswhohavecompleteda didacticandclinicalcurriculumapprovedbythe InternationalAssociationofForensicNursesorother certifyingbody.3 Theyperformforensicsexualassault examsandevidencecollectionwhilemeetingthemedical, psychological,andeducationalneedsofindividuals requiringservices.4 StudieshavedemonstratedthatSANEs providemore “humanizing ” carethannon-SANE emergencypractitionersfromthepatientperspective,5 more comprehensiveandconsistentmedicalservices, 4 andmore thoroughforensicexaminationstoimprovethecriminal justiceresponsetosexualassault. 6 Currently,thereareover 450SANEprogramsintheUS,approximately75%of whichareaf fi liatedwithanED.3, 4 However,nofederal regulationsdictatewhocanprovidesexualassaultcareor overseethequalityofcareforsexualassaultvictims,and requirementsvarybystate. 7 Thestateofnationalaccessto sexualassaultcare,includingtheknowledgeoffrontline healthcliniciansaboutaccessibility,isunclear.Despitethe effectivenessofSANE-ledcare,4 signi fi cantdisparitiesin accessarebelievedtopersist.7, 8

InadditiontoSANEaccessibility,emergency contraception(EC)isanimportantcomponenentofcare aftersexualassault,justasitisanimportantcomponent ofreproductivehealthcare.Endorsedbyleadingmedical organizations,ECisconsideredasafeandeffectivemeans ofpreventingpregnancy,includingincasesofsexual violence.9,10 ProvisionofECisimportantinthecareof survivorsofabuseordomesticviolence.11AccesstoECinthe EDisimportantbothasacomponentofappropriatecarefor sexualassaultandasaserviceforlow-incomeindividuals becausecostremainsabarrierforthem.Indeed,the AffordableCareActrequiresmostprivateinsurersandstate Medicaidprogramstocoverprescriptioncontraceptionbut notEC.12 In2017,thenationalaveragepricefortrade-name, one-doselevonorgestrelwas$49.48andgenericone-dose levonorgestrelwas$38.74.13 Inadditionto financialbarriers, only4.9%ofpharmaciesareopen24hoursperday/seven daysperweek.14 Otherpotentialbarrierstopatientaccess includerefusaltodispensebypharmacists,misinformation duetopersonalreligiousbeliefs,lackofclinicianexposure, andsocialstigma.15

PopulationHealthResearchCapsule

Whatdowealreadyknowaboutthisissue? In2005anestimated16%ofemergency departments(EDs)intheUSprovided unrestrictedaccesstoemergency contraception(EC).Shiftinglegislation mayhaveimpactedaccess.

Whatwastheresearchquestion?

Whatfactorsaffecttheuserexperienceof seekingECandsexualassaultnurseexaminer (SANE)careinUSEDs?

Whatwasthemajor findingofthestudy?

AccesstoSANEandECwerereported tobeavailablein48.9(95%CI45.5 – 52.0) and42.5%(95%CI39.4 – 45.7)of hospitals,respectively.

Howdoesthisimprovepopulationhealth?

AccesstoSANEcareandECinUSEDsis lowandwithcleardisparities.Resultshave implicationsforimprovingEDpolicies regardingcareofsexualassaultvictims.

A2005studyusinga “mysteryclient” surveyfoundthat onlyanestimated16%ofEDsintheUSprovideaccesstoEC withoutrestriction.16 However,thereisreasontobelievethat accesstoECintheEDhaschanged.Theabovestudywas performedpriortonotableexpansionsinECchoicesand accessintheUS.In2006,theUSFoodandDrug Administrationapprovedtheover-the-countersaleof levonorgestreltothose ≥18yearsofage,andthenin2013 expandedaccesstothose ≥15years.17 Newhormonal optionshavealsobecomeavailable.18 Further,since200514 statesandtheDistrictofColumbiahaverequiredEDsto dispenseECtosexualassaultvictimsuponrequest.19 Currentpenaltiesinclude finesorsuspensionorrevocationof hospitallicensure20;however,theabsenceofstrong enforcementmechanismshascorrelatedwithdecreased compliancerates.21 One2019reviewofliteratureonEC provisioninEDsintheUSfoundthat60%hadapolicyon EC,75%officiallyprovidedECcounseling,44%officially offeredEC,and62%officiallyhadECavailabletodispense onsite.22 Itisunknownhowthesestatisticcorrelate withpractice.

MoststudieshaveexaminedaccesstoSANEservicesand ECintheEDfromtheperspectiveofhospitalpersonnel,

basedoninstitutionalpolicy,orpriortochangesinEC legislation.Thestudiesincludedonlyidealcasesratherthan real-worldconditions;thosethatuseda “mysteryclient” approachshowedlowerratesofaccess.16,22 Thus,studies conductedfromtheperspectiveofthepatientorsexual assaultvictimareneededtodefinenationalaccessand ascertainpotentialdiscrepanciesbetweenpredicted (ie,reportedorpreviouslypublished)andobservedrates ofaccesstoSANEservicesandECintheED.

Giventherecentriseinpresentationsofsexualassaultin theUS1 andtheroleoftheEDasapivotalandtime-sensitive pointofaccessincasesofsexualviolence,wesoughtto evaluateSANEandECavailabilityinEDsintheUSfrom theperspectiveofapatientseekingtoknowtheavailability ofcareoverthephone.Oursurveyaddressesuserexperience, providingapragmaticexampleofpatientexperienceswhen attemptingtoaccesssexualassaultservicesandECthrough theED;thisstudyalsoexaminesdifferencesinperceived availabilityoftheseservicesonthebasisofgeographicand institutionalfactors.

MATERIALSANDMETHODS

Wesoughttoupdatethe2005telephone-based, “secret shopper” studyofhospitalsacrosstheUStoinvestigate patientaccesstosexualassaultcareusingtheavailability ofSANEservicesandECasaproxyforaccessto comprehensiveservicesfromtheperspectiveofaprospective patient.Toassessaccessibilityandperceivedavailability,we usedthereportoffrontlinehealthcareworkerslikelytobethe firstpointofcontactforpatientsintheEDasthesourceof informationregardingavailableservices.Wealsosoughtto determinewhethergeographicandinstitutionalfactorswere associatedwithreportedaccess.Moreover,giventhe influenceofgraduatemedicaleducationprogramson institutionalresources,wesoughttodeterminewhether teachingstatusimprovedaccess.Ourstudyincludeda demographicevaluationbasedonsize,ruralvsurbansetting, teachingstatus,andfaith-basedstatusofhospitals.

WeobtainedalistofEDsintheUSfromapublicly availabledatabaseoftheCentersforMedicare&Medicaid Services(CMS)inMarch2019.Thisdatabaseconsistedof 4,806hospitals.Exclusioncriteriaincludedfederal institutions,children’shospitals,tribalhospitals,hospitals withoutEDs,andhospitalslocatedinUSterritories.Ofthe remainingeligible4,360hospitals,25%wererandomly selectedandstratifiedbyregion(Northeast,South,West, andMidwest;see TableS1 forthelistofstatesperregion)and byteachingstatus.Weaimedtosurveygreaterthan20% ofeligiblehospitalswith21%representationofteaching institutions,whichwastheproportionofteaching institutionsintheoverallcohort.Hospitalswereclassifiedas teachinghospitalsonthebasisoftheirregistrationwiththe CMS.Eachregionalsamplewascheckedtoensure representationofhospitalsclassifiedashavingteaching

status.Ingeneral,foreverythreenon-teachinginstitutions, oneteachinginstitutionexistedintheanalysiswithin eachregion.

Fortheanalysis,hospitalswereclassifiedbyregionand stateassmall(<100beds),mid-sized(100–200beds),orlarge (>200);asurban(population ≥50,000)orrural(population <50,000);asacademicornon-academic;asfaith-ornonfaith-based;andbythepresenceofastatelegislative requirementtoofferECtosexualassaultvictims.Ateamof fivewomeninvestigatorssimulatingpotentialpatientscalled publiclyavailableEDphonenumbersforeachhospital betweenJune–September2019,seekingECasdescribedby Harrisonetal.16 CallerscontactedtheEDseekingmedical adviceandaskedaboutECandSANEaccess.The respondentwouldeitherprovidetheresponseortransferthe calltoamoreknowledgeablememberofthemedicalstaff includingadvancedpracticeprovidersandphysicians.

Callersreceivedstructuredtrainingwithstandardized scripts,whichwerethencalibratedthroughaseriesof simulatedcalls.Inaddition,5%ofcallswerescreenedfor fidelityandtoensurestandardizationbycompletingaseries ofobservedcallencounters.Thephonenumbersofthecallers wereconcealed,andthetimeofthedayanddayoftheweek wasrecorded;callstookplaceduringnormalbusinesshours (ie,9 AM-5 PM).Callers firstaskedaboutaccesstoECand thenaskedifitwasavailableinthecaseofsexualassault. TheythenaskedwhetheraSANEwasavailable.Thisscript wasmodeledontheprotocolofthemostrecentsurveyofEC access. 16 Followingthe first5%ofcalls,thescriptwasrevised andstandardizedforincreased fidelityindatacollection. Revisionsincludedminorchangesinwordingandorder ofquestions.

PrimaryoutcomeswerereportedaccesstoSANEsand ECintheED.ASANEwasconsideredavailableifthe respondentreportedthataSANEwasonsiteorcouldbeon sitewithinsixhours.ASANEwasconsiderednotavailableif respondentsweretoldtherewerenoSANEsavailablewithin sixhours.WedefinedECaccessasfull,conditional,orno access.Fullaccessincludedhospitalsthatreportedthatthey hadavailableECwithnorestriction.Conditionalaccesswas definedashospitalsthatreportedthattheyprovidedEConly inthecircumstanceofsexualassault;andnoaccesswas definedashospitalsthatreportedanabsenceofECprovision underanyconditionsorifthecallerwasreferredtoan outpatientpharmacyforaccess.Secondaryoutcomes includedtypeofECoptionsavailable,alternativemethodsof obtainingEC,accesstoreferraltoalternativehealthcare systems,andaccesstosexualassaultresources.Aswesought topragmaticallyimitatetheexperienceofaprospective patientcallingtheED,callersdidnotaskforthe qualificationsofrespondents,nordidtheyasktobe transferredtoaphysicianornurse,althoughtheytooksuch transfersiftheywereoffered.Theyrecordedthe first definitiveresponsetheyreceivedfromanystaffmember.

WemanagedallstudydatainResearchElectronicData Capturev9.11,hostedattheUniversityofFlorida. StatisticalanalyseswereperformedusingSASv9.4(SAS Institute,Inc,Cary,NC).Weinitiallyuseddescriptive statistics,includingmeans,medians,frequencies,and proportions,toexaminesurveyresponserepresentation, variabledistribution,andmissingnesswhereappropriate. Wecalculatedexactconfidenceintervals(CI)usingthe Clopper–Pearsonmethod.Unadjustedandadjustedlogistic regressionmodelswereusedtoevaluatetherelationship betweenhospitalcharacteristicsandoutcomes.We performedanunconditionalhierarchicallogisticregression model,whereEDswerenestedinrespectivestates,toassess thepredictedprobabilityofanEDprovidingECforeach state.Eachstatewasaddedasarandomeffect.

Thisstudyreceivedapprovalforexemptionfromthe UniversityofFloridaInstitutionalReviewBoardprior toinitiation.

RESULTS

BetweenJuly2–September5,2019,callersattemptedto call1,046hospitalsandcompleted960calls(91.7%response rate).Eighty-sixofthecalls(8.2%)failedduetothefollowing reasons:failuretocontact(25,2.3%);refusaltoanswer

questions(13,1.2%);hospitalclosure(20,1.9%);noED(7, 0.6%),oranotherunclassifiedreason(21,2%).The Figure illustratesthe flowofhospitalinclusionorexclusionthrough thestudy. Table1 presentsthecharacteristicsofthe960 hospitalsthatweresuccessfullysurveyed.(See TableS2 for thebreakdownofnumberofhospitalsbystate.)Sexual assaultnurseexaminerswerereportedtobeavailablein 48.9%ofthe960hospitalssurveyed(Table2).

Afteradjustingforcovariates,thefollowingfactorswere independentpredictorsofSANEaccess:region;ECaccess; sizeofhospital;academicstatus;andurbanstatus(Table3). Seeunadjustedcomparisionsin TableS2).Regionwas associatedwithreportedSANEaccess,withhospitalsinthe Northeastbeing4.00timesmorelikely(95%CI2.38–7.14), 2.78timesmorelikel(95%CI1.59–4.76),and2.00times morelikely(95%CI1.19, 3.33)tohaveSANEaccessthan hospitalsintheSouth,West,andMidwest,respectively (Table3).ReportedECaccessincasesofsexualassaultwas alsoassociatedwithSANEpresence,withemployeesatthese hospitals3.94timesmorelikely(95%CI2.66–5.83)toreport

–15.0

Midwest284(29.6)26.7–32.6

South369(38.4)35.4–41.6

West185(19.3)16.8–21.9

Urban–ruralstatus – n(%)

Urbanarea319(33.2)30.3–36.3

Ruralarea641(66.8)63.7–69.8

Sizeofhospital – n(%)

Small469(48.9)45.7–52.1

Medium203(21.2)18.6–23.9

Large288(30.0)27.1–33.0

Numberofbeds – mean ± sd170.5 ± 206.3157.4–183.5

Faith-basedstatus – n(%)

Faithbased173(18.0)15.6–20.6

Non-faithbased787(82.0)79.4–84.4

Hospitaltype – n(%)

Academic237(24.7)22.0–27.5

Non-academic723(75.3)72.5–78.0

StaterequirementifSA – n(%)

In-staterequiringdispense284(29.6)26.7–32.6

Notrequiredtodispense612(63.8)60.6–66.8

Nostatelaw(Ohioand Pennsylvania) 64(6.7)5.2–8.4

CI,confidenceinterval; SA,sexualassault.

Figure. StandardsforReportingofDiagnosticAccuracydiagram reporting flowofparticipantsthroughthestudy.

Table1. Hospitalcharacteristicsofstudysample.

Contraceptionoptions

= 408)

MethodofobtainingEC

= 408)

Accesstoreferrals(ifECnot available,n = 551)

*FullaccessvaluesarehospitalsthatansweredyestohavingEC wheninitiallyasked.

**Conditionalaccessvaluesarehospitalsthatrespondedno tohavingECavailableinitially,butyeswhensexualassault wasreported.

CI,confidenceinterval; EC,emergencycontraception; IUD, intrauterinedevice; SANE,sexualassaultnurseexaminer.

havingSANEswhencomparedtothoseathospitalswithout reportedECaccess(Table3).

Mid-sizedandlargehospitalswere2.96(95%CI 1.72–5.11)and2.43(95%CI1.63–3.61)timesmorelikely, respectively,toreporthavingSANEaccessthansmall hospitals.Priortoadjustingforcovariates,itappearedas thoughurbanhospitalsweremorelikelytoreporthaving SANEaccess(1.39timesmorelikely, Table3).Inthe

adjustedmodel,however,ruralhospitalswere1.48times morelikely(95%CI1.00–2.20)toreporthavingSANEs, illustratingareversaloftheassociationwithhospitalsize actingasthequalitativeconfounder(Table3).Faith-based andnon-faith-basedhospitalsreportedhavingSANEs availableatsimilarratesof51.7%and48.2%,respectively (Table3).Academichospitalswere2.18timesperlikely(95% CI1.42–3.34)toreporthavingSANEaccessthannonacademichospitals(Table3).

Ofthe960hospitalsincluded,551(57.4%)reportedno accesstoEC.Ofthe408(42.5%)reportingECaccess,215 (22.4%)hadfullaccess,and193(20.1%)hadconditional access(Table2).Ofthe551hospitalswithnoaccess,341 (61.9%)hadareferralsystemtoobtainEC.Ofthe408 hospitalswithreportedECaccess,196(48.0%)prescribed levonorgestrel,six(1.5%)prescribedulipristalacetate,and 216(52.9%)ofrespondentsdidnotknowtheavailable options.Nohospitalsreportedthecopperintrauterinedevice (IUD)asanoption.Themajorityofrespondentstoldcallers thatEDsleaveECprovisiontothediscretionofthephysician (62.7%),2.9%requireapregnancytest,and6.4%requirea pelvicexamination(Table2).Nationally,thepredicted probabilityofarespondentreportingthattheirhospitaldid notprovideanyECintheEDwas55.2%(TableS3).

Massachusetts,Oregon,NewJersey,NewYork, Washington,andWisconsinhadasignificantlygreater predictedprobabilityofreportedECaccessinEDsthanthe nationalaverage,whileFlorida,California,Kansas, Louisiana,Texas,andNebraskahadasignificantlylower chanceofhavingECthanthenationalaverage(TableS3). ThepresenceofastaterequirementtoprescribeECfor sexualassaultvictimswasthesecondstrongestpredictorof ECaccess(followingregion),at2.27timesmorelikely(95% CI1.59–3.22).Additionally,ruralhospitalswere1.65times morelikely(95%CI1.11–2.44)thanurbanhospitalstohave anyECaccess,andacademichospitalswere1.58timesmore likely(95%CI1.05–2.39)tthannon-academichospitalsto haveanyECaccess(Table4;seeunadjustedcomparisions in TableS4).

Afteradjustingforcovariates,reportedECaccesswas associatedwithhospitalregion,urbanstatus,academic status,andstaterequirementincasesofsexualassault (Table4).Afterexcludinghospitalswithreported conditionalaccessincasesofsexualassault,faith-based statusbecameanadditionalindependentpredictor,whilethe associationbetweenacademicstatusandECaccesswasno longersignificant(Table5).

DISCUSSION

Globally,ratesofsexualassault,gender-basedviolence, andhumantraffickingforsexualexploitationremainhigh, andaccesstoappropriatecarefollowingasexualassault remainsmarkedbysharpdisparities.23–25 Similarly,our studysuggeststhatthereisinconsistentaccesstoSANEsand

Table2. Emercencycontraceptionsurveyresponseby hospitalsample.

Urban

HospitalcharacteristicsUnadjustedmodelOR(95%CI)AdjustedmodelOR(95%CI)

Sizeofhospital

–0.95)1.48(1.00–2.20)

SmallRefRef

Medium2.26(1.61–3.19)2.96(1.72–5.11)

Large2.76(2.03–3.76)2.43(1.63–3.61)

Numberofbeds(per250increase)1.49(1.24–1.80)0.91(0.70–1.18)

Faith-basedstatus

Non-faithbasedRefRef

Faithbased1.16(0.83–1.63)1.01(0.70–1.46)

HospitalType

Non-academicRefRef

Academic2.87(2.08–3.96)2.18(1.42–3.34)

StaterequirementifSA

In-staterequiringdispenseRefRef

Notrequiredtodispense0.65(0.49–0.87)0.95(0.66–1.38)

Nostatelaw(OhioandPennsylvania)1.42(0.81–2.52)0.81(0.42–1.53)

ECaccess

NoaccessRefRef

Fullaccess*2.87(2.05–4.00)2.33(1.62–3.34)

Conditionalaccess**4.82(3.33–6.97)3.94(2.66–5.83)

OR,oddsratio; CI,confidenceinterval; SA,sexualassault; EC,emergencyconception.

ECinEDsacrosstheUS.Whilethisstudydoesnotestablish thedistributionofabsoluteaccess,ourmethodology providesapragmaticdepictionofthepatientexperience whenattemptingtoaccesssexual-assaultservicesandEC throughanED.Thispicturereflectsstarkdisparitiesin accessaswellasoveralllowlevelsofaccesstoSANEsand ECnationally.Our findingshighlightthedifferencebetween policyandpractice,whichmaybeinfluencedbybias,lackof knowledgeofpolicybyclinicians,andotherfactors.

RoughlyhalfoftheEDssurveyedreportedthattheycould notprovideSANEsforsexualassaultvictimsonsitewithin sixhours,andrespondersintheSouthweretwiceaslikelynot toknowwhethertherewasaSANEavailable.This findingis incontrasttootherstudiesconductedintheSoutheasternUS thatreliedonclinicianandadministratorsurveys,which

foundthataccesstoSANEandECwasconsistentwiththe standardofcare.26 Itis,therefore,unclearwhetherthis regionaldifferencerepresentstrueavailabilityoragapin theeducationoffrontlineemergencycliniciansinthe southernUS.

Largeracademicinstitutionsweremorelikelytohavea SANEavailable,possiblybecauseforthoseinstitutionsit waslessofa financialburden.ThecosttodevelopaSANE programcanbeupto$40,000.27 Accordingtothe InternationalAssociationofForensicNursing(IAFN),only 1,200IAFN-certifiedSANEsforadultsandadolescentsare availableinternationally.28 Asaresult,disparitiesinaccess arelikely,andalthoughthereasonsarenotwellstudied,they likelyincludeanumberofvariablessuchashighcosts, limitedtrainingopportunities,andalackofsupportive

Table3. Unadjustedandadjustedoddsratiosofaccesstoasexualassaultnurseexaminerbyhospitalcharacteristics(availablevs notavailable).

HospitalcharacteristicsUnadjustedmodelsOR(95%CI)AdjustedmodelOR(95%CI) Region

NortheastRefRef

Midwest0.31(0.20–

–0.65)

South0.16(0.10–0.26)0.25(0.15–0.43)

West0.32(0.20–

Urban–ruralstatus

UrbanareaRefRef

–0.57)

Ruralarea1.05(0.80–1.37)1.65(1.11–2.44)

Sizeofhospital

SmallRefRef

Medium1.31(0.94–1.84)1.35(0.91–2.00)

Large1.59(1.81–2.15)1.60(0.93–2.73)

Numberofbeds(per250increase)1.18(1.01–1.39)1.07(0.83–1.38)

Faith-basedstatus

Non-faithbasedRefRef

Faithbased0.91(0.65–1.28)0.90(0.62–1.30)

Hospitaltype

Non-academicRefRef

Academic1.67(1.25–2.25)1.58(1.05–2.39)

StaterequirementifSA

In-staterequiringdispenseRefRef

Notrequiredtodispense0.33(0.25–0.45)0.44(0.31–0.63)

Nostatelaw(OhioandPennsylvania)0.93(0.54–1.62)0.55(0.30–1.02)

Anyaccess*includeshospitalswithfullaccess**orconditionalaccess***.

Fullaccess**valuesarehospitalsthatansweredyestohavingemergencycontraceptionavailablewheninitiallyasked. Conditionalaccess***valuesarehospitalsthatrespondednotohavingECavailableinitially,butyeswhensexualassaultwasreported. OR,oddsratio; CI,confidenceinterval; SA,sexualassault.

resources,particularlyinalreadyunderservedareas.29 Our results,incombinationwiththeincreaseinthenumberof sexualassaultpatientsbeingseenintheED,1 highlightthe needforhospitalstobepreparedwithproperlytrainedstaff totreatthispatientpopulation.Onepossiblesolutiontothe costofSANEservicesforindividualhospitalsisto regionalizeresources.

Intheunadjustedmodel,ruralhospitalsappearedless likelytohaveSANEsavailable;however,onceadjustedfor hospitalsize,ruralhospitalsweremorelikelytoreport havingaSANE.Thisiscontrarytopreviousresearchin Pennsylvania,Washington,andOregon,which demonstratedthatprogramsinruralareaswerelackingin SANEsandfacilities,resultinginurbanprogramsabsorbing patientsfromunderservedareas,7,8 Thismaybearesultof theavailabilityofsexualassaultresourcesoutsidetheEDin urbanareas,orofthecentralizationofSANEsatasingle hospitalinanurbancenter.Iftheresultsofthisstudy representaccesstoSANEs,ratherthanalackofknowledge

amongfrontlinehealthcarepractitioners,thereisastrong disparityinSANEaccessforsexualassaultpatientsbasedon regionandhospitalsize.Thisdisparitymayaffectthequality ofcounselingandforensicevidencecollectionbasedonthe locationofthehospital,whichcouldhavelegalramifications forvictimsashospitalsindifferentlocationsmaynotequally facilitatethecollectionofhigh-qualityevidenceincasesof sexualassault.

Only22.4%ofEDfrontlinehealthcarepractitioners reportedthattheyprovideECwithoutrestriction; furthermore,anadditional20.1%reportedthatthey providedEConlyincasesofsexualassault.Ourresultsalign withthosereportedbyHarrisonetalin2005,withaminority (31.5%)ofsurveyedEDsfoundtoprovideEC.16 Thepoor accesstoECfoundinthisstudymayinpartreflectincreased accesstoalternativeresources,suchasover-the-counterEC atpharmaciesorwomen’sspecialtyclinics.Thelowrateof accessreportedbyEDpersonnelmayalsobeduetolackof knowledgeofhospitalpoliciesregardingECamongfrontline

Table4. Unadjustedandadjustedoddsratiosofemergencycontraceptionaccessbyhospitalcharacteristics(anyaccess*vsnoaccess).

HospitalcharacteristicsUnadjustedmodelsOR(95%CI)AdjustedmodelOR(95%CI) Region

NortheastRefRef

Midwest0.22(0.13

South0.14(0.08

West0.30(0.18–

Urban–ruralstatus

UrbanareaRefRef

Ruralarea1.14(0.81–1.60)1.74(1.05–2.87)

Sizeofhospital

SmallRefRef

Medium1.34(0.75–1.72)1.11(0.67–1.82)

Large1.42(0.99–2.04)1.28(0.67–2.46)

Numberofbeds(per250increase)1.16(0.97–1.39)1.16(0.86–1.55)

Faith-basedstatus

Non-faithbasedRefRef

Faithbased0.45(0.27–0.75)0.44(0.25–0.76)

Hospitaltype

Non-academicRefRef

Academic1.61(1.12–2.30)1.67(0.99–2.82)

StaterequirementifSA

In-staterequiringdispenseRefRef

Notrequiredtodispense0.32(0.22–0.45)0.42(0.27–0.67)

Nostatelaw(OhioandPennsylvania)0.93(0.54–1.62)0.54(0.25–1.14)

Fullaccess*valuesarehospitalsthatansweredyestohavingemergencycontraceptionwheninitiallyasked. OR,oddsratio; CI,confidenceinterval; SA,sexualassault.

EDstaff,especiallyaboutcostsandbarriersassociatedwith thesealternativeresources.27 SimilartowhatHarrisonetal reported,respondentsinourstudyfrequentlyprovided incorrectormisguidedcommentsregardingEC.Several respondentsreferredtoECasan “abortionpill,” possibly mifepristone,orahysterectomyduringthephonecall.

AccordingtotheEDnon-physicianpractitionerssurveyed, 63%ofECprovisionwasbasedonindividualphysician discretion,whichisnotrequiredinmanystates.

Studieshavedemonstratedthatlessthan50%ofvictimsof sexualassaultseekmedicalattention.Whilethereasonsare multifactorial,itisclearthatvictimsexperienceserious psychosocialandemotionalstressthatmaycontributetoa reluctancetobesubjecttoadditionalscrutiny,lossof privacy,orinvasiveexaminations.30 Inoursurvey,many reportedthatECwasdispensedonlyfollowingaphysician assessment,whichwouldincludeapelvicexamination. ManyrespondentsinourstudystatedthattheirEDdidnot takesexualassaultcasesandthatthepatientwouldneedto betransferredtoanotherfacilityorseekguidancefrom lawenforcement.

FewrespondentsprovidedthespecificbrandofEC available,andnoneofferedthecopperIUDasanoption. Manyrespondentscommonlyreferredpatientstoprivate pharmaciesforEC,aproblematicpracticegivencoverageof costsandpotentiallogisticaldifficultiesanddelays.

Importantly,thereferralofthoseseekingECtoprivate pharmacieslimitsaccesstoconsultationonsexually transmitteddisease,behavioralhealth,ortheopportunityto reporttolawenforcementinthecaseofvictimsofsexual assault,domesticviolence,ortrafficking.These findings underscoretheneedforincreasedtrainingforhealthcare practitionersresponsiblefortriageandresponsetoinquiries.

InstateswithlegislationrequiringaccesstoECincasesof sexualassault,EDsweremorethantwiceaslikelytoreport thatECwasavailablewithoutrestriction,demonstrating thatsuchlegislationmayhaveanimpact.Withcurrent enforcementmechanismsinplaceforonly13states, thereisroomforexpansionoflegislationtocoverthe remainingstates.

Perhapsunexpectedly,ECwasmorelikelytobeavailable inruralhospitalsafteradjustingforcovariates.Rural

Table5. Unadjustedandadjustedoddsratiosoffullemergencycontraceptionaccessbyhospitalcharacteristics(fullaccess*vsnoaccess).

hospitalsoftenserveascriticalaccesspointsforremoteor underservedcommunities.Non-faith-basedhospitalswere morethantwiceaslikelytoreportprovidingECthanfaithbasedhospitals,consistentwithapreviousstudyinwhich non-CatholichospitalsweremorelikelytoprovideECthan Catholichospitals.21 This findingmaybebasedon localinstitutionalpolicieslimitingaccessamong faith-basedinstitutions.

Thereisaneedforimprovededucationonsexualassault care,aswellasanincreaseinSANEaccessamonghospitals. HospitalsshouldconsiderbuildingSANEresourcesintoED protocols.Hospitaladministratorscancollaboratewithlocal rapecrisiscentersorapplyforfederalgrantsorfundingto defraythecostoftrainingandsupplies.

LIMITATIONS

Theprimarylimitationofthisstudywastheinconsistency inknowledgeofprotocolsrelatedtothistopicand willingnesstoprovideaccurateinformationoverthe telephone.Itisplausiblethatcallerswouldhavereceived differentinformationhadtheencounterbeeninperson. However,aphoneprotocolwasspecificallychosenasa pragmaticapproachusedbyapotentialmemberofthe communityseekingservices.

Thespeci ficinquiryregardingsexualassaultratherthan theinitialrequestforECmayhavein fl uencedthe respondent ’ sresponseregardingresourcesandaccess. Respondentsinthisstudymayhavebeenmoremotivatedto fi ndananswertoquestionswhenthetopicofsexualassault wasintroduced.Forexample,somerespondentswhostated ECwasnotavailablechangedtheirresponseuponthe callers’ disclosurethattherehadbeenasexualassault. WhenrespondentsendorsedSANEaccess,callersdidnot recordon-siteavailability,northehourswhenaccesswas available.Calltimingwasvariedrandomlybetween 9 AM– 5 PM butwasnotstandardized.Inaddition,asmany sexualassaultspresentoutsidenormalworkinghours, itispossiblethattheavailabilitycouldbeevenlower duringoff-hours.

CONCLUSION

Accesstoemergencycontraceptionandsexualassault nurseexaminersinEDsremainslimitedwithdisparitiesin accessacrossthenation.Variableaccessibilitydependingon thegeographiclocationofthehospitalorthelegislative statusofthestatesuggeststhatthoseseekingtheseresources mightreceivesubstandardqualityofhealthcaredepending ontheinstitutionwheretheyhavechosentoseekcare.Given theimportanceofECandsexualassaultservices,emergency physiciansmay finditworthwhiletoexaminetheirhospitals’ existingprotocolsregardingdispensingprescriptionsofthese medicationsandavailabilityofSANEs.Hospitalsshould considerprovidingtrainingforallEDstaff,especiallythose

who firstinteractwithpatients,topreventmisinformation aboutpatientaccesstoECorSANEs.

AddressforCorrespondence:Marie-CarmelleElie-Turenne,MD, UniversityofAlabamaatBirmingham,HeersinkSchoolofMedicine, DepartmentofEmergencyMedicine,1670UniversityBlvd, Birmingham,AL35233.Email: melie@uabmc.edu

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024Cowderyetal.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES

1.VogtEL,JiangC,JenkinsQ,etal.(2022).TrendsinUSemergency departmentuseaftersexualassault,2006-2019. JAMANetwOpen 2022;5(10):e2236273.

2.NielsonMH,StrongL,StewartJG.Doessexualassaultnurse examiner(SANE)trainingaffectattitudesofemergencydepartment nursestowardsexualassaultsurvivors? JForensicNurs 2015;11(3);137–43.

3.CianconeAC,WilsonC,ColletteR,etal.Sexualassaultnurse examinerprogramsintheUnitedStates. AnnEmergMed 2000;35(4):353–7.

4.CampbellR,PattersonD,LichtyLF.Theeffectivenessofsexual assaultnurseexaminer(SANE)programs:areviewofpsychological, medical,legal,andcommunityoutcomes. TraumaViolenceAbuse 2005;6(4):313–29.

5.Fehler-CabralG,CampbellR,PattersonD.Adultsexualassault survivors’ experienceswithsexualassaultnurseexaminers(SANEs). JInterpersViolence.2011;26(18):3618–39.

6.SchmittT,CrossTP,AlderdenM.Qualitativeanalysisofprosecutors’ perspectivesonsexualassaultnurseexaminersandthecriminaljustice responsetosexualassault. JofForensicNurs.2017;13(2):62–8.

7.ClowersN.Sexualassault:informationontheavailabilityofforensic examiners. JusticeLawEnforcRep.2018;2019:279–92.

8.ThiedeEandMiyamotoS.Ruralavailabilityofsexualassaultnurse examiners(SANEs). JRuralHealth.2021;37(1),81–91.

9.PracticeBulletinNo.152:EmergencyContraception. ObstetGynecol 2015;126(3):e1–11.

10.AmericanCollegeofEmergencyPhysicians.Emergencycontraception forwomenatriskofunintendedandpreventablepregnancy. Annof EmergMed.2005;46(1):103.

11.LewisNV,MooreTH,FederGS,etal.Useofemergencycontraception amongwomenwithexperienceofdomesticviolenceandabuse: asystematicreview. BMCWomensHealth, 2018;18(1):156.

12.GuttmacherInstitute.InsuranceCoverageofContraceptives.Available at: https://www.guttmacher.org/state-policy/explore/ insurance-coverage-contraceptives.AccessedMarch3,2020.

13.ContraceptionASoE.NotThereYet:ASEC’s2017ECAccessStudy. Availableat: https://www.americansocietyforec.org/_files/ugd/ 7f2e0b_0e4b5a581359421c84b67abafabf8e30.pdf AccessedMarch3,2020.

14.QatoDM,ZenkS,WilderJ,etal.Theavailabilityofpharmaciesinthe UnitedStates:2007–2015. PloSOne.2017;12(8):e0183172.

15.PragerS.Accesstoemergencycontraception. ObstetGynecol 2017;130(1):E48–52.

16.HarrisonT.Availabilityofemergencycontraception:asurvey ofhospitalemergencydepartmentstaff. AnnEmergMed 2005;46(2):105–10.

17.ClelandK,ZhuH,GoldstuckN,etal.Theefficacyofintrauterine devicesforemergencycontraception:asystematicreviewof35yearsof experience. HumReprod.2012;27(7):1994–2000.

18.AdministrationUSFaD.Ella(ulipristalacetate)Tablets.2015.Available at: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2015/ 022474Orig1s007TOC.cfm.AccessedMarch3,2020.

19.GuttmacherInstitute.EmergencyContraception.Availableat: https:// www.guttmacher.org/state-policy/explore/emergency-contraception

AccessedMarch3,2020.

20.NationalWomen’sLawCenter.ProvidingEmergencyContraceptionto SexualAssaultSurvivors:ElementsofaSuccessfulStateECintheER Law.2013.Availableat: https://nwlc.org/wp-content/uploads/2015/08/ providing_ec_to_sexual_assault_survivors_factsheet_6-28-13.pdf AccessedMarch3,2020.

21.PolisC,SchafferK,HarrisonT.Accessibilityofemergency contraceptioninCalifornia’sCatholichospitals. WomensHealthIssues 2005;15(4):174–8.

22.WangMJ,KhodadadiAB,TuranJM,etal.Scopingreviewofaccessto emergencycontraceptionforsexualassaultvictimsinemergency departmentsintheUnitedStates. TraumaViolenceAbuse 2021;22(2):413–21.

23.CepedaI,Lacalle-CalderonM,TorralbaM.Measuringviolence againstwomen:aglobalindex. JInterpersViolence 2022;37(19-20):NP18614–38.

24.DworkinER,KrahéB,ZinzowH.Theglobalprevalenceofsexual assault:asystematicreviewofinternationalresearchsince2010. PsycholViolence.2021;11(5):497–508.

25.UnitedNationsOfficeonDrugsandCrime.GlobalReportonTrafficking inPerson.2009.Availableat: https://www.unodc.org/documents/ Global_Report_on_TIP.pdf.AccessedSeptember15,2021.

26.HarrisonT,FosterAM,MartinSB,etal.Assessingsexualassault survivors’ accesstoemergencycontraception:resultsfromamixed methodsstudyinSouthCarolina. WomensHealthIssues 2010;20(4):248–53.

27.HoumesBV,FaganMM,QuintanaNM.Establishingasexualassault nurseexaminer(SANE)programintheemergencydepartment. JEmergMed.2003;25(1):111–21.

28.ColbertAMandSekulaLK.AmajorinvestmentintheUnitedStates sexualassaultnurseexaminerworkforce. JForensicNurs 2022;18(1):1–3.

29.IritaniK.Sexualassault:informationontraining,funding,andthe availabilityofforensicexaminers.2016.Availableat: dspacedirect.org AccessedFebruary4,2024.

30.ReevesEAandHumphreysJC.Describingthehealthcareexperiences andstrategiesofwomensurvivorsofviolence. JClinNurs 2018;27(5–6):1170–82.

LETTERSTOTHE EDITOR

FactorsAssociatedwithOverutilizationofComputedTomography

CervicalSpineImaging

TessyLaTorreTorres,DO

JonathanMcGhee,DO

ChristianaCareHospital,DepartmentofEmergencyMedicine,Newark,Delaware

SectionEditor:MarkLangdorf,MD,MHPE

Submissionhistory:SubmittedNovember12,2023;RevisionreceivedNovember12,2023;AcceptedDecember8,2023 ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18570

DearEditor:

Wearewritingtoprovidesomecommentsonthescientific paperrecentlypublishedinyourjournaltitled “Factors AssociatedwithOverutilizationofComputedTomography oftheCervicalSpine.”1

Firstly,wecommendtheauthorsforputtingtogethera relevantandwell-donemulticenterstudythatboth revalidatestheNEXUScriteriaandoffersinsightintothe overutilizationofcomputedtomography(CT)fortraumatic injuries.However,wedohavesomeconcernsaboutthe methodologyused.Havingasinglereviewercollectingchart dataonNEXUScriteria criteriathatweknowincludethe subjectivecomponentofadistractinginjuryordeficitnot attributabletopain introducesthepossibilityofbias;it would,therefore,bebeneficialtoseecongruenceofchart analysisbetweendifferentreviewers.Itisalsorecognizable thatthereweretimingconstraintsrelatedtofeasibility,thus allowingforonlyonepersontorevieweachchartforthe presenceofNEXUScriteria.Theprocessinvolvedcombing throughmorethan800individualrecordsthatincluded physiciandocumentation,imaging,labstudies,andnursing notes.Thisaddedsignificantlytotheworkloadofthesingle reviewer,whichcouldhaveimpactedtheoverallaccuracyof thedatacollected.Additionally,itwasunclearwhetherthe reviewerwasblindtotheresultoftheCTwhenreviewingthe chart,openingupfurtheropportunitiesforbias.

Secondly,theshorttimeframeinthiscaselinkedtoskiing/ wintersports-relatedinjuriesmayprovideonlyapartial picture,limitingtheapplicabilityofresults.Imaginethe studyhadbeenconductedforlongerthantwomonths outsidethewinterseason.Wouldtherebeadditional variablesregardingthemechanismofinjuryassociatedwith theoverutilizationofCTimagingnototherwiseuncoveredin theirinitialreview?Additionally,thebaselinecharacteristics forthepresentingmechanismofinjuryincludedfalls,which constitutedapproximately75%ofthetotalnumber.Further

characterizationofthemechanismofinjurymayalsohave beenbeneficial fallfromstandingvsfromaheight,or motorvehiclecollisionwithairbagdeploymentvswithout couldallrevealassociationsofinjurythatwouldcause physicianstobypasstheNEXUScriteriaaltogether.

Futurestudiesshouldlooktoinvestigatewhethera physician-in-triagestructureisassociatedwithincreasedCT overutilization.Weareseeingmoreprotocolsbeing implementedinemergencymedicine,includingwithinthe triageprocess,anditwould,therefore,beinterestingtosee howthisalternateworkflowwouldaffectresults.

Overall,wefoundtheauthors’ studytobeextremely informative,andweappreciatetheircontributiontotheever evolvingandhighlychallenging fieldofemergencymedicine.

AddressforCorrespondence:TessyLaTorreTorres,DO, ChristianaCareHospital,DepartmentofEmergencyMedicine, 4755Ogletown,StantonRd,Newark,DE19718.

Email: tessy.latorretorres@christianacare.org

ConflictsofInterest:Bythe WestJEMarticlesubmissionagreement, allauthorsarerequiredtodiscloseallaffiliations,fundingsources and financialormanagementrelationshipsthatcouldbeperceived aspotentialsourcesofbias.Noauthorhasprofessionalor financial relationshipswithanycompaniesthatarerelevanttothisstudy. Therearenoconflictsofinterestorsourcesoffundingtodeclare.

Copyright:©2024LaTorreTorresetal.Thisisanopenaccess articledistributedinaccordancewiththetermsoftheCreative CommonsAttribution(CCBY4.0)License.See: http:// creativecommons.org/licenses/by/4.0/

REFERENCE

1.ChamberlinKT,CanellasMM,ReznekMA,etal.Factorsassociated withoverutilizationofcomputedtomographyofthecervicalspine. WestJEmergMed. 2023;24(5):967–73.

LETTERSTOTHE EDITOR

Replyto “FactorsAssociatedwithOverutilizationofComputed TomographyCervicalSpineImaging”

KarlChamberlin,MD,MBA*† *UniversityofMassachusettsChanMedicalSchool,EmergencyMedicine, Worcester,Massachusetts

† UMassMemorialHealth,DepartmentofEmergencyMedicine, Worcester,Massachusetts

SectionEditor:MarkLangdorf,MD,MHPE

Submissionhistory:SubmittedDecember5,2023;RevisionreceivedDecember5,2023;AcceptedDecember8,2023 ElectronicallypublishedFebruary28,2024

Fulltextavailablethroughopenaccessat http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.18614

November29,2023

DearEditor:

Weappreciatethefeedbackandcommentaryonour recentlypublishedstudy.First,wewouldliketo acknowledgethelimitationsofhavingasinglereviewer performingallchartreviews.Tomitigatepotentialbias, wecreatedobjectivebinarydefinitionsforeachNEXUS criterion.Forexample,weexplicitlydefineddistracting injuriestoincludeonlyradiographic findingsoflongbone fracturesormultipleribfractures,oranyinjuriesthat weredescribedexplicitlyas “distracting” intheclinical documentation.Similarly,alldocumentedneurological deficitswereassumedtofulfillthiscriterionunlesstheclinical documentationexplicitlyascribedthedeficittopain.While thesedefinitionsaremoreexplicitandobjectivethanthe originalNEXUScriteria,wefeltthismodificationwas necessaryduetotheretrospectivenatureofourstudy,asour methodologyinherentlyprohibitedusfromprospectively askingclinicianswhetheraninjurywasdistracting,for example.Furthermore,wefeltthatourdefinitionswere alignedascloselyaspossibletotheoriginalcriteriaand, therefore,providedanaccurateestimationofoverutilization. Ideally,futurestudiesonthetopicshoulduse multiplereviewersandprospectivelycollectdataonthe subjectivecriteria.

Secondly,weagreethatamoregranularanalysisof mechanismsofinjurycouldrevealadditionalassociations withoverutilization.Inouranalysis,noneofthemechanisms ofinjuryweresignificantlyassociatedwithCT overutilization,andseasonalmechanismsaccountedfora verysmallpercentage.Ourcategorizationofmechanismwas intentionallybroad,aswesoughttoidentifypotentially meaningfultargetsforfutureinterventionsaimedatreducing overutilization.Whileitispossiblethatasubsetof mechanismswithinagivencategorycouldbestatistically

significant,thiswouldrequirefurtherinvestigation,anditis notclearwhetherthose findingswouldyieldclinically importanttargetsforintervention.

Lastly,andmostimportantly,wewouldliketoemphasize theletterwriter’spointabout flowandtriageprocessesthat mayaffectoverutilizationofCTresources.Becausenoneof thesitesincludedinthisstudyuseaclinician-in-triage process,wewereunabletodirectlyassessthisassociation fromourdataset.However,strategiestoimproveemergency department(ED)throughput(eg,clinician-in-triagestaffing models,direct-to-CTprotocols,nurse-initiatedorders)have proliferatedacrossthecountryinrecentyears,despite thesestrategieshavinguncertainimpactsonresource overutilization.Aswedevelopandimplementnovel flowand triageprocesses,itisparamountthatweconsiderthe secondaryeffectsonhealthcarecosts,radiationexposure, incidental findings,andhospitalresources.Atatime whenEDcrowdingandboardinghavereachedcrisis levels,effectiveresourceutilizationisessentialfor operationalsuccess.

Thankyou.

KarlChamberlin,MD,MBA

AddressforCorrespondence:KarlChamberlin,MD,MBA,University ofMassachusettsChanMedicalSchool,DepartmentofEmergency Medicine,55LakeAvenueNorth,Worcester,MA01655.

Email: karl.chamberlin@umassmemorial.org

Copyright:©2024Chamberlin.Thisisanopenaccessarticle distributedinaccordancewiththetermsoftheCreativeCommons Attribution(CCBY4.0)License.See: http://creativecommons.org/ licenses/by/4.0/

CALIFORNIAACEPSECURED $200MANNUALLYTO INCREASEMEDI-CAL REIMBURSEMENTFOR EMERGENCYPHYSICIANS!

EMERGENCY MEDICINE IS THE ONLY SPECIALTY WITH ITS OWN BUDGET ALLOCATION.