College of Computing & Informatics

Neha Kamireddi

School of Biomedical Engineering, Science, & Health Systems

Biomedical Engineering

Faculty Mentor: Dr. Aleksandra Sarcevic

Information Science

Swathi Jagannath Co-Mentor

Speech Modeling for Automatic Activity Recognition in Trauma Resuscitation

Trauma is the leading cause of death and disability in children and young adults. Although trauma teams follow a standardized protocol, Advanced Trauma Life Support (ATLS), errors are still observed. In trauma resuscitations, verbal communication carries invaluable information for an activity recognition system. However, two main challenges in using speech are: (1) few existing models concerning the nature of speech in a medical setting, and (2) due to frequent parallel speech, verbal communication often becomes jumbled and information gets lost.

For this project, 98 audio recordings were collected from a level 1 trauma center of an urban, pediatric teaching hospital in the U.S. mid-Atlantic region. These audio recordings were manually transcribed into spreadsheets and the speech lines were correlated to twenty-nine activities defined in the ATLS protocol. This speech was then analyzed to construct speech workflow models, or narrative schemas for all activities. These schemas will serve as the conceptual basis for speech input in an automatic activity recognition system that will be able to provide real time feedback about errors and process deviations to clinicians.