Realistic Face and Lip Expressions for a Bilingual Humanoid Robot Authors Amna Al-Zeyara (CS 2014) Nawal Behih (CS 2014)
Faculty Advisor Majd F. Sakr, Ph.D.
Category Computer Science
Abstract Hala is a bilingual (Arabic and English) robot receptionist located at Carnegie Mellon in Qatar. Users type to her, and she replies in speech along with realistic facial expressions. However there are two existing problems with the current model. Her Arabic lip-movements are not realistic because they were adopted from English utterances. Secondly, her animation engine does not adhere to existing research on face dynamics, which makes it difficult to leverage/create natural and interesting facial expressions. In this work we address these two limitations. The sounds humans make while talking are symbolically captured as “phonemes.” The corresponding shapes of the lips, for these sounds (i.e. phonemes), are called “visemes.” We used existing research to develop Arabic visemes that accurately capture the Arabic language and its pronunciations. Hala can utilize English visemes, but now also additional Arabic visemes for visually realistic lip-movement and syncing. Hala’s current facial animations are not rich. We used Paul Ekman’s seminal work on Facial Action Coding System (FACS) to demarcate her face into basic muscle-primitives. These muscle-primitives were used to compose complex, yet natural, facial expressions. Additionally, FACS categorizes all possible human facial emotions and we implemented a subset of these. We also identified existing limitations with the model that prevented us from creating certain facial expressions.
9
MoM_Digest_allPages.indd 14
4/20/11 3:03 PM