Emotion AI: Separating Facts from Fiction

I N T E R D I S C I P L I N A RY AFFECTIVE SCIENCE L A B O R AT O RY

Emotion AI: Separating Facts From Fictions

Lisa Feldman Barrett, Ph.D. Northeastern University

Massachusetts General Hospital

Emotions are universally displayed on the face with expressions we recognize.

N O. 1

fl

Con ating observations and inferences

Reliability vs. False Positives 1 0.9 0.8 0.7 .62

0.6 0.5

.60

.55

.58 .52

.50 .42

0.4 .35

0.3

.28 .25 .20

0.2

.13

0.1 0

N=3

N=1

N=5 N=3

Anger

Disgust

Fear

Happiness

N=2

Sadness

N=16

Surprise

Duran & Fernandez-Dols (2021) Le Mau et al (2021)

Barrett, Adolphs, et al. (2019)

Actual Movements vs. Perceptions 1 0.9 .79

0.8 0.7

.68

.68 .65 .61

0.6

.58

0.5 .42

0.4 .35

0.3

.28 .25 .20

0.2

.13

0.1 0

N=3

N=1

N=5 N=3

N=2

N=16

Duran & Fernandez-Dols (2021) Elfenbein & Ambady (2002)

Barrett, Adolphs, et al. (2019)

Small Scale Cultures: 1969-2007

Fore of Papua New Guinea Ekman, Sorenson & Friesen (1969) Sadong of Borneo Ekman et al. (1969)

Fore of Papua New Guinea Ekman & Freisen (1971) Dani of Indonesia (Western New Guinea) Ekman (1972)

Gendron et al. (2018). Current Directions in Psychological Science

Small Scale Cultures: 2008-2020

Shuar of Amazonian Ecuador Bryant & Barrett (2008)

Dioula of Burkina Faso Tracy & Robins (2008)

Himba of Namibia (2 samples) Gendron et al. (2014)

Hadza of Tanzania Gendron et al. (2020)

Trobrianders of Papua New Guinea Crivelli, Jarillo et al. (2016)

Mwani of Mozambique Crivelli, Jarillo et al. (2016)

Trobrianders of Papua New Guinea (2 samples) Crivelli, Russell et al. (2016) Trobrianders of Papua New Guinea (2 samples) Crivelli, Russell et al. (2017) Gendron et al. (2018). Current Directions in Psychological Science

19

STEREOTYPES Barrett, Adolphs, Marsella, Martinez, & Pollak (2019), PSPI.

Variation is the norm

Le Mau et al (2021). Nature Communications

Physical signals have no inherent psychological meaning

Barrett (2022) American Psychologist

Anger

Physical signals have no inherent psychological meaning

Barrett (2022) American Psychologist

Oxygen

Glucose

Vibration

Your brain models the sensory surfaces of the body

Brightness

Muscle tension Barrett (2022) American Psychologist

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

Makes Meaning

Each emotion has a dedicated pattern of bodily changes that is universal.

N O. 2

Anger

Fear

Disgust

Happy

Sad

0.2

0.4

0.6

Neutral

Heart Rate

Cardiac Output

Total Peripheral Resistance

Heart Rate Variability

Diastolic Blood Pressure

Systolic Blood Pressure

Respiration Rate

Skin Conductance Level Skin Conductance Response -0.4

-0.2

0

Change in signal during instance of emotion

0.8 Siegel et al. (2018). Psychological Bulletin

32

Physiological changes are linked to actions

Obrist et al. (1970); Obrist (1981)

Siegel et al. (2018). Psych Bull

fi

Hoemann et al. (2020). Scienti c Reports

Siegel et al. (2018). Psych Bull

fi

Hoemann et al. (2020). Scienti c Reports

Siegel et al. (2018). Psych Bull

fi

Hoemann et al. (2020). Scienti c Reports

A relational view of emotional meaning

Low Glucose

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

A relational view of emotional meaning

Hunger Low Glucose

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

A relational view of emotional meaning

Low Glucose

Fatigue

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

A relational view of emotional meaning

Low Glucose

Sadness

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

There are dedicated emotion circuits in the brain.

N O. 3

Top of brain

Left

Right Bottom of brain

Amygdala

Lindquist et al. (2012), BBS.

Proportion of Studies Reporting Increased Amygdala Activity 0

10

20

30

40

50

60

70

Anger Disgust Fear Happy Sad

Lindquist et al. (2012), BBS.

s e t a d i d n a c y r e g r u s y s p e l i p e n i s e d o r l e ct Front of brain

300-600 ms after onset of a threat only when the patient is paying attention

Back of Brain

Amygdala Thanks to Ralph Adolphs for the image!

Wang et al (2019) Brain Mormann et al (2008) J Neuroscience

Monozygotic Twins with Amygdala Lesions : Degeneracy Edelman & Gally (2001) PNAS

Amygdala Lesions Front of brain

Back of Brain

BG

AM Becker et al. (2012) Biological Psychiatry

Variation in Patterns for Fear Barrett (2017); Doyle et al. (2022)

Wager et al. (2015)

Kragel et al. (2015)

N = 148 studies

N = 32 (59% female)

Azari, Westlin et al. (under review) N = 16 (50% female)

A pattern is a statistical summary, not a brain state.

Clark-Polner, Johnson & Barrett (2017) Cerebral Cortex

Variation is real.

The signals in your brain constrains this variation to create meaning

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

We reify those experiences and assume that we are detecting experiences

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

Rather than creating them.

Emotions that seem to happen to you are made by you.

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

The emotions you seem to detect in other people are partly inside your own head.

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

Barrett (2017). SCAN Barrett (2022). American Psychologist Barrett (2017). How Emotions Are Made

Thank You.