Functional roles of metacognition

Functional roles of metacognition Marion Rouault Wellcome Centre for Human Neuroimaging, UCL marion.rouault@gmail.com metacoglab.org

Outline of the course 1. Introduction to metacognition: theory and measurement 2. Origins of metacognition in evolution and development 3. Neural architecture of metacognition 4. Functional roles of metacognition in behavioural control

Reminder • Most perceptions, memories and choices are accompanied by subjective estimates of their reliability i.e. confidence estimates Kepecs et al 2008; Kiani & Shadlen 2009

• Metacognition refers to our ability to monitor and evaluate our own cognitive processes Fleming & Dolan 2012; Gehring et al 1993 subjective confidence objective performance

Closing the loop: reciprocal interactions between decision-making and metacognition Decision performance - Evidence - Accuracy - ReacAon Ames - Motor hesitaAon - …

Maniscalco and Lau, 2012 QuesAenne et al., 2017 Kiani et al., 2014 Pouget et al., 2016

Confidence

- behavioural control e.g. setting the switches Donoso et al., 2014 Boureau et al., 2015

- self-efficacy, goals one chooses to pursue Decision-making

Bandura, 1984 Elliott et al., 2003 Wells, 2012

Closing the loop: reciprocal interactions between decision-making and metacognition Decision performance - Evidence - Accuracy - ReacAon Ames - Motor hesitaAon - …

Maniscalco and Lau, 2012 QuesAenne et al., 2017 Kiani et al., 2014 Pouget et al., 2016

Confidence

- behavioural control e.g. setting the switches Donoso et al., 2014 Boureau et al., 2015

- self-efficacy, goals one chooses to pursue Decision-making

Bandura., 1984 Elliott et al., 2003 Wells, 2012

Functional roles of metacognition Outline of the course LOCAL

• Control of subsequent decision-making - Revise immediate decisions - Influence decisions over longer time-scales

• Role of metacognition in learning across domains • Confidence generalisation and transfer • Metacognition: a trait characteristic?

GLOBAL

Control of subsequent decision-making

Control of subsequent decision-making Rat orbitofrontal cortex: ‘confidence’-like paUern

⇒ Adapt waiAng Ame according to confidence esAmate in obtaining reward ⇒ Before iniAaAng a new trial Kepecs et al., 2008

Changes of mind

• Hand movement trajectory betrayed changes of mind of some trials • Evidence accumulaAon goes on a[er iniAal decision is made • Possibility for self-correcAon Resulaj et al., 2009

Changes of mind Dri[-diusion model framework

Post-decision evidence accumula1on van den Berg et al., 2016

See also: Bronfman et al., 2015 Murphy et al., 2015

Changes of mind in the value domain: Folke et al., 2016

Metacognition computations & adaptive behavior • Task with two levels of uncertainty (equivalent to risk and ambiguity in economics) Meyniel et al., 2015 • Subjects had to detect jumps in abstract sequences

Metacognition computations & adaptive behavior • Subjects‘ behaviour displays features of the ideal observer model • CorrelaAon between esAmate of probability of the next sAmuli by the model and by subject • Similarly for confidence esAmate about such probability

Meyniel et al., 2015

Confidence guiding behaviour over longer time-scales

Choice confidence: disambiguating negative feedback Initial diagnosis

Confidence in treatment High conf Reconsider diagnostic

Low conf Persist

‘Global’ confidence to set the decision strategy and guide future behavior

Switching over longer time-scales InteracAons across a hierarchy of decision processes in a dynamically changing environment

Purcell and Kiani, 2016

Switching over longer time-scales InteracAons across a hierarchy of decision processes in a dynamically changing environment

Purcell and Kiani, 2016

Switching over longer time-scales

Correct direction choice + correct environment choice"

Either one or both incorrect"

Purcell and Kiani, 2016

Switching over longer time-scales Unpredictable reversals for the correct environment

Decision strategy: PERSIST with current environment or SWITCH Purcell and Kiani, 2016

Switching over longer time-scales

Purcell and Kiani, 2016

Switching over longer time-scales

Critical factors driving switch: feedback and expected direction choice accuracy Purcell and Kiani, 2016

Switching over longer time-scales

• Expected accuracy i.e. ‘global confidence’ is the key variable determining switching • Different combinations of motion strength and duration that produced the same expected accuracy also produced a similar probability of switching Purcell and Kiani, 2016

Switching over longer time-scales • Effects of expected accuracy = confidence over Ame on switching, over and above effects of feedback • Many real-life decisions lack immediate feedback • Could confidence act as a signal for guiding choices in the absence of feedback?

Learning about self-performance in the absence of feedback • Learning from explicit feedback has been extensively studied

Schultz et al. 1997; SuUon and Barto 1998; Oaksford and Chater 2009, less

is

• Behavioral and computaAonal mechanisms underlying metacogniAve learning in the absence of feedback?

• Hypothesis: confidence could act as a learning signal

Rouault, Dayan and Fleming (in prep)

Task cue"

Rouault, Dayan and Fleming (in prep)

Which box has more dots?"

Correct"

…"

Which game should we use to calculate your bonus?!

Which game should we use to calculate your bonus?!

…"

2 dependent variables indexing confidence: -  Task choice -  Task ability ratings

Which task should we use to calculate your bonus?!

2 by 2 factorial design within-subject Learning block" 2 types of trials interleaved"

Test block" "

Task ability ratings"

Task choice: Prospective metacognitive evaluation

Feedback" No Feedback" Easy" Difficult"

1

24

30 trials

0.9 0.8 0.7 0.6 0.5

0.5 0.4 0.3 0.2 0.1 0

1

Task ability ratings

Mean performance

1

Task choice frequency

Dissociation between objective and perceived ability

0.8 0.6

FB easy FB diff NO FB easy NO FB diff

0.4 0.2 0

In the absence of feedback, task is chosen less often, whereas objective performance is equally good

Learning dynamics Learning blocks 2 types of trials interleaved 6 pairings 5 possible lengths

Which task?

Which game should we use to calculate your bonus?

Retrospective metacognitive evaluation

Feedback" No Feedback" Easy" Difficult"

1

nb of trials

1 0.9 0.8 0.7 0.6 0.5

0.5 Task choice frequency

Mean performance in learning blocks

Performance perceived worse without feedback

0.4

FB easy FB diff NO FB easy NO FB diff

0.3 0.2 0.1 0

Subjects trade-off subjecAve ability against gekng external feedback (replicaAon)

Fluctuations in local performance influence self-performance estimates B

1

1

1

0.5 1

0.5

0.5

0.9 0

0

0

1 0.8

1

1

0.5 0.7

0.5

0.5

FB easy FB diff NO FB easy NO FB diff

0.6 0

S A L

0

S A L

0

0.5 Task choice frequency

number of trials

Mean performance in learning blocks

Task choice frequency

arning blocks

0.4 0.3 0.2 0.1 S A L

S: blocks with Small difference in performance between tasks 0.5 0 A: All blocks together L: blocks with Large difference in performance between tasks

FB easy FB diff NO FB easy NO FB diff

Learning about self-performance in the absence of feedback Feedback Confidence rating Easy

0.7 Diff.

Easy

0.9 0.8 0.7 0.6 0.5

***

Task choice frequency

D Confidence level

C

2.5

0.8

0.6

Contribution to task cho

1

2

*** 1.5 1

~

0.5 0

baseline accDiff

F

0.5

0

S

A

L

S: Small difference in confidence between tasks A: All blocks together L: Large difference in confidence between tasks

3

Metacognitive efficiency

Experiment 3

E

Correct Error

regression coefficient (a. u.)

B Confidence level

A

=0.31 p=0.07

2 1 0

0

rtDiff co

Learning about self-performance in the absence of feedback Feedback Confidence rating Easy

0.7 Diff.

Easy

0.9 0.8 0.7 0.6 0.5

***

Task choice frequency

D Confidence level

C

2.5

0.8

0.6

Contribution to task cho

1

2

*** 1.5 1

~

0.5 0

baseline accDiff

F

0.5

0

S

A

L

S: Small difference in confidence between tasks A: All blocks together L: Large difference in confidence between tasks

3

Metacognitive efficiency

Experiment 3

E

Correct Error

regression coefficient (a. u.)

B Confidence level

A

=0.31 p=0.07

2 1 0

0

rtDiff co

Learning about self-performance in the absence of feedback

y

2.5

0.8 0.7 0.6

Diff.

Easy

D Task choice frequency

Contribution to task choice

1

***

2

*** 1.5 1

~

0.5 0

baseline accDiff

F

rtDiff confDiff

3 =0.31 Local fluctuaAons in confidence explained variance in task choice p=0.07 over and above fluctuaAons in accuracy or reacAon Ames 2

0.5

0

S

A

L

S: Small difference in confidence between tasks A: All blocks together L: Large difference in confidence between tasks

tacognitive efficiency

e rating

E

Correct Error

regression coefficient (a. u.)

Confidence level

B

1 0

Learning about self-performance in the absence of feedback • Subjects trade-off gekng external feedback against internal esAmates of self-performance • Internal confidence as a mediaAng factor when not receiving external feedback • Subjects are able to aggregate confidence over Ame to construct global esAmates about self-performance

Role of metacognition in learning across domains

Learning in the absence of feedback

Hainguerlot et al., 2018

Learning in the absence of feedback

Hainguerlot et al., 2018

Metacognition computations & adaptive behavior

• Subjects learn cues predicAng informaAon about the sAmulus • On some trials, no sAmulus: subjects had to give a prospecAve confidence judgement about the likelihood that they would have goUen it correct Schiffer, Boldt et al., OSF

Metacognition computations & adaptive behavior • ParAcipants used cues to predict selfperformance esAmates in the absence of sAmulus • One key purpose of confidence: preparing future encounters of the task Schiffer, Boldt et al., OSF

In the memory domain: confidence mediating offloading behavior

• Sekng reminders might depend on confidence for further recall • Strategy choices for offloading intenAons, especially when accuracy is incenAvised Risko & Gilbert, 2016; Dunlosky & Metcalfe, 2008

Strategy selection in arithmetic problems • ArithmeAc problems with controlled features:

26 x 42" • Strategy selecAon and/or execuAon: rounding up or down

• Prospec1ve metacogniAve judgements: parAcipants unable to esAmate the ease of strategy selecAon • Retrospec1ve metacogniAve judgements: higher levels of retrospecAve confidence associated with beUer strategy selecAon on the next problem

Geurten & Lemaire, 2017; 2018 Siedlecka et al., 2016

Confidence generalisation and transfer

Confidence could act as a global prior

Rouault et al., 2018

Confidence could act as a global prior

‘Multi-cue’ models of confidence ⇒ Not only single decision evidence and RT ⇒ Partial motor contributions ⇒ Global priors about general ability Koriat and Levy-Sadot, 2001 ⇒ …

QuesAenne et al., 2017 Gajdos et al., 2018 Boldt et al., 2017

Confidence: a common currency?

• Confidence raAngs are as accurate across two visual tasks de Gardelle et al., 2014

• Same result across a visual and an auditory task de Gardelle et al., 2016

Generalisation across different confidence formats

• Perceptual decision-making task • Domain-general overconfidence transcending temporal focus Fleming, Massoni, Gajdos and Vergnaud, 2016

Generalisation and transfer across tasks?

• Same parAcipants performed a visual, an auditory and a tacAle task • ‘Global’ confidence in an abstract, task-independent format?

• GeneralisaAon could be adapAve or harmful Faivre et al., 2018

Confidence leak

Task A: judge colour Task B: judge symbol

Confidence in one task was influenced by confidence in the other task beyond fluctuaAons in actual task performance Rahnev et al., 2015

Confidence leak

Task A: judge colour Task B: judge symbol

Confidence in one task was influenced by confidence in the other task beyond fluctuaAons in actual task performance Rahnev et al., 2015

vmPFC: a central hub for confidence signals?

Conjunction between parametric effects of desirability and confidence (desirability2) ratings

Interaction between coherence and distance

Value domain

Self-performance at ‘mini-games’ (Ame or color tasks)

Perceptual domain

Lebreton et al., 2015

Wittmann et al., 2016

Bang et al., 2018

Metacognitive ability: a trait characteristic

Metacognitive ability: a trait characteristic?

• Inter-individual variability in metacogniAve evaluaAon Fleming et al., 2010, McCurdy et al., 2013, Ais et al., 2016

Individual confidence ‘profiles’

Correlation between metaperception and metamemory

Metacognitive ability: a trait characteristic?

DistorAons in self-evaluaAon in many psychiatric disorders Silverstone and Salsali, 2003; Stephan et al., 2009; Orth and Robbins, 2013

underconfidence

overconfidence

Metacognitive ability: a trait characteristic?

N=1000 participants"

Perceptual decision-making task

Self-reported symptom questionnaires

Rouault, Seow, Gillan and Fleming (2018)"

Perceptual decision-making task QUANTIFY

+ Evidence level = dots difference

Fixation 1000 ms

- Decision process Accuracy (Drift-diffusion model)

Stimuli 300 ms

- Metacognition

Choice

Rate your confidence:

Response 500 ms

Fleming et al., 2014 Gilbert et al., 2015

certainly wrong

certainly correct

⇒ Confidence level (bias) ⇒ Metacognitive efficiency

Confidence level and metacognitive efficiency B

Metacognitive bias high

low Error Correct

omain-specific low

Metacognitive sensitivity high

low

high

low

high

Confidence rating

Fleming and Lau, 2014

Transdiagnostic dimensions cutting across standard categories Schizotypy

Trait Anxiety

Depression

easily confused if too much happens at the same time

feel like a failure

feel down-hearted and blue

hard to make decisions

feel that difficulties are piling up so that cannot be overcome

difficulty to make decisions

feels complicated to make decisions

mind not as clear as used to be

‌

not anymore enjoy things I used to do

very few things enjoyed doing feel like doing the opposite of what other people suggest even though knowing they are right? thought to be special, almost magical powers? ‌

‌

Transdiagnostic dimensions cutting across standard categories Schizotypy

Trait Anxiety

Depression

easily confused if too much happens at the same time

feel like a failure

feel down-hearted and blue

hard to make decisions

feel that difficulties are piling up so that cannot be overcome

difficulty to make decisions

feels complicated to make decisions

mind not as clear as used to be

‌

not anymore enjoy things I used to do

very few things enjoyed doing feel like doing the opposite of what other people suggest even though knowing they are right? thought to be special, almost magical powers? ‌

‌

Transdiagnostic dimensions cutting across standard categories Schizotypy

Trait Anxiety

Depression

Transdiagnostic dimensions cutting across standard categories Schizotypy

Trait Anxiety

Depression

3 latent transdiagnostic symptom dimensions:

Anxious/" Depression"

Compulsive Behavior and Intrusive Thought"

Social Withdrawal"

Metacognition and symptom dimensions

Decision

Metacognition

*** *

*** Accuracy

*

Confidence Metacognitive level sensitivity

Controlled for age, IQ, gender" *p<.05 uncorr, ***p<.001, corrected for multiple comparisons"

Metacognition and symptom dimensions

Decision

Metacognition

*** *

*** Accuracy

*

Confidence Metacognitive level sensitivity

Controlled for age, IQ, gender" *p<.05 uncorr, ***p<.001, corrected for multiple comparisons"

Self-efficacy • Belief or set of beliefs that an individual holds about his/her own capacities • Strong influence on subjective ability to exert control over one’s own motivation, behaviour and social environment • Sense of mastery • Ability to overcome challenges

Bandura, 1977; 1994

Self-esteem • Related but different from self-efficacy • Different from narcissism or overconfidence • Applies to many areas: personal, professional, social • Complicated evaluation of the benefits of high self-esteem: reciprocal influence of objective performance • Potential for metacognitive beliefs in the clinics is enormous, especially motivational disorders such as apathy Rosenberg, 1965; Wells, 2012

Conclusions • Instantaneous confidence estimates can be used to revise immediate decisions, to decide to collect more evidence, to switch environments • Local confidence can be integrated over longer time-scales to form ‘global’ self-performance estimates • Confidence can be used to control learning prospectively and to influence the learning process: multiple cues contribute to confidence • Some evidence that confidence may generalise across tasks, even more so in similar cognitive domains • Metacognitive profile could constitute an individual characteristic, transcending cognitive domains

Understanding the functional roles of metacognition => modifying beliefs about self-ability • Potential cognitive target for treatment • Modifying self-beliefs building blocks using neurofeedback:

Cortese et al., 2016

Thank you for your attention!

More references: • Fleming SM (2014) The Power of Reflection Scientific American MIND • Dunlosky & Tauber (2015) A brief history of metamemory research and handbook overview. In J. Dunlosky & S. K. Tauber (Eds) The Oxford Handbook of Metamemory • Smith JD (2009) The study of animal metacognition. Trends in Cognitive Sciences • Goupil & Kouider (2016) Behavioural and neural indices of metacognitive sensitivity in preverbal infants Current Biology • Fleming & Dolan (2014) The neural basis of metacognitive ability. In S.M. Fleming & C.D. Frith (Edts) The Cognitive Neuroscience of Metacognition • Rouault, Seow, Gillan & Fleming (2018) Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance.

Biological Psychiatry