The Vaccine Trust - The Tool

The Vaccine Trust Tool Read more: www.thevaccinetrustproject.com

Approach to survey compression The Trust Tool

2

T R U S T

T O O L

|

A P P R O A C H

We have compressed the full Vaccine Trust Survey into a feasible Vaccine Trust Tool that allows policy-makers and immunization planners to assess vaccine trust THE FULL VA C C I N E T RU S T S U RV E Y

C U R R E N T S U RV E Y

Behavioral variables

OBJECTIVE

Selecting central background and behavioral variables to measure trust against OBJECTIVE

Trust Assessment

1

OUTCOME

Two versions of the Vaccine Trust Tool

VA C C I N E T RU S T T O O L

Background Variables

OBJECTIVE

VERSION 1

An add-on Vaccine Trust Survey module

2

Identifying questions with highest explanatory power to measure Trust quadrants country- and vaccineagnostically

VERSION 2

A 20-minutes stand-alone Vaccine Trust Survey

3

T R U S T

T O O L

|

A P P R O A C H

The Vaccine Trust Tool is modular: it exists as an add-on survey module for diagnosing trust level and a stand-alone survey V1: Add-on survey module

V2: Short stand-alone survey

Objective

Assessing trust levels and vaccine uptake barriers

Assessing key barriers to rapid vaccine uptake and associated investments

Type & length

A 13-minute integrated module

A feasible ~20-minute stand-alone survey

Variation of the full trust quadrant explained by the reduced version (R2)

70-80%

59-80%. *The stand-alone survey includes fewer trusts questions to

Ability to predict behavior

Components in the tool

allow room for key background variables which reduces the R2 slightly

Similar positive and highly significant correlations with vaccination rates across HPV, COVID19 and childhood vaccines as observed with the full Vaccine Trust Survey Behavioral variables

Trust Assessment

~7 behavioral variables to assess and track effects of trust levels 18 trust assessment variables to provide picture of overall trust levels

Background variables Behavioral variables

Trust Assessment

*Length of completion is estimated based on the assumption that respondents will answer 2 questions per. Minute. For the stand-alone survey, the 20 minutes includes consent collection and an introduction to the survey.

~14 central background variables to assess where to target efforts ~8 behavioral variables to assess immediate consequences 14 trust assessment variables to provide broad picture trust barriers to rapid uptake 4

T R U S T

T O O L

|

A P P R O A C H

The selected questions hold most explanatory power within each trust quadrant, while covering the most important trust dimensions QUESTION SELECTION STATISTICAL

SELECTION

T RU S T

We use principal component analysis (PCA) to first identify the ‘latent’ components that drive trust patterns for each trust quadrant and the variance each component allows us to capture, using data on C19 from both Pakistan and Kenya to increase our data pool.

1,0 100% of variance

0,8 0,6

73% of variance

0,4 40% of variance

0,2 0,0

1

2

3

4

5

6

7

8

9

Next, we identify the questions that hold the highest factor loadings, i.e., capture most variance within the most influential components—guided by our overall objective of reducing the survey to a feasible Vaccine Trust Tool.

FRAMEWORK ALIGNEMENT

In addition to PCA, our question selection is informed by the objective of including a variety of dimensions within each quadrant.

This is done to avoid reducing a quadrant to a single dimension, while ensuring that selected questions capture a high degree of variation within the most influential components.

Factor loadings PC1

PC2

0.35

-0.44

0.27

-0.36

-0.14

-0.28

…

…

5

T R U S T

T O O L

|

A P P R O A C H

The Vaccine Trust Tool is validated by assessing the reduced quadrants’ correlations with full quadrants, as well as their ability to predict vaccination rates QUA DR ANT

VA L I DAT I O N

For validation, we compute a reduced quadrant score derived from PCA-identified questions, correlating it with observed scores across Kenya and Pakistan to assess coefficient of determination (R2)—i.e., how much of the true variation we can explain with the reduced quadrant. Vaccine-specific quadrants are validated against full quadrants for HPV.1

PREDICTIVE

PERFORMANCE

Next, we test the reduced quadrants’ ability to predict vaccination uptake. We do this primarily using HPV vaccination as target, comparing coefficients of the full quadrants against those of the reduced quadrants, and accepting significance at 5% 2

Predicted likelihood of vaccination

1

0

1) Pooling all data from Pakistan and Kenya strongly reduces overlapping datapoints across both vaccines and vaccination status. With the HPV quadrants containing more variables, this vaccine offers the most refined benchmark for validation purposes.

0

Level of trust (measured by the reduced quadrant)

100

We also test quadrants’ ability to predict vaccination uptake for C19 and childhood vaccines and observe significant results; due to the Trust Framework originally being developed in an HPVcontext, some questions will need adjustments for optimal performance for other vaccines

2) As expected, coefficients are generally smaller for reduced quadrants compared to full quadrants–however, the reduced quadrants remain significant when predicting HPV vaccination throughout 6

Approach to survey compression

The Trust Tool

7

T H E

VA C C I N E

T R U S T

T O O L

Survey questions measuring trust in the health system promise

From 9 questions to

4 questions p<0.01 69% of full quadrant explained (R2)

1

How capable* is the healthcare system of treating you or a family member with everyday health issues?

VERSION

2

Short stand-alone survey

of full quadrant explained (R2)

significance level when predicting HPV vaccination (⍺)

VERSION

5 questions p<0.01 75%

Add-on survey module

Q U E S T I O N S I N C L U D E D I N T H E VA C C I N E T RU S T T O O L

From 9 questions to

How much do you agree with the following statement: “The healthcare system is sufficiently resourced to treat critical health issues such as cancer.”

How much do you agree with the following statement: “The healthcare system prioritizes my good health” How much do you agree with the following statement: The healthcare system gives me a choice in whether or not I get a health service, such as a vaccine, test or check-up.

How much do you agree with the following statement: “The healthcare system treats me differently because of my ethnicity, religion, gender, or socioeconomic group.”

significance level when predicting HPV vaccination (⍺) 8

T H E

VA C C I N E

T R U S T

T O O L

Survey questions measuring trust in healthcare delivery

4 questions p<0.01 74%

1

Short stand-alone survey

From 18 questions to

2

significance level when predicting HPV vaccination (⍺)

Have you skipped healthcare treatment from a health providers because you worried that people in your community would learn about it, for example, by seeing you at the health facility?

VERSION

of quadrant explained (R2)

VERSION

5 questions p<0.01 78%

Add-on survey module

Q U E S T I O N S I N C L U D E D I N T H E VA C C I N E T RU S T T O O L

From 18 questions to

How much do you agree with the following statement: “Health providers have enough skills and education to treat my health issues” How much do you agree with the following statement: “My health provider cares about me and my situation” How concerned are you about the cost for health services you and your family need?

How often do you skip or postpone healthcare treatment because it is too expensive?

significance level when of quadrant explained predicting HPV (R2) vaccination (⍺) 9

T H E

VA C C I N E

T R U S T

T O O L

Survey questions measuring trust in the vaccine promise

3 questions p<0.01 59% of full quadrant explained (R2)

1

Short stand-alone survey

From 7 questions to

2

significance level when predicting HPV vaccination (⍺)

How important do you think it is for [insert vaccine target group] adults to get the [insert vaccine name] to protect your community?

VERSION

of quadrant explained (R2)

VERSION

4 questions p<0.01 75%

Add-on survey module

Q U E S T I O N S I N C L U D E D I N T H E VA C C I N E T RU S T T O O L

From 7 questions to

How important is it for [you/your children] to get [insert vaccine name] to protect [your/their] health?

How much do you agree with the following statement: [I am]/[My child is] not at risk of [insert disease name that vaccine protects against]

How much do you agree with the following statement: “I am concerned about serious side effects of the [insert vaccine name]"

significance level when predicting HPV vaccination (⍺) 10

T H E

VA C C I N E

T R U S T

T O O L

Survey questions measuring trust in vaccine delivery

3 questions p<0.01 76% of full quadrant explained (R2)

significance level when predicting HPV vaccination (⍺)

1

Short stand-alone survey

From 6 questions to

2

significance level when predicting HPV vaccination (⍺)

Were all your decisions about whether or not to accept vaccines in the past respected?

VERSION

of quadrant explained (R2)

VERSION

4 questions p<0.01 77%

Add-on survey module

Q U E S T I O N S I N C L U D E D I N T H E VA C C I N E T RU S T T O O L

From 6 questions to

E VA L UAT E G E N E R A L VA C C I N E D E L I V E RY

E VA L UAT E A S P E C I F I C VA C C I N E D E L I V E RY

How much do you agree with the following statement: In the past, the information I received about vaccines gave me all the knowledge I needed to make a decision on whether to accept it or not.

How much do you agree with the following statement: The information I received about [insert relevant vaccine name] gave me all the knowledge I needed to make a decision on whether to accept it or not.

Would you say…

Have you ever received a vaccine without being asked for your consent?

Were you asked for your consent before [you/[insert name of selected_child]] got the [insert relevant vaccine name]?

How much do you agree with the following statement? The places where me and my children have received vaccines in the past were adequately equipped* to give the vaccination.

How much do you agree with the following statement? The places where [I/[insert name of selected_child] got the [insert relevant vaccine name] was adequately equipped* to give the vaccination.

11

T H E

VA C C I N E

T R U S T

T O O L

Background & behavioral variables: The two versions of the tool consist of different background and behavioral variables

COMPONENTS

V1: Add-on survey module

V2: Short stand-alone survey

Background variables

Background variables

Will be covered by existing questions in parent survey

Behavioral variables

Childhood Childhoodvaccine vaccine status status(2 (2versions) versions)

COVID19 vaccine status

~7 behavioral variables to assess and track effects of trust levels

COVID19 vaccine offered

Vaccine status for the vaccine of interest

Vaccine of interest offered

Intention to vaccinate for the vaccine of interest

…

…

Additional questions pertaining to specific health behaviors e.g. health provider type and cervical cancer screenings can be added

Gender Gender(respondent (respondent++

Age Age (respondent (respondent++

Income

Employment (2 Employment versions)

Region

Urban/rural

Household size

Head of household

Health Healthdecisiondecisionmaker and child’s health) maker(own (own and child’s health)

Identify the caregiver

Behavioral variables

Childhood Childhoodvaccine vaccine status status(2 (2versions) versions)

COVID19 vaccine status

~8 behavioral variables to assess and track effects of trust levels

COVID19 vaccine offered

Awareness of vaccine of interest

Vaccine status for the vaccine of interest

Vaccine of interest offered

Intention to vaccinate for the vaccine of interest

…

~14 central background variables to assess where to target efforts

child) child)

child) child)

12

The versions of the tool drastically reduce the number of questions compared to the full Trust Survey—making for a feasible, yet holistic approach to measuring trust V1: Add-on survey module COMPONENTS

Background variables

From ~25 to 0 background variables

Background variables

From ~25 to 14 background variables

Behavioral variables

From ~15 to 7 behavioral variables

Behavioral variables

From ~15 to 8 behavioral variables

Trust Assessment

THE TOOL

V2: Short stand-alone survey

From 50 to 18 trust assessment variables

25-question add-on survey module

Trust Assessment

From 50 to 14 trust assessment variables

36-question stand-alone survey 13

Original strategies for the worlds of tomorrow

www.redassociates.com COPENHAGEN | Kronprinsessegade 20 | 1306 Copenhagen K | Denmark | +45 3333 7044 | cph@redassociates.com NEW YORK | 26 Broadway Suite 2505 | NY 10004 New York | USA | +1 212 483 8788 | nyc@redassociates.com PARIS | 198 rue de Rivoli | 75001 Paris France | +33 6 28 76 24 79| paris@redassociates.com