The Vaccine Trust - Methods - Survey

Vaccine Trust Survey Building the Survey and Methods Read more: www.thevaccinetrustproject.com

Methodology

2

C H A P T E R

O V E R V I E W

In the following chapter, we outline the methodological choices behind the Vaccine Trust Framework and the survey strategy S U M M A RY

CHAPTER OUTLINE

Challenges with existing trust measurements We outline challenges with existing trust measures and how the Vaccine Trust Framework seeks to improve these challenges

Operationalization

We walk through the process of operationalizing the survey, crafting the questions and outline key definitions

Testing and validating the survey tool

Sampling strategy and data processing

We lay forward how the survey tool itself was tested and validated to ensure high data quality

We present the sampling strategy and why we report on non-weighted data

The operationalization of the Vaccine Trust Framework is based on an extensive literature review and two rounds of ethnographic research in Kenya and Pakistan.

The survey was reviewed by experts and tested extensively before going to the field. The survey strategically oversamples caregivers to girls aged 10-14 across Kenya and Pakistan.

The results are unweighted.

3

M E T H O D O L O G Y

|

T R U S T

L I T E R AT U R E

Recap | Our review of the trust literature revealed four major challenges with approaches to measuring trust in a health system and vaccine context CHALLENGES

Trust measures are situational and imprecise

There is a lack of agreement on what trust is

Most trust measures are Western-centric

Trust measures are removed from decision-making

Existing trust measures are developed in an ad hoc manner and are rarely grounded in a deep, contextual understanding of trust

There is disagreement within the quantitative trust literature on what trust is, which dimensions are important, and how to measure them

Existing trust measures are developed and validated in a Western context with insufficient attention to potential LMIC specificities

Existing trust measures devote little attention to solutions and decisionsmakers’ perspectives and are rarely applied outside of academia

“More work should also be done to improve existing trust measures.

Validity of the measures could be strengthened by using qualitative methods and pilot-testing scales and indices.” 1

“People's trust in the health system plays a role in explaining one’s adherence, access to and utilization of medical care [...]. Yet it is not easy to find trust measures and understand what they are measuring.” 2

“While we found growing numbers of health systems trust measures, very few were developed and validated in lowand middle-income countries.” 3

“Improving well-being requires solid evidence that can inform policymakers and citizens where, when, and for whom life is getting better…

Nevertheless, certain topics have not yet received the attention […]Trust is one of these topics.” 4

1) Ozawa, Sachiko, and Pooja Sripad. "How do you measure trust in the health system? A systematic review of the literature."

Social science & medicine 91 (2013): 10-14.

2) *How do you measure trust in the health system? A systematic review of the literature. Ozawa & Sripad. Soc Sci Med. August 2013

3) Ozawa, Sachiko, and Pooja Sripad. "How do you measure trust in the health system? A systematic review of the literature."

Social science & medicine 91 (2013): 10-14.

4) OECD Guidelines on Measuring Trust. OECD. 2017

4

M E T H O D O L O G Y

|

T R U S T

L I T E R AT U R E

Recap | The Vaccine Trust Framework survey seek to improve each of these to deliver a data tool that is optimized for decision makers in LMIC contexts CHALLENGES

Trust measures are situational and imprecise

There is a lack of agreement on what trust is

Most trust measures are Western-centric

Trust measures are removed from decision-making

SOLUTIONS • The survey has been developed based on two rounds of detailed qualitative research

• Health- and vaccine-related trust have been operationalized meticulously

• The survey has undergone both

to capture their multiple dimensions and optimize measurement validity

ensure that it’s fit-for-purpose and that central concepts are understood • The survey integrates trust and vaccine measures to bring the two fields together

• Existing validated items from the literature have been used

expert review, cognitive testing, and pilot testing to

to the extent possible to further advance the field

• The survey has first and foremost been developed and validated for use in LMIC contexts, which have particular dynamics that aren’t captured by Westerncentric measures • The survey enables testing of hypotheses linking trust to vaccine acceptance in LMIC

contexts – and identification of relevant proxies for situations where a particular vaccine hasn’t been introduced yet

• Key decision makers both globally and nationally have been engaged from the outset

of the work to design a relevant data tool for their use cases • Data has been collected to not

only map trust nationally but also regionally, with an aim to

inform targeted approaches and vaccines to drive trust and vaccine uptake

5

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

Operationalization

Expansion

HEALTH SYSTEM

VACCINES

DIMENSION #1

SUB-DIMENSION #1  Items SUB-DIMENSION #...

HPV COVID-19

Combining the insights from the C19 and HPV work has revealed imperatives for measuring trust validly EXAMPLE

INSIGHT

Access to healthcare is more than just distance to a clinic – it’s about knowing where to go in the first place, understanding the language and not having to wait for hours

PROCESS

PROMISE

Through ethnographic fieldwork in Kenya and Pakistan, the Vaccine Trust Framework has been operationalized and expanded to an additional vaccine; the HPV vaccine

S U RV E Y I M P L I C A T I O N

Measure trust in various sub-dimensions of access Operationalization has entailed defining the dimensions of each quadrant in a data-driven way and specifying valid indicators to measures them

Expansion has entailed shifting the focus from C19 to HPV vaccination to understand its trust dynamics and qualify the Trust Framework

The questions on access includes distance to clinic, language barriers, wait time and overall difficulty of accessing healthcare and was validated during the cognitive testing and pilot test.

6

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

Systematically working from the C19 & HPV research and literature in the field enabled identification of the dimensions that constitute trust in each quadrant of the framework C19

RE SE ARCH

HPV

RESE ARCH

LIT

REVIEW

T RU S T

DIMENSIONS

7

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

The trust quadrants and dimensions proceeded by breaking dimensions into sub-dimensions and finally survey items to measure them From quadrants to dimensions and sub-dimensions…

… measured through sets of items

50

items to measure the quadrants Trust Measurement Package Documents literature review

Where possible, validated items from the literature were used to build on best practice, enable comparisons, and contribute to cumulative knowledge. However, because of the granularity of the trust framework, a range of new items also had to be created and tested.

+75

items capturing, e.g., demographics, family composition, and vaccine behaviors were also added to fulfil the secondary aims 8

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

Beyond trust items, a secondary aim was to gauge opinions on the HPV vaccine – and the questions was developed based on qualitative findings and existing HPV survey items EXAMPLE

OF

HPV

QUESTIONS

Examples of HPV specific questions

Please tell me if you do not at all agree, somewhat agree, or strongly agree: “Vaccines that are specifically made for adolescent girls are suspicious”

Please tell me if you do not at all agree, somewhat agree, or strongly agree: “I am concerned about serious side effects of the HPV vaccine” Please tell if you “Would you accept the HPV vaccine if it was recommended by… Your family? Your friends? Their teacher?

Your church? A healthcare provider?

Please tell me why you did not accept the HPV vaccine: Examples of options: “The HPV vaccine is not effective”, ”My daughter is not at risk of getting cervical cancer”, “I am against vaccines in general”

Please tell me why you did accept the HPV vaccine: Examples of options: “The HPV vaccine will protect my daughter's future”, “The HPV vaccine prevents my daughter from getting HPV”,

=

46 items related to HPV

By combining learnings from the qualitative research and existing HPV survey items we ensured that our questions captured the most important HPV dynamics…

QUA L R E S E A RC H

E X I S T I N G S U RV E Y S

Our research revealed existing mental models around the HPV vaccine, cervical cancer and barriers to uptake. These formed the foundation for which questions to ask and the answer categories e.g., on why/why not people had accepted the vaccine.

When possible, we used participants own words and descriptions to ensure the questions were understandable to the survey audience.

We reviewed existing HPV surveys and identified common categories of questions incl. fear of side effects, trust in the HPV and trust in the benefits of the vaccine to ensure we didn’t miss important categories of questions.

Most HPV surveys are developed in a European or North-American context so, we did not carry any questions directly over to the Trust Survey but changed language following a no harm principle, to ensure our survey didn’t raise participants’ concerns about the HPV vaccine.

9

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

Rather than structuring the survey strictly according to the logic of the Vaccine Trust Framework, it has been structured to have a meaningful flow for respondents The survey consists of 11 main sections SCREENING

HPV AWARENESS & VACCINATION STATUS

PERCEPTION OF THE HEALTHCARE SYSTEM

ACCESS TO HEALTHCARE SERVICES

PERCEPTION OF HEALTHCARE PROVIDERS

ATTITUDES TOWARDS HPV VACCINES

ATTITUDES TOWARDS C19 VACCINES

HEALTHSEEKING BEHAVIOR

BACKGROUND QUESTIONS AND HOUSEHOLD INFO

GENERAL TRUST QUESTIONS

157

items in total

Singlechoice

Multiplechoice

QUESTION TYPES

REDUCED FULL

Likert scales

ATTITUDES TOWARDS CHILDHOOD VACCINES

Designed for a CAPI mode of administration – taking ~45 minutes to complete

A sub-aim of this initial proof-of-concept run of the survey is to identify how best to compress it to around ~20 minutes to also be suitable for other modes of administration. Individual items/sections can also be included in other surveys.

10

M E T H O D O L O G Y

|

O P E R AT I O N A L I Z AT I O N

Key socio-demographic variables are defined based on best practice and with input from experts…

KEY DEFINITION

VA L I DAT E D S O U RC E

CAREGIVERS Adults who are responsible for the well-being of children, which could include: making financial, educational, or health-related decisions on behalf of children, being engaged in the raising, rearing, or bringing up of children, or responsible for disciplining children. The caregiver does not necessarily have to be the parent of a child.

Based on recommendation from the Ipsos team incl. LMIC survey experts

WORK Engaging in activities for which you are paid in cash or kind.

The question is followed up with the question “What work do you do?” with country specific answer categories developed with input from the project’s Gender Advisors.

Based on the World Bank’s definition of work1 and adjusted based on input from the project’s Gender Advisors 1) https://databank.worldbank.org/metadataglossary/worlddevelopment-indicators/series/SL.IND.EMPL.ZS

VULNERABILITY (KE) Vulnerability in Kenya is defined according to the Pathway Vulnerability Typing Tool.

Borrowed from the Pathway Typing Tool 11

M E T H O D O L O G Y

|

T E S T

&

VA L I D AT I O N

… and different experts reviewed and provide feedback on the survey from their respective vantage points to Institutional Review Boards Consultative group composed of experts from modeling, immunization, GE, and PHC Suggested including vulnerability items from the PATHWAYS typing tool

Advisors with both subject matter, survey design, and sampling expertise Suggested a stratified multi-stage clustered sample design to enable both national and regional analyses

The survey was reviewed by an IRB in both countries and approved by relevant authorities Suggested minor language tweaks and ensure ethical standard of the survey CROSS-INSTITUTIONAL PANEL

Gender Advisors Two gender experts – specialized in Prof. Emma Varley Rhoda Maina Pakistan and Kenya respectively – have been advising the work all throughout and applied their deep contextual understanding to the survey.

Authorities in the field of vaccine confidence and HPV vaccination from the Vaccine Confidence Project incl. Professor Heidi Larrson reviewed the survey with particular attention to comparability with existing measures.

Experts from a community of interest around measuring trust in a health context – and sociobehavioral drivers more generally – provided feedback based on their own and their institutions’ experience with measuring complex social phenomena.

EXAMPLE CHANGE

EXAMPLE CHANGE

EXAMPLE CHANGE

A set of general vaccine confidence / hesitancy questions from VCP’s work were included

Sharpen question formulation and identify key trust measures to include.

Include additional categories of employment to capture different women’s experiences.

12

M E T H O D O L O G Y

|

T E S T

&

VA L I D AT I O N

The survey went through multiple rounds of testing to optimize it in terms of relevance, comprehensibility, and feasibility GENERATIVE

TESTING

• During ethnographic fieldwork • Building the survey on contextual understanding

COGNITIVE

TESTING

• 16 in-depth cognitive tests carried out • Focusing on expected challenging parts of the survey

PILOT

TESTING

• 289 tests carried out (KE = 192; PK=97) • Testing the full survey to simulate the final experience

PURPOSE

PURPOSE

PURPOSE

Getting an initial sense of people’s ability to answer trust and vaccine-related questions by testing an alphaversion of the survey that was developed based on hypotheses from the C19 work and other existing survey instruments.

Testing the comprehensibility of specific parts of the survey to assess the reliability of responses and identify issues. An example was whether respondents understood and answered based on the definition provided of “abstract” concepts such as ‘health system’.

Simulating the experience of answering the survey in its final form with a larger group of pilot respondents to identify parts for trimming, test phrasing and translations, evaluate the structure/flow/filters, and refine response categories.

EXAMPLE IMPLICATIONS

EXAMPLE IMPLICATIONS

EXAMPLE IMPLICATIONS

• Making informed decisions about whether to refer to ‘system’, ‘authorities’, or ‘providers’ • Avoiding perceived insensitive HPV questions

• It was verified that respondents generally understood and could answer in relation to the ‘health system’ and distinguish between ‘community’ and ‘hospital’ providers • Particularly in Kenya, a 5-pt response scale posed difficulties in Swahili – therefore, a 3-pt scale was used instead for most items

• Compression of a few trust sub-dimensions – e.g., skills & education weren’t sufficiently distinct to warrant separate items • Removal of a few response categories in multiplechoice matrices to make them more manageable – e.g., when asking who people trust 13

M E T H O D O L O G Y

|

S A M P L I N G

S T R AT E G Y

&

D ATA

P R O C E S S I N G

The sampling strategy was designed to produce a nationally representative sample of caregivers of children aged 10-14, plus representative samples of select areas of interest The primary goal of the study is to achieve a nationally representative sample for caregivers of children age 1014 in Kenya and Pakistan* To obtain national representation, we employed stratified multistage cluster sampling in collaboration with Ipsos Global Stratification & sample sizing: We first stratified Kenya and Pakistan into 13 and 9 strata respectively. Sample sizes prioritized six strata of interest in Kenya, and five in Pakistan. Remaining strata were allocated sample sizes proportionate to their populations with slight reductions due to prioritized areas.

Selecting sampling units: We divided each stratum into primary sampling units, then secondary sampling units, and finally, blocks as the lowest cluster level Selection via random walks: At the final cluster level, we screened every kth household for adolescent children, collecting demographic information for each household. For each cluster, we conducted 8-9 interviews with qualifying households *) For security reasons, we excluded military areas in Pakistan and some areas in the Northeastern part of Kenya

The secondary goal is to produce representative samples of geographical areas of interest

PA K I S TA N

Lahore Karachi Islamabad

Azad Jammu and Kashmir Gilgit-Baltistan Balochistan

K E N YA

Nairobi Kisumu Kilifi

Kitui Isiolo Uasin Gishu

14

M E T H O D O L O G Y

|

S A M P L I N G

S T R AT E G Y

&

D ATA

P R O C E S S I N G

The definition of caregiving was kept deliberately inclusive to accommodate diverse family structures, with caregivers responding on behalf of a child in the household

We assumed a broad definition of caregiving to ensure the representation of caregivers beyond parents1

DEFINITION A caregiver is an adult who either: • • •

Makes financial, educational, or healthrelated decisions on behalf of children Engages in raising, rearing, or bringing up children Is responsible for disciplining children

ELIGIBLE

Each caregiver was asked to answer on behalf of either a boy or a girl 1) See the exact definition on the next slide 2) This practice was adopted once fieldwork had begun; first field guide version employed randomized selection with 70% chance of selection allocated to girls

SELECTED

If both a boy and a girl between the ages of 10-14 were part of the household, caregivers were asked to answer on behalf of the girl2 15

M E T H O D O L O G Y

|

S A M P L I N G

S T R AT E G Y

&

D ATA

P R O C E S S I N G

As a result, the Vaccine Trust Survey offers insight into an understudied yet crucial population— caregivers to adolescents—who are key to understand vaccine behavior W H AT W E H AV E D O N E …

W H AT O T H E R S H AV E D O N E …

Populations  

Male caregivers to adolescents Female caregivers to adolescents

Vaccine behaviors     

COVID-19 vaccine status HPV vaccine status Childhood vaccine status Zero dose status HPV vaccine awareness

Trust patterns      

Trust in the health system promise Trust in the healthcare delivery Trust in vaccine promises Trust in vaccine delivery Interpersonal trust Institutional trust

FinAccess Household survey

 Female caregiver  Male caregivers × No trust data × No vaccine data Kenya 2021 & Pakistan 2018 (Wave 7)

Kenya 2021

The Social & Living Standards Measurement Survey Pakistan 2018

Kenya 2021 & Pakistan 2018

× Only childhood vaccine status children under 5yo × No HPV vaccine status × No HPV awareness × No trust data

× Only childhood vaccine status children under 3yr × No link between household data & vaccine data × No trust data

Global Monitor: Kenya 2018 & Pakistan 2018

Kenya 2022 (Round 8)

 Institutional trust & interpersonal trust  Trust in healthcare system × No vaccine data

 Institutional trust & interpersonal trust  Trust in healthcare system × No caregiver data × No vaccine data

 Institutional trust & interpersonal trust  Trust in healthcare system × No caregiver data × No vaccine data 16

M E T H O D O L O G Y

|

S A M P L I N G

S T R AT E G Y

&

D ATA

P R O C E S S I N G

All the results are calculated based on unweighted data to ensure transparency – and comparing weighted and unweighted results does not revealed significant differences C L E A N I N G DATA

W E I G T I N G T H E DATA

The final data set has been cleaned including the following steps:

This report presents the unweighted data to comply with best practices and ensure transparency around our results

• The data has been anonymized • Outliers have been removed based on income in Kenya • Variables have been recoded to have the same direction

There is very little difference in the trust scores in both Kenya and Pakistan when comparing the weighted and unweighted* results – suggesting a limited benefit of weighting the data PAK I STAN

”Allowing readers to understand who was interviewed and who was excluded, and to contextualize results, accordingly, may often be far more informative than any statistical adjustment done "behind the scenes".”

- Collins et al. 20221

Unweighted

K E N YA

Weighted Unweighted

Weighted

Health system promise

64

64

70

71

Healthcare delivery

61

61

62

63

Vaccine promise

66

68

74

72

Vaccine delivery

71

72

70

70

*Weights are calculated based on income. The weights are available upon requests.

1) Collin et al. 2022 “Representativeness of remote methods in LMICs: A cross-national analysis of pandemic-era studies

17

Quantifying the VaccineTrust Framework 18

C H A P T E R

O V E R V I E W

In the following chapter, we build and test the validity of the Vaccine Trust Framework and its constitutive parts CHAPTER OUTLINE

Approach to building the Vaccine Trust Framework

We outline how the Trust Framework is quantified bottom-up.

Kenya and Pakistan’s Vaccine Trust Scores

Imputing HPV vaccine trust in Pakistan

Testing the validity of the Vaccine Trust Framework

We present the trust scores for Kenya and Pakistan.

We impute HPV vaccine trust in Pakistan with relevant proxies.

We test the internal and external validity of the Vaccine Trust Framework.

19

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

The Vaccine Trust Framework assumes that each individual has their own trust profile and scores – aggregating the individual trust scores yields the trust score T RU S T S C O R E S RESPONDENT 1

T RU S T S C O R E S RESPONDENT 2

T RU S T S C O R E S RESPONDENT 3

71

74

68

65

75

68

67

70

73

51

72

54

K E N YA’ S

T RU S T

70

SCOR

74

69 67

64

70

72

The trust score is reported on a scale from 0 – 100, with 0 indicating low trust, and 100 indicating high trust*. * Statistically the trust score is normalized to a scale on 0 – 1, before being multiplying with 100 20

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

To calculate the trust score, the Vaccine Trust Framework is quantified bottom-up progressing from dimensions, quadrants, levels, and ultimately, the full framework

From indicators to dimensions

From dimensions to quadrants

From quadrants to levels

Each dimension consists of a number of indicators, i.e., questions included in the survey

The Framework’s trust quadrants aggregate the scores of the trust dimensions for the quadrant in question

The level scores for the health system and vaccines respectively aggregate the quadrants that connect vertically

The full Trust Framework

21

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

Out of the 150 questions included in the survey, 50 questions serve as indicators for the trust scores that aggregate into a trust score at the dimension level S U RV E Y Q U E S T I O N S

Please tell me if you do not at all agree, somewhat agree, or strongly agree: “The healthcare system prioritizes my good health” Please tell me if you do not at all agree, somewhat agree, or strongly agree: “The healthcare system prioritizes the health issues that are most important for me and my community”

PRIORITY ALIGNMENT

0 Low trust

77

100 High trust

The trust score is reported on a scale from 0 – 100, with 0 indicating low trust, and 100 indicating high trust*.

By averaging the values, we obtain a unified index for each trust dimension

Each dimension consists of 1-9 indicators * Statistically the trust score is normalized to a scale on 0 – 1, before being multiplying with 100 22

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

The scores on dimensions are aggregated into the trust score on a quadrant level, again by averaging the individual’s trust score

64

PRIORITY ALIGNMENT

77

T RU S T S C O R E AT T H E Q UA D R A N T L E V E L

By calculating the average of the four dimensions that make up the quadrant for each individual and aggregating them, we obtain the trust score at the quadrant level —in this case, for trust in the health system promise

AUTONOMY

CAPA BI LI T Y FAIRNESS

77

49

51

23

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

Averaging the two quadrant scores for the health system or the vaccine produces the trust score at either the health system or vaccine level 77 49

64 T RU S T

SCORE

51

77 H E A LT H

S YS T E M

H E A LT H C A R E

60 T RU S T

P RO M I S E

D E L I V E RY

T RU S T S C O R E AT T H E H E A LT H S YS T E M L E V E L

71 SCORE

73

62

51

47 56

In this case, we have calculated the trust score at a health system level by averaging each individual’s trust scores at the quadrant level

24

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

A P P R O A C H

92 By combining the two levels, we obtain the full Vaccine Trust Framework and the trust scores at the vaccine and system 77level

64 QUA DR ANT LEVEL

QUA DR ANT LEVEL

53

S YS T E M

H E A LT H C A R E

P RO M I S E

VA C C I N E

D E L I V E RY

VA C C I N E

71

QUADRANT SCORE

62

69

51

77 H E A LT H

60

63

49

HEALTH SYSTEM LEVEL

70 73

51

47 56

61

80

P RO M I S E D E L I V E RY

70

QUA DR ANT LEVEL

70

VA C C I N E L E V E L 25

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

T R U S T

S C O R E S

K E N YA

gn ali

omy

es riti

Be nef it

io Pr

n Auto

This approach produces the following framework and trust scores for Kenya, with the vaccine level for the HPV vaccine n me t

Ca pa

70 S Y S T E M

P RO M I S E

D E L I V E RY lity dentia Confi

on De liv er y

bility

ss

Afford a

HEALTH SYSTEM LEVEL

ssi

Ac ce

a mp Co

ty

se t

tin

g

74

VA C C I N E

P RO M I S E

VA C C I N E

D E L I V E RY

Adeq u

cy en Ag

66

s

en ce

62

e Saf

pe t

H E A LT H C A R E

nes

e nc

bil ity

Co m

H E A LT H

Fair

va le Re

acy o f info

70

72

VA C C I N E L E V E L 26

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

T R U S T

S C O R E S

K E N YA

n Auto

io Pr gn ali

omy

es riti

Be nef it

…And a total trust score of 69, compounding the health system and vaccine level

n me t

S Y S T E M

lity dentia Confi

HEALTH SYSTEM LEVEL

ssi

on

e Saf

ss

ty

eli ve ry

se t

tin

g

74

VA C C I N E

P RO M I S E

VA C C I N E

D E L I V E RY

Adeq u

G E N E R A L T R U S T S C O R ED

Ac ce

a mp Co

cy en Ag

66

69

D E L I V E RY

en ce

62

P RO M I S E

pe t

H E A LT H C A R E

s

Co m

H E A LT H

nes

bility

70

Fair

e nc

bil ity

Afford a

Ca pa

va le Re

acy o f info

70

72

VA C C I N E L E V E L 27

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

T R U S T

S C O R E S

PA K I S TA N

gn ali

o my

es riti

Be nef it

io Pr

n Auto

In Pakistan, the HPV vaccine remains to be rolled out—which, at first, leaves us with the trust score at the health system level alone n me t

ce an ev l Re

Ca pa

64 S Y S T E M

ty

P RO M I S E

D E L I V E RY lity dentia Confi

De liv er y

se t

tin g

ability Afford

ncy

HEALTH SYSTEM LEVEL

acy o f info

i on ss

Ac ces s

a mp Co

Adeq u

e Ag

62

s

ce

61

e Saf

en

H E A LT H C A R E

nes

Co m pe t

H E A LT H

Fair

bi l ity

28

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I M P U T I N G

H P V

R E S P O N S E

However, we have observed that the HPV vaccine occupies a space between childhood and adult vaccines in fieldwork—and we see a similar pattern in the Kenyan survey data During fieldwork, we observed how the HPV vaccine occupied its own space in respondents’ mental models around vaccines…

This informed the hypothesis that trust in the HPV vaccine is neither entirely correlated with C19, nor childhood vaccines 1 – trust in HPV vac.

1 - trust in HPV vac.

1 – Trust in childhood vac.

βC19 0.24 βchildhood 0.68

Fieldwork in this project’s earlier phase suggested that the HPV vaccine targeting adolescent girls carries potential adult connotations while still being a vaccination decision made by caregivers on behalf of a child

βC19 0.55

0 Trust in Childhood vac. - 1

0

P-value <0.0001

P-value <0.0001

Trust in C19 vac. - 1

0

Trust in C19 vac. - 1

P-value <0.0001

Both childhood and C19 vaccine items help explain unique variation in the response to the HPV vaccine in Kenya and remain significant when included in the same model —supporting the interpretation that both vaccine items can be used to impute HPV responses 29

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I M P U T I N G

H P V

R E S P O N S E

With this in mind, we can impute the response to the HPV vaccine by averaging trust levels in COVID-19 and childhood vaccines B E N E F I T

T E S T I N G M E T H O D I N K E N YA

R E L E VA N C E

When averaging trust in childhood and C19 vaccines in Kenya to impute trust in the HPV vaccine, we find that each imputed dimension varies by a maximum of +/- 10 in trust score compared to observed scores

S A F E T Y

C AV E AT S

Pulling the average assumes that the context observed in Kenya—HPV occupying its own space due to its target group being adolescents—is also applicable to Pakistan.

FINDING

A D E QUAC Y O F I N F O

D E L I V E RY S E T T I N G S AG E N C Y

Calculating the average trust in childhood and C19 vaccines provides a relatively accurate picture of the observed HPV response in Kenya

Observed HPV response

An estimate based on childhood vaccines and C19 unifies two types of vaccines with diverse timelines, presenting a challenge in pinpointing when the imputed trust levels for the HPV vaccine would materialize.

In particular trust in childhood vaccines appears to contribute to a slight overestimation of trust in the HPV vaccine.

Imputed HPV response 30

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

T R U S T

S C O R E S

PA K I S TA N

fit Be ne

en nm lig sa

o my

itie ior Pr

n Auto

This method provides us with the following framework and trust scores for Pakistan, including imputed trust scores for the vaccine level t

ce an ev l Re

Ca pa

64 S Y S T E M

lity dentia Confi

De liv er y

bility Afford a

HEALTH SYSTEM LEVEL

n sio

ss

as mp Co

ty

se t

tin g

66

VA C C I N E

P RO M I S E

VA C C I N E

D E L I V E RY

Adeq uacy of

cy en Ag

63

D E L I V E RY

Ac ce

61

P RO M I S E

nc e

H E A LT H C A R E

e Saf

Fair nes s

Co m pe te

H E A LT H

bil ity

info

71

68

VA C C I N E L E V E L 31

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

T R U S T

S C O R E S

PA K I S TA N

o my

en nm lig sa

Be ne

n Auto

itie ior Pr

fit

…And a compound trust score of 66 for Pakistan

t

ce an ev l Re

S Y S T E M

HEALTH SYSTEM LEVEL

n sio

e Saf

ty

el ive ry

se t

tin g

66

VA C C I N E

P RO M I S E

VA C C I N E

D E L I V E RY

Adeq uacy of

G E N E R A L T R U S T S C O R ED ss

as mp Co

cy en Ag

63

66

lity dentia Confi

Ac ce

61

D E L I V E RY

nc e

H E A LT H C A R E

P RO M I S E

Co m pe te

H E A LT H

Fair nes s

bility

64

bil ity

Afford a

Ca pa

info

71

68

VA C C I N E L E V E L 32

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

The Trust Framework’s components are interrelated, but also all uniquely necessary to conceptualize trust—striking a balance between consistency and distinctiveness

FUL LY DI STI NC T

F U L LY C O N S I S T E N T

Producing fully consistent quadrants and dimensions, i.e., obtaining perfect correlation, would indicate that the Framework’s different components are obsolete

CRITERIA 1

The Trust Framework

Within a level & quadrant

Striking a balance between consistency and distinctiveness within a quadrant, i.e., ensuring that dimensions contribute to measuring a specific type of trust

CRITERIA 2

Producing fully distinct quadrants and dimensions, i.e., obtaining zero correlation, would indicate that the Framework’s components do not measure the same phenomena

Between quadrants

Striking a balance between consistency and distinctiveness between quadrants, i.e., ensuring that all four kinds of trust are in fact related and needed to conceptualize trust as a whole

33

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

|

W I T H I N

L E V E L S

Within levels: Dimensions are more closely connected within each level on average, suggesting that the Framework’s health system and vaccine levels are statistically consistent

HEA LT H S YSTE M

HEA LTH SYS TEM LE VEL

Autonomy Priority alignment Capability Fairness Confidentiality Compassion Competence Access Affordability

VA C C I N E

Benefit Relevance Safety Adequacy of info Delivery setting Agency

VA C C I N E L E V E L

Autonomy

Priority alignment

Capability

Fairness

Confidentiality

Compassion

Competence

Access

Affordability

Benefit

Relevance

Safety

NA

0.21

0.07

0.10

0.24

0.17

0.08

0.19

0.09

0.11

0.10

0.09

0.16

0.16

0.06

0.21

NA

0.26

0.04

0.17

0.29

0.11

0.17

0.09

0.14

0.11

0.09

0.19

0.22

0.12

0.07

0.26

NA

-0.05

0.11

0.24

0.17

0.18

0.15

0.08

0.06

0.12

0.15

0.16

0.14

0.16

NA

0.12

0.07

0.05

0.12

0.16

0.24

0.11

0.13

0.12

0.31

0.23

0.10

0.15

0.11

0.08

0.13 0.04 Average correlation NA within the health 0.37 0.04 0.37 NA system level

0.06

0.13

Average correlation between 0.04 0.09 0.04 0.04 vaccine0.15 dimensions and 0.06 0.08 0.17 dimensions 0.15system 0.18 0.23 0.07

0.06

0.16

0.31

NA

0.16

0.14

0.07

0.07

0.13

0.15

0.11

0.11

0.24

0.23

0.16

NA

0.33

0.10

0.08

0.09

0.11

0.09

0.05

0.10

0.10

Adequacy of info Delivery setting

Agency

0.04

-0.05

0.17

0.11

0.29

0.24

0.11

0.17

0.19

0.17

0.18

0.12

0.09

0.09

0.15

0.07

0.11

0.10

0.14

0.33

NA

-0.01

0.00

0.09

0.07

0.03

0.05

0.11

0.14

0.08

0.05

0.13

0.15

0.07

0.10

-0.01

NA

0.43

0.22

0.29

0.40

0.13

0.10

0.11

0.06

0.12

0.12

0.11

0.07

0.08

0.00

0.43

NA

0.27

0.31

0.29

0.07

0.09

0.10

0.09

0.09

0.22

0.27

0.24 0.17

0.16 0.16 0.06

0.09

0.12

0.19

0.15

0.22

0.16

Average correlation between 0.13 0.08 0.15 vaccine dimensions and 0.09 0.15 0.18 0.15 system dimensions 0.04 0.17 0.23 0.11

0.12

0.14

0.04

0.04

0.06

0.07

0.11

0.29

0.11

0.07

0.29

0.31

0.09

0.03

0.40

0.29

0.05

0.05

0.13

0.07

0.08 Average correlation 0.28 0.41 withinNAthe vaccine level 0.24 NA

0.28

0.29

0.29

0.41

NA

0.33

0.08

0.24

0.33

NA 34

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

|

W I T H I N

Q U A D R A N T S

Within quadrants: Using the Vaccine Trust Framework to allocate dimensions, we improve internal correlation in three out of four quadrants compared to distribution at random P RO C E D U R E

RESULTS Avg. correlation (at random)

While we have qualitative reason to group dimensions as per the Vaccine Trust Framework, we also tested internal consistency within quadrants by comparing internal quadrant correlation with the correlation coefficient had the dimensions been distributed at random.

Avg. correlation within true quadrant

Change (%)

Health system promise

0.15

0.10

-29%

Healthcare delivery

0.15

0.22

48%

Vaccine promise

0.15

0.31

111%

Vaccine delivery

0.15

0.32

122%

Health system promise is the only quadrant that doesn’t improve its internal correlation when compared with the random distribution. This is expected as the quadrant is the most abstract and difficult to measure, compounding large concepts such as fairness and autonomy.

35

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

|

W I T H I N

Q U A D R A N T S

Within quadrants: Health system promise has relatively low internal consistency due to two dimensions showing negative correlation—a challenge the Vaccine Trust Tool will address Two dimensions in Health system promise, Capability and Fairness, are negatively correlated, resulting in a reduced correlation score for this quadrant Autonomy

Priority alignment

Capability

Fairness

The Trust Tool aims to shed light on this dynamic through re-operationalization of Fairness The observed negative correlation could indicate a scenario in which some respondents view the health system as capable of treating illnesses, but also experience that they are to some extent excluded from treatment because of socio-demographic markers such as ethnicity, gender, and geography.

Autonomy

1

0.21

0.07

0.10

Priority alignment

0.21

1

0.26

0.04

Capability

0.07

0.26

1

-0.05

Notably, Fairness represents a complex dimension that by definition could negatively correlate with other dimensions e.g. capability as people might evaluate the health system positively for others than themselves.

Fairness

0.10

0.04

-0.05

1

To examine this, the Vaccine Trust Tool will revisit the operationalization of dimensions to clarify this dynamic.

CURRENT C A P A B I L I T Y

VERSIONS F A I R N E S S

Being treated differently in the health system because of …

REVISED

VERSION

F A I R N E S S

Being treated more poorly in the health system because of…

36 PROPRIETARY AND CONFIDENTIAL | 36

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

|

B E T W E E N

Q U A D R A N T S

Between quadrants: We find that the quadrants hold unique variation when fitted in the same regression model—indicating that they are related, but also distinct RESULTS

P RO C E D U R E

System

Independent variable II

Vaccine

Independent variable I

All quadrants hold unique variation

Dependent variable

Independent variable III

To test relatedness and distinctiveness between quadrants, we fit four regression models with one quadrant as the ‘dependent’ variable and the remaining three quadrants as independent variables, assessing whether they all hold unique variation when predicting the last quadrant

All quadrants are highly significant (p-values < 0.0001) in all models. As expected, same-level quadrants produce higher coefficients, and opposite-level quadrants slightly lower coefficients.

OLS Dep: Health System Promise

RESULTS*

C O E F.

Dep: Vaccine promise

C O E F.

Healthcare delivery

0.42

Health system promise

0.09

Vaccine promise

0.06

Healthcare delivery

0.14

Vaccine delivery

0.1

Vaccine delivery

0.29

Dep: Healthcare delivery

C O E F.

Dep: Vaccine delivery

C O E F.

Health system promise

0.27

Health system promise

0.24

Vaccine promise

0.05

Healthcare delivery

0.19

Vaccine delivery

0.05

Vaccine promise

0.43

*) Intercepts omitted for simplicity

37

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

I N T E R N A L

VA L I D AT I O N

|

M U LT I C O L L I N E A R I T Y

Our data shows multicollinearity between the four trust quadrants—but results remain statistically significant throughout when predicting vaccination behavior The Vaccine Trust Framework conceptualizes trust as four related trust types, making multicollinearity a premise for analysis The trust quadrants display low to moderate pairwise correlation—however, Variance Inflation Factors (VIF) are high (+10), suggesting that the combination of trust quadrants drive multicollinearity, and in turn, increases statistical uncertainty of results PA I RW I S E C O R R E L AT I O N * Health promise

Healthcare delivery

Vaccine promise

VIF ANALYSIS* Vaccine delivery

Health promise

20

Health promise

1

0.4

0.2

0.26

Healthcare delivery

21

Healthcare delivery

0.4

1

0.19

0.22

Vaccine promise

14

Vaccine promise

0.2

0.19

1

0.4

Vaccine delivery

11

Vaccine delivery

0.26

0.22

0.4

1

*) Calculated based on results from Kenya on the HPV vaccine

Despite multicollinearity, we observe significant results across analyses of the four trust quadrants This indicates that although results are subject to higher statistical uncertainty, trust comes out as a strong predictor of vaccine uptake.

38

Q U A N T I F Y I N G

T H E

F R A M E W O R K

|

E X T E R N A L

VA L I D AT I O N

External validation: Compared to other widely used trust measures, the Vaccine Trust Tool turns out to add signficantly more explanation to variations in vaccine uptake Interpersonal trust measures are not helpful in explaining variation in vaccine uptake, producing statistically insignificant results.

For the Vaccine Trust Tool, the correlations are stronger and highly significant indicating a strong relationship between trust and vaccine uptake.

Logistical regression correlations between vaccine uptake and different trust measures reported in log odds

C19 vaccine uptake

Interpersonal trust2

NOT

0.13

INTERPERSONAL TRUST

Interpersonal trust is a widely used and accepted measure of trust in the trust literature – often applied in studies on health-seeking behaviors and vaccine uptake. The question: “Generally speaking, would you say that most people can be trusted or that you need to be very careful in dealing with people?” stems from the World Value Survey1.

1) World Value Survey Wave 7 (2017-2022). Documentation: The questionnaire. https://www.worldvaluessurvey.org/WVSDocumentationWV7.jsp

Childhood vaccine uptake

HPV vaccine uptake

SIGNIFICANT

0.19

-0.17

SIGNIFICANT

Vaccine Trust Tool3

2.58***

1.42**

3.68***

Correlations are controlled for respondent gender, age, educational attainment, language, religion and province. **p<0.01; ***p<0.001 2) Measured by the binary question “Generally speaking, would you say that most people can be trusted or that you must be very careful in dealing with people? ” 3) Measured as a combined measure of the four trust quadrants in the Vaccine Trust Framework: Trust in the health system promise, the healthcare delivery, the vaccine promise, and the vaccine delivery.

39