Vaccine Trust Survey Building the Survey and Methods Read more: www.thevaccinetrustproject.com
Methodology
2
C H A P T E R
O V E R V I E W
In the following chapter, we outline the methodological choices behind the Vaccine Trust Framework and the survey strategy S U M M A RY
CHAPTER OUTLINE
Challenges with existing trust measurements We outline challenges with existing trust measures and how the Vaccine Trust Framework seeks to improve these challenges
Operationalization
We walk through the process of operationalizing the survey, crafting the questions and outline key definitions
Testing and validating the survey tool
Sampling strategy and data processing
We lay forward how the survey tool itself was tested and validated to ensure high data quality
We present the sampling strategy and why we report on non-weighted data
The operationalization of the Vaccine Trust Framework is based on an extensive literature review and two rounds of ethnographic research in Kenya and Pakistan. The survey was reviewed by experts and tested extensively before going to the field. The survey strategically oversamples caregivers to girls aged 10-14 across Kenya and Pakistan. The results are unweighted. 3
M E T H O D O L O G Y
|
T R U S T
L I T E R AT U R E
Recap | Our review of the trust literature revealed four major challenges with approaches to measuring trust in a health system and vaccine context CHALLENGES
Trust measures are situational and imprecise
There is a lack of agreement on what trust is
Most trust measures are Western-centric
Trust measures are removed from decision-making
Existing trust measures are developed in an ad hoc manner and are rarely grounded in a deep, contextual understanding of trust
There is disagreement within the quantitative trust literature on what trust is, which dimensions are important, and how to measure them
Existing trust measures are developed and validated in a Western context with insufficient attention to potential LMIC specificities
Existing trust measures devote little attention to solutions and decisionsmakers’ perspectives and are rarely applied outside of academia
“More work should also be done to improve existing trust measures. Validity of the measures could be strengthened by using qualitative methods and pilot-testing scales and indices.” 1
“People's trust in the health system plays a role in explaining one’s adherence, access to and utilization of medical care [...]. Yet it is not easy to find trust measures and understand what they are measuring.” 2
“While we found growing numbers of health systems trust measures, very few were developed and validated in lowand middle-income countries.” 3
“Improving well-being requires solid evidence that can inform policymakers and citizens where, when, and for whom life is getting better… Nevertheless, certain topics have not yet received the attention […]Trust is one of these topics.” 4
1) Ozawa, Sachiko, and Pooja Sripad. "How do you measure trust in the health system? A systematic review of the literature." Social science & medicine 91 (2013): 10-14. 2) *How do you measure trust in the health system? A systematic review of the literature. Ozawa & Sripad. Soc Sci Med. August 2013
3) Ozawa, Sachiko, and Pooja Sripad. "How do you measure trust in the health system? A systematic review of the literature." Social science & medicine 91 (2013): 10-14. 4) OECD Guidelines on Measuring Trust. OECD. 2017
4
M E T H O D O L O G Y
|
T R U S T
L I T E R AT U R E
Recap | The Vaccine Trust Framework survey seek to improve each of these to deliver a data tool that is optimized for decision makers in LMIC contexts CHALLENGES
Trust measures are situational and imprecise
There is a lack of agreement on what trust is
Most trust measures are Western-centric
Trust measures are removed from decision-making
SOLUTIONS • The survey has been developed based on two rounds of detailed qualitative research
• Health- and vaccine-related trust have been operationalized meticulously
• The survey has undergone both
to capture their multiple dimensions and optimize measurement validity
ensure that it’s fit-for-purpose and that central concepts are understood • The survey integrates trust and vaccine measures to bring the two fields together
• Existing validated items from the literature have been used
expert review, cognitive testing, and pilot testing to
to the extent possible to further advance the field
• The survey has first and foremost been developed and validated for use in LMIC contexts, which have particular dynamics that aren’t captured by Westerncentric measures • The survey enables testing of hypotheses linking trust to vaccine acceptance in LMIC
contexts – and identification of relevant proxies for situations where a particular vaccine hasn’t been introduced yet
• Key decision makers both globally and nationally have been engaged from the outset
of the work to design a relevant data tool for their use cases • Data has been collected to not
only map trust nationally but also regionally, with an aim to
inform targeted approaches and vaccines to drive trust and vaccine uptake
5
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
Operationalization
Expansion
HEALTH SYSTEM
VACCINES
DIMENSION #1
SUB-DIMENSION #1 Items SUB-DIMENSION #...
HPV COVID-19
Combining the insights from the C19 and HPV work has revealed imperatives for measuring trust validly EXAMPLE
INSIGHT
Access to healthcare is more than just distance to a clinic – it’s about knowing where to go in the first place, understanding the language and not having to wait for hours
PROCESS
PROMISE
Through ethnographic fieldwork in Kenya and Pakistan, the Vaccine Trust Framework has been operationalized and expanded to an additional vaccine; the HPV vaccine
S U RV E Y I M P L I C A T I O N
Measure trust in various sub-dimensions of access Operationalization has entailed defining the dimensions of each quadrant in a data-driven way and specifying valid indicators to measures them
Expansion has entailed shifting the focus from C19 to HPV vaccination to understand its trust dynamics and qualify the Trust Framework
The questions on access includes distance to clinic, language barriers, wait time and overall difficulty of accessing healthcare and was validated during the cognitive testing and pilot test.
6
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
Systematically working from the C19 & HPV research and literature in the field enabled identification of the dimensions that constitute trust in each quadrant of the framework C19
RE SE ARCH
HPV
RESE ARCH
LIT
REVIEW
T RU S T
DIMENSIONS
7
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
The trust quadrants and dimensions proceeded by breaking dimensions into sub-dimensions and finally survey items to measure them From quadrants to dimensions and sub-dimensions…
… measured through sets of items
50
items to measure the quadrants Trust Measurement Package Documents literature review
Where possible, validated items from the literature were used to build on best practice, enable comparisons, and contribute to cumulative knowledge. However, because of the granularity of the trust framework, a range of new items also had to be created and tested.
+75
items capturing, e.g., demographics, family composition, and vaccine behaviors were also added to fulfil the secondary aims 8
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
Beyond trust items, a secondary aim was to gauge opinions on the HPV vaccine – and the questions was developed based on qualitative findings and existing HPV survey items EXAMPLE
OF
HPV
QUESTIONS
Examples of HPV specific questions
Please tell me if you do not at all agree, somewhat agree, or strongly agree: “Vaccines that are specifically made for adolescent girls are suspicious”
Please tell me if you do not at all agree, somewhat agree, or strongly agree: “I am concerned about serious side effects of the HPV vaccine” Please tell if you “Would you accept the HPV vaccine if it was recommended by… Your family? Your friends? Their teacher? Your church? A healthcare provider? Please tell me why you did not accept the HPV vaccine: Examples of options: “The HPV vaccine is not effective”, ”My daughter is not at risk of getting cervical cancer”, “I am against vaccines in general”
Please tell me why you did accept the HPV vaccine: Examples of options: “The HPV vaccine will protect my daughter's future”, “The HPV vaccine prevents my daughter from getting HPV”,
=
46 items related to HPV
By combining learnings from the qualitative research and existing HPV survey items we ensured that our questions captured the most important HPV dynamics… QUA L R E S E A RC H
E X I S T I N G S U RV E Y S
Our research revealed existing mental models around the HPV vaccine, cervical cancer and barriers to uptake. These formed the foundation for which questions to ask and the answer categories e.g., on why/why not people had accepted the vaccine. When possible, we used participants own words and descriptions to ensure the questions were understandable to the survey audience.
We reviewed existing HPV surveys and identified common categories of questions incl. fear of side effects, trust in the HPV and trust in the benefits of the vaccine to ensure we didn’t miss important categories of questions. Most HPV surveys are developed in a European or North-American context so, we did not carry any questions directly over to the Trust Survey but changed language following a no harm principle, to ensure our survey didn’t raise participants’ concerns about the HPV vaccine. 9
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
Rather than structuring the survey strictly according to the logic of the Vaccine Trust Framework, it has been structured to have a meaningful flow for respondents The survey consists of 11 main sections SCREENING
HPV AWARENESS & VACCINATION STATUS
PERCEPTION OF THE HEALTHCARE SYSTEM
ACCESS TO HEALTHCARE SERVICES
PERCEPTION OF HEALTHCARE PROVIDERS
ATTITUDES TOWARDS HPV VACCINES
ATTITUDES TOWARDS C19 VACCINES
HEALTHSEEKING BEHAVIOR
BACKGROUND QUESTIONS AND HOUSEHOLD INFO
GENERAL TRUST QUESTIONS
157
items in total
Singlechoice
Multiplechoice
QUESTION TYPES
REDUCED FULL
Likert scales
ATTITUDES TOWARDS CHILDHOOD VACCINES
Designed for a CAPI mode of administration – taking ~45 minutes to complete
A sub-aim of this initial proof-of-concept run of the survey is to identify how best to compress it to around ~20 minutes to also be suitable for other modes of administration. Individual items/sections can also be included in other surveys.
10
M E T H O D O L O G Y
|
O P E R AT I O N A L I Z AT I O N
Key socio-demographic variables are defined based on best practice and with input from experts…
KEY DEFINITION
VA L I DAT E D S O U RC E
CAREGIVERS Adults who are responsible for the well-being of children, which could include: making financial, educational, or health-related decisions on behalf of children, being engaged in the raising, rearing, or bringing up of children, or responsible for disciplining children. The caregiver does not necessarily have to be the parent of a child.
Based on recommendation from the Ipsos team incl. LMIC survey experts
WORK Engaging in activities for which you are paid in cash or kind. The question is followed up with the question “What work do you do?” with country specific answer categories developed with input from the project’s Gender Advisors.
Based on the World Bank’s definition of work1 and adjusted based on input from the project’s Gender Advisors 1) https://databank.worldbank.org/metadataglossary/worlddevelopment-indicators/series/SL.IND.EMPL.ZS
VULNERABILITY (KE) Vulnerability in Kenya is defined according to the Pathway Vulnerability Typing Tool.
Borrowed from the Pathway Typing Tool 11
M E T H O D O L O G Y
|
T E S T
&
VA L I D AT I O N
… and different experts reviewed and provide feedback on the survey from their respective vantage points to Institutional Review Boards Consultative group composed of experts from modeling, immunization, GE, and PHC Suggested including vulnerability items from the PATHWAYS typing tool
Advisors with both subject matter, survey design, and sampling expertise Suggested a stratified multi-stage clustered sample design to enable both national and regional analyses
The survey was reviewed by an IRB in both countries and approved by relevant authorities Suggested minor language tweaks and ensure ethical standard of the survey CROSS-INSTITUTIONAL PANEL
Gender Advisors Two gender experts – specialized in Prof. Emma Varley Rhoda Maina Pakistan and Kenya respectively – have been advising the work all throughout and applied their deep contextual understanding to the survey.
Authorities in the field of vaccine confidence and HPV vaccination from the Vaccine Confidence Project incl. Professor Heidi Larrson reviewed the survey with particular attention to comparability with existing measures.
Experts from a community of interest around measuring trust in a health context – and sociobehavioral drivers more generally – provided feedback based on their own and their institutions’ experience with measuring complex social phenomena.
EXAMPLE CHANGE
EXAMPLE CHANGE
EXAMPLE CHANGE
A set of general vaccine confidence / hesitancy questions from VCP’s work were included
Sharpen question formulation and identify key trust measures to include.
Include additional categories of employment to capture different women’s experiences.
12
M E T H O D O L O G Y
|
T E S T
&
VA L I D AT I O N
The survey went through multiple rounds of testing to optimize it in terms of relevance, comprehensibility, and feasibility GENERATIVE
TESTING
• During ethnographic fieldwork • Building the survey on contextual understanding
COGNITIVE
TESTING
• 16 in-depth cognitive tests carried out • Focusing on expected challenging parts of the survey
PILOT
TESTING
• 289 tests carried out (KE = 192; PK=97) • Testing the full survey to simulate the final experience
PURPOSE
PURPOSE
PURPOSE
Getting an initial sense of people’s ability to answer trust and vaccine-related questions by testing an alphaversion of the survey that was developed based on hypotheses from the C19 work and other existing survey instruments.
Testing the comprehensibility of specific parts of the survey to assess the reliability of responses and identify issues. An example was whether respondents understood and answered based on the definition provided of “abstract” concepts such as ‘health system’.
Simulating the experience of answering the survey in its final form with a larger group of pilot respondents to identify parts for trimming, test phrasing and translations, evaluate the structure/flow/filters, and refine response categories.
EXAMPLE IMPLICATIONS
EXAMPLE IMPLICATIONS
EXAMPLE IMPLICATIONS
• Making informed decisions about whether to refer to ‘system’, ‘authorities’, or ‘providers’ • Avoiding perceived insensitive HPV questions
• It was verified that respondents generally understood and could answer in relation to the ‘health system’ and distinguish between ‘community’ and ‘hospital’ providers • Particularly in Kenya, a 5-pt response scale posed difficulties in Swahili – therefore, a 3-pt scale was used instead for most items
• Compression of a few trust sub-dimensions – e.g., skills & education weren’t sufficiently distinct to warrant separate items • Removal of a few response categories in multiplechoice matrices to make them more manageable – e.g., when asking who people trust 13
M E T H O D O L O G Y
|
S A M P L I N G
S T R AT E G Y
&
D ATA
P R O C E S S I N G
The sampling strategy was designed to produce a nationally representative sample of caregivers of children aged 10-14, plus representative samples of select areas of interest The primary goal of the study is to achieve a nationally representative sample for caregivers of children age 1014 in Kenya and Pakistan* To obtain national representation, we employed stratified multistage cluster sampling in collaboration with Ipsos Global Stratification & sample sizing: We first stratified Kenya and Pakistan into 13 and 9 strata respectively. Sample sizes prioritized six strata of interest in Kenya, and five in Pakistan. Remaining strata were allocated sample sizes proportionate to their populations with slight reductions due to prioritized areas. Selecting sampling units: We divided each stratum into primary sampling units, then secondary sampling units, and finally, blocks as the lowest cluster level Selection via random walks: At the final cluster level, we screened every kth household for adolescent children, collecting demographic information for each household. For each cluster, we conducted 8-9 interviews with qualifying households *) For security reasons, we excluded military areas in Pakistan and some areas in the Northeastern part of Kenya
The secondary goal is to produce representative samples of geographical areas of interest
PA K I S TA N
Lahore Karachi Islamabad
Azad Jammu and Kashmir Gilgit-Baltistan Balochistan
K E N YA
Nairobi Kisumu Kilifi
Kitui Isiolo Uasin Gishu
14
M E T H O D O L O G Y
|
S A M P L I N G
S T R AT E G Y
&
D ATA
P R O C E S S I N G
The definition of caregiving was kept deliberately inclusive to accommodate diverse family structures, with caregivers responding on behalf of a child in the household
We assumed a broad definition of caregiving to ensure the representation of caregivers beyond parents1
DEFINITION A caregiver is an adult who either: • • •
Makes financial, educational, or healthrelated decisions on behalf of children Engages in raising, rearing, or bringing up children Is responsible for disciplining children
ELIGIBLE
Each caregiver was asked to answer on behalf of either a boy or a girl 1) See the exact definition on the next slide 2) This practice was adopted once fieldwork had begun; first field guide version employed randomized selection with 70% chance of selection allocated to girls
SELECTED
If both a boy and a girl between the ages of 10-14 were part of the household, caregivers were asked to answer on behalf of the girl2 15
M E T H O D O L O G Y
|
S A M P L I N G
S T R AT E G Y
&
D ATA
P R O C E S S I N G
As a result, the Vaccine Trust Survey offers insight into an understudied yet crucial population— caregivers to adolescents—who are key to understand vaccine behavior W H AT W E H AV E D O N E …
W H AT O T H E R S H AV E D O N E …
Populations
Male caregivers to adolescents Female caregivers to adolescents
Vaccine behaviors
COVID-19 vaccine status HPV vaccine status Childhood vaccine status Zero dose status HPV vaccine awareness
Trust patterns
Trust in the health system promise Trust in the healthcare delivery Trust in vaccine promises Trust in vaccine delivery Interpersonal trust Institutional trust
FinAccess Household survey
Female caregiver Male caregivers × No trust data × No vaccine data Kenya 2021 & Pakistan 2018 (Wave 7)
Kenya 2021
The Social & Living Standards Measurement Survey Pakistan 2018
Kenya 2021 & Pakistan 2018
× Only childhood vaccine status children under 5yo × No HPV vaccine status × No HPV awareness × No trust data
× Only childhood vaccine status children under 3yr × No link between household data & vaccine data × No trust data
Global Monitor: Kenya 2018 & Pakistan 2018
Kenya 2022 (Round 8)
Institutional trust & interpersonal trust Trust in healthcare system × No vaccine data
Institutional trust & interpersonal trust Trust in healthcare system × No caregiver data × No vaccine data
Institutional trust & interpersonal trust Trust in healthcare system × No caregiver data × No vaccine data 16
M E T H O D O L O G Y
|
S A M P L I N G
S T R AT E G Y
&
D ATA
P R O C E S S I N G
All the results are calculated based on unweighted data to ensure transparency – and comparing weighted and unweighted results does not revealed significant differences C L E A N I N G DATA
W E I G T I N G T H E DATA
The final data set has been cleaned including the following steps:
This report presents the unweighted data to comply with best practices and ensure transparency around our results
• The data has been anonymized • Outliers have been removed based on income in Kenya • Variables have been recoded to have the same direction
There is very little difference in the trust scores in both Kenya and Pakistan when comparing the weighted and unweighted* results – suggesting a limited benefit of weighting the data PAK I STAN
”Allowing readers to understand who was interviewed and who was excluded, and to contextualize results, accordingly, may often be far more informative than any statistical adjustment done "behind the scenes".” - Collins et al. 20221
Unweighted
K E N YA
Weighted Unweighted
Weighted
Health system promise
64
64
70
71
Healthcare delivery
61
61
62
63
Vaccine promise
66
68
74
72
Vaccine delivery
71
72
70
70
*Weights are calculated based on income. The weights are available upon requests.
1) Collin et al. 2022 “Representativeness of remote methods in LMICs: A cross-national analysis of pandemic-era studies
17
Quantifying the VaccineTrust Framework 18
C H A P T E R
O V E R V I E W
In the following chapter, we build and test the validity of the Vaccine Trust Framework and its constitutive parts CHAPTER OUTLINE
Approach to building the Vaccine Trust Framework
We outline how the Trust Framework is quantified bottom-up.
Kenya and Pakistan’s Vaccine Trust Scores
Imputing HPV vaccine trust in Pakistan
Testing the validity of the Vaccine Trust Framework
We present the trust scores for Kenya and Pakistan.
We impute HPV vaccine trust in Pakistan with relevant proxies.
We test the internal and external validity of the Vaccine Trust Framework.
19
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
The Vaccine Trust Framework assumes that each individual has their own trust profile and scores – aggregating the individual trust scores yields the trust score T RU S T S C O R E S RESPONDENT 1
T RU S T S C O R E S RESPONDENT 2
T RU S T S C O R E S RESPONDENT 3
71
74
68
65
75
68
67
70
73
51
72
54
K E N YA’ S
T RU S T
70
SCOR
74
69 67
64
70
72
The trust score is reported on a scale from 0 – 100, with 0 indicating low trust, and 100 indicating high trust*. * Statistically the trust score is normalized to a scale on 0 – 1, before being multiplying with 100 20
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
To calculate the trust score, the Vaccine Trust Framework is quantified bottom-up progressing from dimensions, quadrants, levels, and ultimately, the full framework
From indicators to dimensions
From dimensions to quadrants
From quadrants to levels
Each dimension consists of a number of indicators, i.e., questions included in the survey
The Framework’s trust quadrants aggregate the scores of the trust dimensions for the quadrant in question
The level scores for the health system and vaccines respectively aggregate the quadrants that connect vertically
The full Trust Framework
21
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
Out of the 150 questions included in the survey, 50 questions serve as indicators for the trust scores that aggregate into a trust score at the dimension level S U RV E Y Q U E S T I O N S
Please tell me if you do not at all agree, somewhat agree, or strongly agree: “The healthcare system prioritizes my good health” Please tell me if you do not at all agree, somewhat agree, or strongly agree: “The healthcare system prioritizes the health issues that are most important for me and my community”
PRIORITY ALIGNMENT
0 Low trust
77
100 High trust
The trust score is reported on a scale from 0 – 100, with 0 indicating low trust, and 100 indicating high trust*.
By averaging the values, we obtain a unified index for each trust dimension
Each dimension consists of 1-9 indicators * Statistically the trust score is normalized to a scale on 0 – 1, before being multiplying with 100 22
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
The scores on dimensions are aggregated into the trust score on a quadrant level, again by averaging the individual’s trust score
64
PRIORITY ALIGNMENT
77
T RU S T S C O R E AT T H E Q UA D R A N T L E V E L
By calculating the average of the four dimensions that make up the quadrant for each individual and aggregating them, we obtain the trust score at the quadrant level —in this case, for trust in the health system promise
AUTONOMY
CAPA BI LI T Y FAIRNESS
77
49
51
23
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
Averaging the two quadrant scores for the health system or the vaccine produces the trust score at either the health system or vaccine level 77 49
64 T RU S T
SCORE
51
77 H E A LT H
S YS T E M
H E A LT H C A R E
60 T RU S T
P RO M I S E
D E L I V E RY
T RU S T S C O R E AT T H E H E A LT H S YS T E M L E V E L
71 SCORE
73
62
51
47 56
In this case, we have calculated the trust score at a health system level by averaging each individual’s trust scores at the quadrant level
24
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
A P P R O A C H
92 By combining the two levels, we obtain the full Vaccine Trust Framework and the trust scores at the vaccine and system 77level
64 QUA DR ANT LEVEL
QUA DR ANT LEVEL
53
S YS T E M
H E A LT H C A R E
P RO M I S E
VA C C I N E
D E L I V E RY
VA C C I N E
71
QUADRANT SCORE
62
69
51
77 H E A LT H
60
63
49
HEALTH SYSTEM LEVEL
70 73
51
47 56
61
80
P RO M I S E D E L I V E RY
70
QUA DR ANT LEVEL
70
VA C C I N E L E V E L 25
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
T R U S T
S C O R E S
K E N YA
gn ali
omy
es riti
Be nef it
io Pr
n Auto
This approach produces the following framework and trust scores for Kenya, with the vaccine level for the HPV vaccine n me t
Ca pa
70 S Y S T E M
P RO M I S E
D E L I V E RY lity dentia Confi
on De liv er y
bility
ss
Afford a
HEALTH SYSTEM LEVEL
ssi
Ac ce
a mp Co
ty
se t
tin
g
74
VA C C I N E
P RO M I S E
VA C C I N E
D E L I V E RY
Adeq u
cy en Ag
66
s
en ce
62
e Saf
pe t
H E A LT H C A R E
nes
e nc
bil ity
Co m
H E A LT H
Fair
va le Re
acy o f info
70
72
VA C C I N E L E V E L 26
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
T R U S T
S C O R E S
K E N YA
n Auto
io Pr gn ali
omy
es riti
Be nef it
…And a total trust score of 69, compounding the health system and vaccine level
n me t
S Y S T E M
lity dentia Confi
HEALTH SYSTEM LEVEL
ssi
on
e Saf
ss
ty
eli ve ry
se t
tin
g
74
VA C C I N E
P RO M I S E
VA C C I N E
D E L I V E RY
Adeq u
G E N E R A L T R U S T S C O R ED
Ac ce
a mp Co
cy en Ag
66
69
D E L I V E RY
en ce
62
P RO M I S E
pe t
H E A LT H C A R E
s
Co m
H E A LT H
nes
bility
70
Fair
e nc
bil ity
Afford a
Ca pa
va le Re
acy o f info
70
72
VA C C I N E L E V E L 27
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
T R U S T
S C O R E S
PA K I S TA N
gn ali
o my
es riti
Be nef it
io Pr
n Auto
In Pakistan, the HPV vaccine remains to be rolled out—which, at first, leaves us with the trust score at the health system level alone n me t
ce an ev l Re
Ca pa
64 S Y S T E M
ty
P RO M I S E
D E L I V E RY lity dentia Confi
De liv er y
se t
tin g
ability Afford
ncy
HEALTH SYSTEM LEVEL
acy o f info
i on ss
Ac ces s
a mp Co
Adeq u
e Ag
62
s
ce
61
e Saf
en
H E A LT H C A R E
nes
Co m pe t
H E A LT H
Fair
bi l ity
28
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I M P U T I N G
H P V
R E S P O N S E
However, we have observed that the HPV vaccine occupies a space between childhood and adult vaccines in fieldwork—and we see a similar pattern in the Kenyan survey data During fieldwork, we observed how the HPV vaccine occupied its own space in respondents’ mental models around vaccines…
This informed the hypothesis that trust in the HPV vaccine is neither entirely correlated with C19, nor childhood vaccines 1 – trust in HPV vac.
1 - trust in HPV vac.
1 – Trust in childhood vac.
βC19 0.24 βchildhood 0.68
Fieldwork in this project’s earlier phase suggested that the HPV vaccine targeting adolescent girls carries potential adult connotations while still being a vaccination decision made by caregivers on behalf of a child
βC19 0.55
0 Trust in Childhood vac. - 1
0
P-value <0.0001
P-value <0.0001
Trust in C19 vac. - 1
0
Trust in C19 vac. - 1
P-value <0.0001
Both childhood and C19 vaccine items help explain unique variation in the response to the HPV vaccine in Kenya and remain significant when included in the same model —supporting the interpretation that both vaccine items can be used to impute HPV responses 29
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I M P U T I N G
H P V
R E S P O N S E
With this in mind, we can impute the response to the HPV vaccine by averaging trust levels in COVID-19 and childhood vaccines B E N E F I T
T E S T I N G M E T H O D I N K E N YA
R E L E VA N C E
When averaging trust in childhood and C19 vaccines in Kenya to impute trust in the HPV vaccine, we find that each imputed dimension varies by a maximum of +/- 10 in trust score compared to observed scores
S A F E T Y
C AV E AT S
Pulling the average assumes that the context observed in Kenya—HPV occupying its own space due to its target group being adolescents—is also applicable to Pakistan.
FINDING
A D E QUAC Y O F I N F O
D E L I V E RY S E T T I N G S AG E N C Y
Calculating the average trust in childhood and C19 vaccines provides a relatively accurate picture of the observed HPV response in Kenya
Observed HPV response
An estimate based on childhood vaccines and C19 unifies two types of vaccines with diverse timelines, presenting a challenge in pinpointing when the imputed trust levels for the HPV vaccine would materialize. In particular trust in childhood vaccines appears to contribute to a slight overestimation of trust in the HPV vaccine.
Imputed HPV response 30
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
T R U S T
S C O R E S
PA K I S TA N
fit Be ne
en nm lig sa
o my
itie ior Pr
n Auto
This method provides us with the following framework and trust scores for Pakistan, including imputed trust scores for the vaccine level t
ce an ev l Re
Ca pa
64 S Y S T E M
lity dentia Confi
De liv er y
bility Afford a
HEALTH SYSTEM LEVEL
n sio
ss
as mp Co
ty
se t
tin g
66
VA C C I N E
P RO M I S E
VA C C I N E
D E L I V E RY
Adeq uacy of
cy en Ag
63
D E L I V E RY
Ac ce
61
P RO M I S E
nc e
H E A LT H C A R E
e Saf
Fair nes s
Co m pe te
H E A LT H
bil ity
info
71
68
VA C C I N E L E V E L 31
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
T R U S T
S C O R E S
PA K I S TA N
o my
en nm lig sa
Be ne
n Auto
itie ior Pr
fit
…And a compound trust score of 66 for Pakistan
t
ce an ev l Re
S Y S T E M
HEALTH SYSTEM LEVEL
n sio
e Saf
ty
el ive ry
se t
tin g
66
VA C C I N E
P RO M I S E
VA C C I N E
D E L I V E RY
Adeq uacy of
G E N E R A L T R U S T S C O R ED ss
as mp Co
cy en Ag
63
66
lity dentia Confi
Ac ce
61
D E L I V E RY
nc e
H E A LT H C A R E
P RO M I S E
Co m pe te
H E A LT H
Fair nes s
bility
64
bil ity
Afford a
Ca pa
info
71
68
VA C C I N E L E V E L 32
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
The Trust Framework’s components are interrelated, but also all uniquely necessary to conceptualize trust—striking a balance between consistency and distinctiveness
FUL LY DI STI NC T
F U L LY C O N S I S T E N T
Producing fully consistent quadrants and dimensions, i.e., obtaining perfect correlation, would indicate that the Framework’s different components are obsolete
CRITERIA 1
The Trust Framework
Within a level & quadrant
Striking a balance between consistency and distinctiveness within a quadrant, i.e., ensuring that dimensions contribute to measuring a specific type of trust
CRITERIA 2
Producing fully distinct quadrants and dimensions, i.e., obtaining zero correlation, would indicate that the Framework’s components do not measure the same phenomena
Between quadrants
Striking a balance between consistency and distinctiveness between quadrants, i.e., ensuring that all four kinds of trust are in fact related and needed to conceptualize trust as a whole
33
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
|
W I T H I N
L E V E L S
Within levels: Dimensions are more closely connected within each level on average, suggesting that the Framework’s health system and vaccine levels are statistically consistent
HEA LT H S YSTE M
HEA LTH SYS TEM LE VEL
Autonomy Priority alignment Capability Fairness Confidentiality Compassion Competence Access Affordability
VA C C I N E
Benefit Relevance Safety Adequacy of info Delivery setting Agency
VA C C I N E L E V E L
Autonomy
Priority alignment
Capability
Fairness
Confidentiality
Compassion
Competence
Access
Affordability
Benefit
Relevance
Safety
NA
0.21
0.07
0.10
0.24
0.17
0.08
0.19
0.09
0.11
0.10
0.09
0.16
0.16
0.06
0.21
NA
0.26
0.04
0.17
0.29
0.11
0.17
0.09
0.14
0.11
0.09
0.19
0.22
0.12
0.07
0.26
NA
-0.05
0.11
0.24
0.17
0.18
0.15
0.08
0.06
0.12
0.15
0.16
0.14
0.16
NA
0.12
0.07
0.05
0.12
0.16
0.24
0.11
0.13
0.12
0.31
0.23
0.10
0.15
0.11
0.08
0.13 0.04 Average correlation NA within the health 0.37 0.04 0.37 NA system level
0.06
0.13
Average correlation between 0.04 0.09 0.04 0.04 vaccine0.15 dimensions and 0.06 0.08 0.17 dimensions 0.15system 0.18 0.23 0.07
0.06
0.16
0.31
NA
0.16
0.14
0.07
0.07
0.13
0.15
0.11
0.11
0.24
0.23
0.16
NA
0.33
0.10
0.08
0.09
0.11
0.09
0.05
0.10
0.10
Adequacy of info Delivery setting
Agency
0.04
-0.05
0.17
0.11
0.29
0.24
0.11
0.17
0.19
0.17
0.18
0.12
0.09
0.09
0.15
0.07
0.11
0.10
0.14
0.33
NA
-0.01
0.00
0.09
0.07
0.03
0.05
0.11
0.14
0.08
0.05
0.13
0.15
0.07
0.10
-0.01
NA
0.43
0.22
0.29
0.40
0.13
0.10
0.11
0.06
0.12
0.12
0.11
0.07
0.08
0.00
0.43
NA
0.27
0.31
0.29
0.07
0.09
0.10
0.09
0.09
0.22
0.27
0.24 0.17
0.16 0.16 0.06
0.09
0.12
0.19
0.15
0.22
0.16
Average correlation between 0.13 0.08 0.15 vaccine dimensions and 0.09 0.15 0.18 0.15 system dimensions 0.04 0.17 0.23 0.11
0.12
0.14
0.04
0.04
0.06
0.07
0.11
0.29
0.11
0.07
0.29
0.31
0.09
0.03
0.40
0.29
0.05
0.05
0.13
0.07
0.08 Average correlation 0.28 0.41 withinNAthe vaccine level 0.24 NA
0.28
0.29
0.29
0.41
NA
0.33
0.08
0.24
0.33
NA 34
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
|
W I T H I N
Q U A D R A N T S
Within quadrants: Using the Vaccine Trust Framework to allocate dimensions, we improve internal correlation in three out of four quadrants compared to distribution at random P RO C E D U R E
RESULTS Avg. correlation (at random)
While we have qualitative reason to group dimensions as per the Vaccine Trust Framework, we also tested internal consistency within quadrants by comparing internal quadrant correlation with the correlation coefficient had the dimensions been distributed at random.
Avg. correlation within true quadrant
Change (%)
Health system promise
0.15
0.10
-29%
Healthcare delivery
0.15
0.22
48%
Vaccine promise
0.15
0.31
111%
Vaccine delivery
0.15
0.32
122%
Health system promise is the only quadrant that doesn’t improve its internal correlation when compared with the random distribution. This is expected as the quadrant is the most abstract and difficult to measure, compounding large concepts such as fairness and autonomy. 35
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
|
W I T H I N
Q U A D R A N T S
Within quadrants: Health system promise has relatively low internal consistency due to two dimensions showing negative correlation—a challenge the Vaccine Trust Tool will address Two dimensions in Health system promise, Capability and Fairness, are negatively correlated, resulting in a reduced correlation score for this quadrant Autonomy
Priority alignment
Capability
Fairness
The Trust Tool aims to shed light on this dynamic through re-operationalization of Fairness The observed negative correlation could indicate a scenario in which some respondents view the health system as capable of treating illnesses, but also experience that they are to some extent excluded from treatment because of socio-demographic markers such as ethnicity, gender, and geography.
Autonomy
1
0.21
0.07
0.10
Priority alignment
0.21
1
0.26
0.04
Capability
0.07
0.26
1
-0.05
Notably, Fairness represents a complex dimension that by definition could negatively correlate with other dimensions e.g. capability as people might evaluate the health system positively for others than themselves.
Fairness
0.10
0.04
-0.05
1
To examine this, the Vaccine Trust Tool will revisit the operationalization of dimensions to clarify this dynamic.
CURRENT C A P A B I L I T Y
VERSIONS F A I R N E S S
Being treated differently in the health system because of …
REVISED
VERSION
F A I R N E S S
Being treated more poorly in the health system because of… 36 PROPRIETARY AND CONFIDENTIAL | 36
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
|
B E T W E E N
Q U A D R A N T S
Between quadrants: We find that the quadrants hold unique variation when fitted in the same regression model—indicating that they are related, but also distinct RESULTS
P RO C E D U R E
System
Independent variable II
Vaccine
Independent variable I
All quadrants hold unique variation
Dependent variable
Independent variable III
To test relatedness and distinctiveness between quadrants, we fit four regression models with one quadrant as the ‘dependent’ variable and the remaining three quadrants as independent variables, assessing whether they all hold unique variation when predicting the last quadrant
All quadrants are highly significant (p-values < 0.0001) in all models. As expected, same-level quadrants produce higher coefficients, and opposite-level quadrants slightly lower coefficients. OLS Dep: Health System Promise
RESULTS*
C O E F.
Dep: Vaccine promise
C O E F.
Healthcare delivery
0.42
Health system promise
0.09
Vaccine promise
0.06
Healthcare delivery
0.14
Vaccine delivery
0.1
Vaccine delivery
0.29
Dep: Healthcare delivery
C O E F.
Dep: Vaccine delivery
C O E F.
Health system promise
0.27
Health system promise
0.24
Vaccine promise
0.05
Healthcare delivery
0.19
Vaccine delivery
0.05
Vaccine promise
0.43
*) Intercepts omitted for simplicity
37
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
I N T E R N A L
VA L I D AT I O N
|
M U LT I C O L L I N E A R I T Y
Our data shows multicollinearity between the four trust quadrants—but results remain statistically significant throughout when predicting vaccination behavior The Vaccine Trust Framework conceptualizes trust as four related trust types, making multicollinearity a premise for analysis The trust quadrants display low to moderate pairwise correlation—however, Variance Inflation Factors (VIF) are high (+10), suggesting that the combination of trust quadrants drive multicollinearity, and in turn, increases statistical uncertainty of results PA I RW I S E C O R R E L AT I O N * Health promise
Healthcare delivery
Vaccine promise
VIF ANALYSIS* Vaccine delivery
Health promise
20
Health promise
1
0.4
0.2
0.26
Healthcare delivery
21
Healthcare delivery
0.4
1
0.19
0.22
Vaccine promise
14
Vaccine promise
0.2
0.19
1
0.4
Vaccine delivery
11
Vaccine delivery
0.26
0.22
0.4
1
*) Calculated based on results from Kenya on the HPV vaccine
Despite multicollinearity, we observe significant results across analyses of the four trust quadrants This indicates that although results are subject to higher statistical uncertainty, trust comes out as a strong predictor of vaccine uptake.
38
Q U A N T I F Y I N G
T H E
F R A M E W O R K
|
E X T E R N A L
VA L I D AT I O N
External validation: Compared to other widely used trust measures, the Vaccine Trust Tool turns out to add signficantly more explanation to variations in vaccine uptake Interpersonal trust measures are not helpful in explaining variation in vaccine uptake, producing statistically insignificant results. For the Vaccine Trust Tool, the correlations are stronger and highly significant indicating a strong relationship between trust and vaccine uptake.
Logistical regression correlations between vaccine uptake and different trust measures reported in log odds
C19 vaccine uptake
Interpersonal trust2
NOT
0.13
INTERPERSONAL TRUST
Interpersonal trust is a widely used and accepted measure of trust in the trust literature – often applied in studies on health-seeking behaviors and vaccine uptake. The question: “Generally speaking, would you say that most people can be trusted or that you need to be very careful in dealing with people?” stems from the World Value Survey1.
1) World Value Survey Wave 7 (2017-2022). Documentation: The questionnaire. https://www.worldvaluessurvey.org/WVSDocumentationWV7.jsp
Childhood vaccine uptake
HPV vaccine uptake
SIGNIFICANT
0.19
-0.17
SIGNIFICANT
Vaccine Trust Tool3
2.58***
1.42**
3.68***
Correlations are controlled for respondent gender, age, educational attainment, language, religion and province. **p<0.01; ***p<0.001 2) Measured by the binary question “Generally speaking, would you say that most people can be trusted or that you must be very careful in dealing with people? ” 3) Measured as a combined measure of the four trust quadrants in the Vaccine Trust Framework: Trust in the health system promise, the healthcare delivery, the vaccine promise, and the vaccine delivery.
39