6.1 Inclusion criteria for the systematic review and meta-analysis

While cognizant of this limitation, a quantitative synthesis of individual studies is still useful and timely, as financial incentive interventions as a whole, and each of the three intervention groups individually, have welldefined common characteristics (figure 6.1). Because of these common features, policy discussions typically aggregate “financial incentives in health” into three general groups: PBF, vouchers, and CCT programs. Obtaining mean effect sizes through meta-analysis of all available evidence is therefore preferable over the ad-hoc, implicit aggregation of often selective study results, which is frequently undertaken.

Methodology This chapter’s results were obtained using systematic reviews and metaanalysis methodology (see, for example, Higgins and Green 2011; Waddington et al. 2012; Card 2015). Table 6.1 summarizes the main inclusion criteria that were used to identify relevant studies.

Table 6.1 Inclusion criteria for the systematic review and meta-analysis

Type of inclusion criteria Criteria used in this study Publication format Studies in English that were published in peer-reviewed scientific journals, as part of a working paper series, in books (with ISBN numbers), as doctoral dissertations, or official research or project reports Interventions Performance-based financing, voucher, and conditional cash transfer schemesa occurring in countries classified as low or middle income by the World Bank Outcomes Six indicators that represent the official and supplemental reproductive, maternal, and child health indicators of the Millennium Development Goals (Wagstaff and Claeson 2004) or are intermediate indicators critical to their achievement, namely, the shares of (1) women of fertile age who use modern contraceptives, (2) pregnancies with four or more antenatal care checks, (3) pregnant women receiving tetanus vaccinations, (4) births occurring in health facilities, (5) births with postnatal care, and (6) children receiving the full course of vaccinations recommended for the first year of life Data source Only evidence from household survey data due to sample selectivity and reporting bias concerns in health facility and administrative data sets from low- and middle-income countries (Chiba, Oguttu, and Nakayama 2012; Hahn, Wanjala, and Marx 2013; Sharma et al. 2016) Study design Randomized controlled trials as well as evaluations of nonrandomized interventions that identify impacts using regression discontinuity design, instrumental variables, or double difference and triple difference models

Source: World Bank.

a. Because they are based on a different theory of change, the review does not consider interventions that affect the monetary price of providing or using MCH services only indirectly or implicitly. On the supply side, omitted interventions include health worker training, provider performance tournaments, and the introduction of mobile health units or health worker home visits. On the demand side, excluded interventions include information campaigns, unconditional cash transfers, and conditional cash transfer schemes that do not condition on MCH service use or employ soft conditions or co-responsibilities. MCH = maternal and child health.

The outcomes included are mainly service indicators because they are the ones most often and most uniformly reported by the studies included in the analysis. However, some of the indicators considered, such as tetanus vaccination during antenatal care (ANC), at least four ANC visits, and full vaccination, also have a quality component. Importantly, the analysis only includes impact estimates of outcomes whose provision is financially incentivized by the interventions under study. This incentivization may be direct (for example, a fee a health facility receives for each birth taking place in it) or indirect (for example, a maternal tetanus vaccination being incentivized in a CCT that rewards pregnant women for ANC visits during which maternal tetanus vaccinations are carried out). By contrast, the analysis excludes impact estimates of outcomes without financial incentivization, for example, those measuring an intervention’s unintended consequences.

Further, the parameter of interest in this review is a program’s intentionto-treat effect—the impact on its full target population that consists of both compliers and noncompliers. Thus, the excluded effects are estimated only for compliers, for example, only for enrollees in a CCT scheme as opposed to its entire target group. As a requirement for the meta-analysis, impact estimates are only included if they are presented with a measure of statistical uncertainty.

Intervention characteristics

The results of the search are described in greater detail in box 6.3. Table 6A.1, in annex 6A, provides an overview of the 52 programs in the review and their underlying reports. The study design, program characteristics, and implementation contexts vary both across and within the three intervention groups. About 55 percent of the studies in the review have randomized designs, which are most common for CCT programs, and only three studies rely on instrumental variable and regression discontinuity design models to identify program impacts. Due to the review’s strict methodological inclusion criteria, the share of studies with high bias risk is only 29 percent, 45 percent of studies are classified as having medium bias risk, and 26 percent as having low bias risk.

In terms of implementation context, the geographic coverage of the review is illustrated in map 6.1. Among the PBF programs in the review, 82 percent are in Sub-Saharan Africa, compared with 40 percent of the voucher programs and 35 percent of the CCT programs. The distribution is more balanced across country income groups, where 55 percent of PBF,