Discussion and conclusions

however, this effect persisted 12 months after the increase in performance pay ended, suggesting that providers respond to more than just the price. However, the study also finds that the quality of care may have remained a constraint to improving health outcomes as the increase in the early initiation of prenatal care did not have any effect on birth outcomes. Second, lab-in-the-field experimental price elasticity estimates from Nigeria (Bauhoff and Kandpal 2021) cast some doubt on the degree to which providers respond to increases in performance pay. This result suggests that the primary role of the price may be to increase the salience of the information conveyed by the checklist. This in turn suggests another way—instead of risk-based verification—in which PBF programs can be made more cost-effective, which is by simply providing token prices. Finally, it is also important to recognize that PBF can be difficult to implement (Paul et al. 2018), making it an expensive and perhaps risky way of getting money to the frontlines.

Several insights emerge from the analysis presented in this chapter. The results from the meta-analysis indicate that while on average financial incentives increase the coverage of all included RMCH service indicators, the effect sizes are relatively modest, ranging between 2 and 6 percentage points, with the largest effects being for facility delivery and full childhood vaccination (about 5 percentage points). Effect size heterogeneity across financial incentive programs is estimated to be low to moderate for all the indicators except maternal tetanus vaccination.

The low to moderate levels of effect size heterogeneity across financial incentive interventions is reflected in generally small differences in mean effect sizes across the PBF, voucher, and CCT interventions. The analysis is not sufficiently powered to determine precisely the magnitudes of these differences, but, overall, the results indicate that PBF might be slightly less effective in improving RMCH coverage than voucher and CCT schemes.

When testing for other possible drivers of effect size heterogeneity across financial incentive programs, the analysis finds neither systematic evidence of complementarities between supply- and demand-side incentives nor systematic evidence of an influence of baseline indicator levels. The results of the systematic review and meta-analysis are subject to limitations. The methodological inclusion criteria are demanding, which can be considered

a strength of the analysis. The exclusion of studies with less rigorous empirical methods and study designs, however, further limits the statistical power, which, despite a growing evidence base, remains insufficient for conducting a more detailed analysis of the roles of intervention design features and implementation contexts. This limitation applies to comparisons of PBF, voucher, and CCT effect sizes—the differences reported should be given a strictly associational interpretation as the analysis cannot control for confounding factors. To allow more fine-grained subgroup comparisons going forward, future studies of financial incentive interventions should rely on rigorous impact evaluation methods, reduce avoidable heterogeneity by using standard outcome variable definitions, and include detailed information about program design features.

Another limitation is that the scope of indicators targeted by financial incentive interventions is usually larger than the narrow set of outcomes in this review. Many programs incentivize other health coverage indicators in and outside the RMCH domain (for a review and summary of the evidence on the demand side, see, for example, Neelsen et al. (2021)). CCT programs often also include education and job training conditionalities and, like vouchers, can have additional effects on household consumption and welfare. Similarly, incentives to improve the quality of facility equipment and cleanliness and to streamline administrative processes are almost always included in PBF programs, whose impacts on transparency, provider accountability, and data usage are often hoped to have a transformational effect on overall health systems (Friedman and Scheffler 2016; Ma-Nitu et al. 2018). For these reasons, it is important to stress that the results of the meta-analysis only support conclusions about the effects of financial incentives on the six included indicators but not about the overall (cost-) effectiveness of specific programs or entire intervention types.

Beyond the meta-analysis, evidence from the Nigeria impact evaluation suggests that performance pay may be most effective at improving coverage where the baseline levels of the indicator are low, suggesting that paying for indicators where baseline coverage is high may not be effective since the effort required to increase coverage is not commensurate with the price paid. For indicators where baseline coverage is particularly low, demandside barriers may be salient and at least partially addressed using low-cost cash transfers to patients/households.

Many of the constraints to effective coverage are not in the health worker’s locus of control and thus do not respond well to pay-for-performance incentives. As a result, perhaps unsurprisingly, the analysis finds that

DFF, typically paired with facility-level autonomy and supervision reforms, can improve coverage and effective coverage to a similar degree as PBF— often at lower cost, since DFF does not require a verification mechanism. This mixed evidence of effectiveness is a reminder that PBF, as probably any complex intervention, may fail to improve effective coverage, and this failure can be driven by a variety of reasons.

The mixed results on the effectiveness of PBF interventions presented in this chapter highlight the complexities of a PBF intervention and suggest that thinking of DFF, PBF, and demand-side cash transfers as a menu of potentially reinforcing policy options may be a fruitful means for increasing effective coverage. The evidence also raises questions about the appeal of PBF as the primary attempt at health financing in most of the developing world. It also begs the question as to whether the substantial donor finances channeled into PBF could be used more fruitfully—at least partially—in other types of health financing approaches. DFF or direct transfers to households may attain much of the progress achieved through PBF at a lower administrative cost and with less heterogeneity in impact. Even when PBF is used, it may make sense to purchase only a small number of targeted indicators. These may be selected for such strategic purchasing if they are within the health worker’s locus of control. For instance, health workers may not be in a position to respond to incentives for ANC visits because of demand-side constraints, but once a woman uses ANC services, health workers may be able to use quality of care during those visits to convince her to come back for delivery (Basinga et al. 2011; Bonfrer, Van de Poel, and Doorslaer 2014). The chapter also provided several reasons that suggest caution in attributing the observed impact of PBF on institutional deliveries entirely to the price set in the PBF system. It further discussed suggestive evidence that a token price might capture much of the benefit of performance pay, suggesting that even effective PBF programs might be made more cost-effective.

The evidence brought to bear in this report comes, in large part, from a unique effort at systematically learning about the impact of PBF approaches. In 2008, at the onset of the first trust fund, HRITF, that funded these PBF trials and impact evaluations, there were many unanswered questions about the use of PBF to improve effective coverage in LMICs. As this chapter has demonstrated, the large number of pilot projects and impact evaluations funded by the HRITF has led to substantial learning about where, when, and why PBF approaches might work and how they can be strengthened.

Nonetheless, this body of evidence also highlights that PBF is far from the only, or even the most uniformly appealing, option on the path to sustainable health facility financing and universal health coverage. By shedding light on a menu of viable policy options available for health system financing and strengthening, this report aims to provide options for policy makers. Both PBF and DFF represent notable improvements over businessas-usual in moving forward with the desired transformation of health systems. Compared with PBF, the ease of implementation and low administrative costs of DFF may make it particularly appealing to country governments in responding to the health system challenges imposed by COVID-19, but the link to results in PBF approaches may make it more appealing to donors. Even with the gains from PBF or DFF, the end levels of effective coverage remain poor and much work remains to be done to meet the Sustainable Development Goal of good health and well-being for all. PBF or DFF alone is unlikely to close the entirety of the gap that remains. Thus, the report closes with two chapters that look ahead by building on the findings presented here. Chapter 7 presents a research agenda and cautionary evidence on the risks of overutilization of care on using PBF to develop health systems; chapter 8 closes with a discussion of the operational implications of the research reviewed here.