What to Look for When Conducting Method Assessments

Measuring the Performance of Targeting Methods | 473

implementation, which are often reflected in poor outcomes. This is not necessarily due to the design of the program or the method used; it could be due to how the delivery chain components were implemented.

Summary of Elaborations or Corrections of the Initial Limited Analysis of the Outcomes of the Bolsa Escola Case Study • Using the right income eligibility cutoff as opposed to arbitrarily using the poverty line • Restricting the sample to households within the income eligibility to those that also comply with the categorical requirements of the method, namely having children in a certain age group • Checking for the coexistence of other mutually exclusive social programs that might explain low take-up of a specific program • Understanding coverage over the entire population distribution • Understanding the implementation challenges, since it is not always possible to measure them • Using a data set that allows for population-wide analysis of program coverage and incidence.

Some Aspects That Could Affect the Method’s Performance but Often Cannot Be Measured, Especially in Poorer Countries or, in the Case of Brazil, Poorer Municipalities • The year of the survey may be more recent than the year in which the most recent survey sweep/large-scale recertification of beneficiaries was done; thus, some households that were eligible when assessed have prospered in a long-run sense or had a good year. • Families may be excluded due to failures in the delivery system—limited outreach, high transaction costs, lack of identification (ID), and so forth. • An eligibility assessment improperly classifies some households.

What to Look for When Conducting Method Assessments

1. When measuring errors of inclusion and exclusion, it is vital to use the same threshold as used for eligibility rather than a more generous definition of poverty.

As evidenced in the illustrative case study on the Bolsa Escola program in Brazil, using different eligibility cutoffs for analysis than those used in the program design can lead to inaccurate assessments of accuracy and coverage of the eligible population. To illustrate this, consider an economy of 10 individuals, a poverty line of $20, and a government desire to