Formulae

404 | Revisiting Targeting in Social Assistance

accurate PMT prediction might be, as well as identify some variables that would need to be in the model regardless of their significance level.

An intriguing way of overcoming some of the limitations of traditional household surveys that are designed to be representative of the general population is to use data that are more keyed to the poorer population. Some surveys oversample to be able to estimate proper indicators among the poor. For example, in Mexico, the Socioeconomic Conditions Module was created as a complement to the National Survey of Household Income and Expenditure (ENIGH), with the purpose of providing the necessary information for the National Council for the Evaluation of Social Development Policy to carry out the measurement of multidimensional poverty at the national and state scales. The ENIGH and Socioeconomic Conditions Module combination was implemented starting in 2008, and to be able to estimate poverty and multidimensional poverty by state, the ENIGH sample, which was 35,146 households, almost doubled to reach 70,106 households. Another option is to use data or samples from the social registry, which by design is concentrated among the poorer (see box 6.7).

BOX 6.7

Using Data from the Social Registry to Update PMT Formulae

As more countries are investing in developing integrated social registries to support the delivery of one or many programs, the data gathered can be used to calibrate beneficiary selection formulae in ways that overcome some of the limitations of traditional household surveys. Paes de Barros et al. (2016) show that basing a proxy means testing (PMT) formula on the Brazilian Unified Registry of Social Programs (Cadastro Único, or CadÚnico) would improve eligibility determination for the Renda Melhor guaranteed minimum income–type program created by the state of Rio de Janeiro in 2011, to complement the Bolsa Familia program for extremely poor families.

Until 2016, the Renda Melhor program offered each of the Bolsa Familia beneficiaries in the state of Rio de Janeiro an extra benefit that would complement the post–Bolsa Familia beneficiary income by the income gap needed to reach to the extreme poverty line of R$100 (US$29) per capita. Therefore, to be in the program, a family should be eligible for the Bolsa Familia program through its means-test approach

continued next page

How to Harness the Power of Data and Inference | 405

BOX 6.7 (continued)

from the CadÚnico, which is used by more than 20 federal social programs and by many states and municipalities for developing social policies and programs at the local level.a However, while Bolsa Família has been using only declared income as the eligibility criterion, the Renda Melhor program has sought to make full use of the variety of information contained in the CadÚnico. Policy makers started using a secondary approach to estimate the full income of households eligible for Bolsa Família under the assumption that the declared income was not a good representation of the household’s permanent income. Hence, they used a PMT to estimate this permanent income based on the information available in the national household survey data and the CadÚnico. This approach seems promising under the conditions of having a social registry that is large enough, regularly updated, dynamic, and assessed as providing a good representation of the poor as these data can work almost as a census of the poor population. On such basis, the authors conducted the following experiment: • From the CadÚnico, a probabilistic two-stage sample of 4,000 households was extracted and guaranteed representativeness of the CadÚnico population in the State of Rio de Janeiro, and for three income groups: (1) families with self-declared per capita income below half the minimum wage,b (2) families living in households with total self-declared monthly income of up to three times the minimum wage, and (3) families with self-declared incomes greater than three times the minimum wage and receiving any social program from the three levels of government (federal, state, and municipal). • A special survey was administered to households in the sample.

The survey questionnaire was designed to mimic the CadÚnico form, with a few extra variables such as a more detailed income module to address the specific needs of the assessment. • Once the 4,000-household data collection was completed, researchers merged the survey and CadÚnico data to have two measures of income: (1) the self-declared income from CadÚnico and (2) the full and detailed income collected through the survey. • Researchers estimated the underreporting elasticity of the

CadÚnico income by comparing the two incomes and reestimated the PMT model used by the Renda Melhor program. • The new PMT predictor was then applied to all the data for the

State of the Rio de Janeiro to determine eligibility for the Renda

Melhor program.

continued next page

406 | Revisiting Targeting in Social Assistance

BOX 6.7 (continued)

• The researchers estimated that a new PMT could reduce the

Renda Melhor program’s inclusion errors from the original PMT model by 7–33 percent. Exclusion errors remained at the same level.

This exercise highlights the importance of continuously using the information in the social registry as it contains a denser concentration of the poor and vulnerable population than in any household survey. The authors believe that eligibility determination is strengthened by having high-quality information gathering at the application stage, requiring consistent definitions and concepts that are consistent with the national statistical office questionnaire for collecting income/consumption information, and allowing program administrators to define a recurrent monitoring and evaluation strategy to measure the quality of information.

A proposed approach would be routinely sampling a subset of applicants and gathering from them a longer questionnaire as in national survey samples. The sample would also allow administrators to have a deeper understanding of population and poverty dynamics through using statistical pattern recognition, which could in turn help to calibrate circumstances that would trigger the need to update information, or for provision of additional documents for checks and home visit inspections. However, to avoid underdeclaration of income in these survey efforts, it is important to ensure that the process for collecting such data is not seen as a condition for eligibility decisions but as regular monitoring and evaluation of government strategies. For example, data for this sample can be collected later and not at the stage of enrollment in the program.

a. Cadastro Único (CadÚnico) was officially created in 2001 through a Presidential Decree (#3,887) by Fernando Henrique Cardoso. The implementation of the first large-scale expansion of the CadÚnico started in 2003 during the phase of consolidation (2003–05) of four cash transfers schemes (Bolsa Escola, Bolsa Alimentação, Cartão Alimentação, and Vale Gás) into the Bolsa Familia program that formed the initial largest base for the CadÚnico. As the CadÚnico matured during 2006–09, it became the gateway for benefiting from low-income families social policies. Between 2010 and 2013, CadÚnico version 7 introduced online synchronization with the federal center and other systems, such as pension systems. The CadÚnico management involves the three levels of the federation: municipalities, states, and the federal government. The municipalities are the main actors as they are in charge of its implementation (for example, they identify low-income families; interview, collect, and register data in the national database; keep data updated; promote continuous capacity building to agents; and provide and maintain adequate infrastructure in the centers; keep and protect confidential information; and take measures to control and prevent fraud or registry inconsistencies). The states have a more planning and capacity-building role to provide municipalities the right skills and develop specific actions to register

continued next page

How to Harness the Power of Data and Inference | 407

BOX 6.7 (continued)

traditional and specific populations, such as quilombolas, indigenous, and homeless populations. The federal-level managers support coordination, supervision, and monitoring; define strategies and instructions to improve the quality of the information registered; and give financial support to municipalities and states to strengthen their capacity to manage and run the CadÚnico. At present, more than 20 federal social programs make use of the CadÚnico, which is also used by many states and municipalities for developing social policies and programs at the local level. Today, CadÚnico contains the details of 28 million families, of which 14 million are beneficiaries of the Bolsa Familia program.

b. In 2019, the minimum wage in Brazil was set at R$998, equivalent to approximately US$249.27 a month or US$8.40 a day, as of October 2019.

PMT’s Predictive Power beyond Chronic Poverty

PMT models are usually not good at predicting vulnerability to poverty from shocks. The chronic poor are often more vulnerable to falling into poverty from shocks, as they frequently live in more hazardous places, such as along railway tracks or places that are more vulnerable to climate change, and work in more occupations with significantly variable incomes. They also have less ability to cope with shocks when they happen. For these reasons, chapter 3 stresses that getting support out quickly through existing social assistance programs when a shock first happens is a good strategy even when the authorities have not yet identified the most affected people. However, being able to predict vulnerability to shocks before they happen can help guide programs such as insurance or incentives to use insurance or other risk-management strategies.

PMT models are not designed for identifying households after they suffer a shock. These models are designed to identify chronic poverty and low incomes based on proxies that are fixed or change only slowly over time, such as housing quality and demographics. As a consequence, when a household suffers a shock, whether idiosyncratic or covariate, its PMT score may not change or change only a little. For example, the household composition will likely stay the same, or the likely changes, such as sending a child to live with relatives, will make the score higher. Assets accumulated during better times will remain in the household unless they are sold to cope with the shock. Housing quality will not change unless the household moves to cheaper housing to cope with the shock.

Other models have been shown to perform better than PMT at identifying transient food insecurity. Schnitzer (2019) studied the precision and relative performance of PMT and Household Economic Analysis (HEA) for identifying the poor and vulnerable in Niger. Her findings show that given