Its Drivers

How to Harness the Power of Data and Inference | 421

makers and social workers understand why particular households are deemed eligible or not; figure 6.7 provides an example. The figure presents a situation in which a particular household has been scored in the broader income distribution and relative to the eligibility threshold and which of the household’s characteristics are driving this, accounting for demographics, employment, access to services, and assets. This type of intuitive visualization tool can be useful for PMT in general and machine learning models in particular.

Nonetheless, machine learning–driven models and new variables are not a targeting panacea. Significant errors remain even when they are combined; when more labor income is directly measured, means testing and HMT offer more accurate eligibility determination. The Areias et al. (forthcoming) and Noriega-Campero et al. (2020) results show that while machine learning models may offer modest improvements when applied to standard data, it is the incorporation of new data into the models that is likely to drive improvements in accuracy. At the same time, significant errors remain. In Costa Rica, using machine learning and new data improved the inclusion and exclusion errors from around 30 to 24 percent; this is significant but not enough to mollify critics of PMT-based targeting. In Colombia, the inclusion and exclusion errors improved from around 38 to 28 percent, an even larger improvement but still leaving a significant fraction of the population misidentified. Improved PMT is still not a substitute for direct measurement of most or all income nor for interoperability and data integration.

Figure 6.7 Visualization Example of a Household’s PMT Score and Its Drivers

Income groups

EP P V NP Goods

Housing and services Occupation

Household

Sociodemographic

Household in poverty (median)

Source: Carrillo et al. 2021. Note: EP = extreme poverty; NP = nonpoor; P = poverty; PMT = proxy means test; V = vulnerable.