480 | Revisiting Targeting in Social Assistance
happen by chance. The author proposes testing the chance that participation in the program occurs by chance instead of the method. The agreement coefficient varies from −1 to 1, and the value 1 indicates perfect agreement while 0 is the expected value when agreement is purely by chance. The Landis and Koch (1977) table helps to interpret the agreement statistic as follows: a coefficient less than 0.20 means a poor targeting outcome; between 0.21 and 0.40, a fair targeting outcome; between 0.41 and 0.60, a moderate targeting outcome; and greater than 0.61, good performance, as presented in annex 7A. In the 10-person economy example earlier in this section, the estimation for the benchmark poverty line showed moderate performance for scenarios 1 to 3 and poor performance for scenario 4 (table 7.8). For the poorest 20 percent, scenario 1 performed well and scenario 3 performed poorly, as expected, but for scenario 2, the agreement coefficient is estimated at 0.375, meaning a fair performance. Lindert, Skoufias, and Shapiro (2006) bring in the value of the transfer while still basing performance measurement on only inclusion and exclusion errors. The rationale behind their case is that a program can be the following: • Effective in absolute terms because benefits reach a significant share of the desired population, but ineffective in relative terms since the
Table 7.8 Benchmarking the Agreement Coefficient against the Poverty Line Selected as participant
Not selected
Total
Poor
Scenarios 1–3
2
2
4
Nonpoor
0
6
6
Total
2
8
10
Targeting success rate (TSR) equals (2 + 6) / 10 = 0.8 (80%) Joint probability equals {(2 × 4) / (10 × 10) + (8 × 6) / (10 × 10)} = {0.8 + 0.48} = 0.56 The agreement coefficient equals {(0.8 − 0.56) / (1 − 0.56)} = 0.54 Selected as participant
Not selected
Total
Poor
Scenario 4
4
0
4
Nonpoor
6
0
6
Total
10
0
10
Targeting success rate (TSR) equals (4 + 0) / 10 = 0.4 (40%) Joint probability equals {(10 × 4) / (10 × 10) + (0 × 6) / (10 × 10)} = {0.4 + 0} = 0.4 The agreement coefficient equals {(0.4 − 0.4)/(1 − 0.4)} = 0 Source: Original compilation for this publication.