appendix
B
Methods for Evaluation of Public-Private Partnership Programs and Policies in Basic and Secondary Education Randomization and regression discontinuity regressions show the real magnitude of the effects of public-private partnership programs (the estimates are unbiased) under general assumptions. In general, randomized studies randomly assign people to treatment groups. For example, in the secondary school voucher program in Colombia, the number of people applying for the vouchers was larger than the number of places available. Since the program’s budget allocation was not sufficient to cover the demand for vouchers, the recipients of the vouchers were selected using a lottery, creating a treatment group (those selected in the lottery) and a control group (those not selected in the lottery). The two groups had, on average, similar observable and unobservable characteristics. Regression discontinuity analysis is typically applied when a program is allocated using a continuous variable. For instance, some programs use a means-tested index to select the target population. In this way, the program specifies that households that score below a certain cutoff point are eligible for the program and those above the cutoff point are not. In this case, the program’s impact can be assessed by dividing individuals into a treatment group, containing individuals who score just below the cutoff point, and a control group, containing individuals who score just above the cutoff point. The two groups are assumed to have very similar characteristics, with the only difference between them being their inclusion or exclusion from the program. Intuitively, for individuals, the cutoff point is almost a random lottery. An important limitation of this method is that it can assess a program’s impact on the
population close to the cutoff point but not on the general population. In other words, it is a local estimator. Instrumental variable and Heckman correction models produce correct, unbiased estimates under more stringent assumptions. Both methods require a variable with two traits. First, it must explain the decision of the school or student to participate in the program. Second, it cannot be correlated with any unobservable characteristic that explains the outcome of interest, such as test scores. This variable makes it possible to model participation in a program and, therefore, once selfselection is controlled for, it is possible to assess a program’s impact. The difficulty with these two methods is fi nding a valid instrumental variable. The difference in difference method compares beneficiaries and nonbeneficiaries before and after the program. Its key assumptions are that the trend in the outcome of interest before the intervention is equal for beneficiaries and nonbeneficiaries, and that all nonobservable variables that explain the outcome of interest are time-invariant. Propensity score-matching estimators take a slightly different approach. This method assumes that program participation can be fully explained by a large array of observable characteristics measured at a baseline. Based on this information, the treatment and control groups are constructed and their outcome measures compared. The biggest challenge in using both difference in difference and propensity score-matching is obtaining the large array of baseline data needed to ensure the statistical similarity of the two groups.
83