wrong. When that happens, the evaluation design is compromised and the analyst has to invoke additional assumptions and/or use statistical techniques to try to fix the broken design. Unfortunately, the findings from a broken design often depend on the particular assumptions and/or techniques used in the analysis. The resulting ambiguity can lead to disagreements among analysts and confusion among policy makers. Many times the findings from a broken design are simply ignored. Although many things can go wrong in an evaluation, experience suggests that careful attention to planning can reduce the chances for errors and omissions that lead to a broken design. The purpose of this guideline is to help evaluation designers plan ahead and be prepared for some of the most common problems that arise in the evaluation of active labor market programs. The next section begins with a background discussion of the common features of active labor market programs that need to be taken into account in any evaluation project. Section 3 then turns to a discussion of the basic elements in the two types of designs, including the design of data collection procedures. Section 4 discusses the critical importance of a careful monitoring system to oversee implementation (particularly of randomized evaluations), and the value of site-level contextual information. Section 5 presents a brief discussion of issues pertaining to the analysis and interpretation of findings from an evaluation. Section 6 presents an extended example of the application of the ideas to a “real� evaluation of an ALMP in Dominican Republic.
9