I.
INTRODUCTION
1.1
Evaluability is a simple concept: it is the ability of an intervention to demonstrate in measurable terms the results it intends to deliver. Measuring results is necessary for a project or program to be managed by results, and it is also necessary for institutional learning regarding development effectiveness.
1.2
In 2001 the Office of Evaluation and Oversight (OVE) developed a method for assessing evaluability and applied that tool to all of the projects approved that year. A similar exercise using the same tool was completed in 2005. In designing its evaluability instrument, OVE reviewed and adapted existing methodologies for assessing evaluability 1 . The method used by OVE is an assessment based on independent peer reviews, which relies on the expert findings of reviewers to assess proposals in different dimensions of evaluability 2 . OVE’s protocol is similar to methodologies applied in the qualitative review of funding proposals, as well as those applied more broadly in the quality review of projects’ design and analytical work 3 .
1.3
When designing the evaluability instrument, OVE adhered to overarching principles associated with the review of analytical work 4 . These include Accountability, Quality, Inter-subjectivity, and Independence. To ensure Accountability one person is responsible for the exercise implementation and for the final outcome, including the quality of the peer reviews, the documentation of findings, and the reporting of findings. Quality is assured by employing a twostep assessment in which an in-depth assessment of loan proposals is complemented with a quantitative scoring instrument based on a multinomial rating scale. The use of Inter-subjectivity allows for the richness of individual expert reviews, while at the same time minimizing reviewer bias by employing a panel of reviewers. Lastly, Independence of the exercise is assured by utilizing reviewers who were not involved in project preparation, and by preserving their anonymity.
1.4
The evaluability analysis includes a set of formal dimensions, such as the identification of appropriate output and outcome indicators, baselines, and an adequate monitoring and evaluation strategy. It also includes substantive dimensions—characteristics that define the substance of the intervention itself, which are necessary for the proper identification of the formal evaluation dimensions. These include the identification of a problem through a diagnosis, the identification of what is hoped to be achieved through the definition of objectives, the identification of how this is to be achieved through its intervention logic, and the identification of risks that could attenuate effectiveness.
1.5
The review process includes the following steps. First, OVE’s Deputy Director was assigned responsibility for the entire evaluability review exercise, aided by an Evaluability panel comprised of three staff members. Second, an evaluability team was assigned to each project reviewed. This team consisted of a team leader, 1