criterion has an associate question that attempts to explain the meaning of the criterion. The following three standard evaluation criteria as set out in the UNDP Evaluation Policy were used. How effective has UNDP been in contributing to poverty reduction at the country level?
Effectiveness:
Efficiency: Has
UNDP made the best use of its resources in making this contribution? How successful was UNDP in promoting greater sustainability of the results to which it contributed?
Sustainability:
Each of the above criteria raises a number of issues that need to be clarified. First, the usual definition of efficiency, which relates to the efficiency of moving from inputs to outputs, is too projectoriented and difficult to ascertain in a broad thematic evaluation. Rather, the present evaluation will look at the efficiency with which UNDP used its resources by leveraging these resources for a greater contribution to poverty reduction. Second, the sustainability criterion does not relate to the sustainability of particular UNDP interventions per se, but to the sustainability of overall results, in this case a reduction in human poverty, to which they contribute. In addition, ‘relevance’ is sometimes used as an evaluation criterion but in the present case it was deemed to be redundant since reduction in human poverty would in any case be relevant across all UNDP programme countries. This is true even if alternative concepts of poverty dominate national policy-making or discourse (for example, social exclusion or inequality). ‘Relevance’, in terms of UNDP’s approach to supporting national poverty reduction efforts, does have a place in the present evaluation, but as a factor that can determine UNDP’s effectiveness rather than as a separate criterion in itself. Identifying explanatory factors. In examining UNDP’s performance by criterion, the evaluation tries to explain why UNDP has been successful or not. In so doing a number of factors were identified that could be used to explain UNDP’s performance. If the evaluation criteria indicate
4
the values of the organization in determining the ideal ends of its work (effective interventions that are efficient and contribute to sustainable results), the explanatory factors represent the means to achieve these ends. A set of factors has been identified following a basic review of issues taking into account the concerns raised in a variety of sources including the annual reviews of lessons from independent evaluations. These factors have been posed as evaluation questions. A small set of evaluation questions does not mean that the evaluation ignores other factors but simply that it ensures that the priority concerns of the organization and its key stakeholders are addressed. The list of factors that framed the data collection can be found in the evaluation framework in Annex 4. Methodological challenges and solutions. The evaluation was conducted in a highly complex environment and this by itself represented a major methodological challenge. In addition, the evaluation was subject to two other major challenges, both of which are well known but worth repeating here. First, the challenge concerning the aggregation of UNDP’s contribution in order to provide accountability to stakeholders: how can one hold UNDP accountable when its overall contribution is the sum of contributions through hundreds of projects and other activities spread around 177 countries and territories within more than 130 country and multi-country programmes, together with the activities of headquarter units and regional service centres? It is difficult to aggregate results from all of them so as to achieve an overall assessment for the purpose of holding UNDP accountable to its stakeholders beyond the country level. At the same time, it is difficult to undertake statistical generalization from a set of country examples. This is not to imply that UNDP’s accountability for contributing to the goals set out in its various planning instruments was ignored. In fact, accountability follows logically from assessment of learning. At the country level assessments were made according to UNDP’s contribution
C H A P T E R 1 . introd u ction