Investment Lending Quality and Outcomes by Fiscal Year of Approval
90% 85%
Quality of supervision
80% 75% Outcomes
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
1996
1995
1994
1993
1992
1991
50%
2000
Quality at entry
55%
1999
60%
1998
65%
1997
Introduction of matrix system
70%
1990
Share of moderately satisfactory or better (3�yr centered avg)
Figure 4.2
Source: IEG evaluations.
Three indicators offer additional clues to the quality of the portfolio, based on the information made available in real time to task teams and managers. Without candid information on the likelihood an operation will meet its objectives, Bank managers and government implementing agencies cannot allocate attention and resources to where they are needed. The net disconnect compares self-ratings to ultimate ratings in independent evaluations: it is the difference between the percentage of projects rated as moderately unsatisfactory or worse by IEG and the percentage rated as such by the Regions in the final Implementation Status and Results report. The Bank has never met its target of 5 percent or lower. The disconnect has been at least 14 percent in every year since FY07. The realism index compares self-ratings to recent history: it is the ratio of the percentage of actual problem projects in the active portfolio and the percentage of operations with moderately unsatisfactory or worse outcomes as rated by IEG for the most recent completions (1,000 evaluations), on a rolling basis. The Bank has set a target of 70 percent for its realism, and despite improvement over the past decade, the target has only been met once (in FY09) in the past 10 years IEG ratings of the quality of monitoring and evaluation (M&E) in operations assess the extent to which M&E systems were appropriately designed, implemented, and utilized and serve as a measure of the real-time feedback available for project management. These ratings were fully introduced for operations exiting the portfolio beginning in FY05. A high quality rating would indicate an M&E system with a strong influence on project performance. M&E systems have been weak during the entire period for which data are available, with just 25 percent rated of substantial or high quality among operations closing in
Delivering Quality Services
71