It then presents an overview of recent trends in portfolio outcomes to assess if there have been any changes in quality and in the patterns of sector and thematic distribution of the portfolio. The chapter then focuses on incentives, drawing on data from survey results and manager interviews as well as analysis of budgetary incentives. The chapter then presents a brief review of quality assurance mechanisms and accountabilities for results. Based on these findings, the evaluation draws conclusions regarding the efficiency and effectiveness of the matrix system in achieving the related intermediate outcome—“Incentives and accountability ensure teamwork and high quality country dialogue and client services.”
Portfolio Trends in Quality Outcomes of Bank-financed operations as measured by year of exit from the portfolio improved significantly over the past two decades, rising from less than two-thirds satisfactory in the early 1990s to a peak of nearly 80 percent satisfactory in the mid-2000s, as shown in Figure 4.1a. Reporting outcomes by year of exit is a lagging indicator of quality at entry, reflecting performance of projects prepared an average of seven years earlier and affected in part by Bank supervision. Of the lending operations exiting the portfolio in FY01-10, 76 percent of operations were rated moderately satisfactory or better, compared to 68 percent for those exiting in FY1991-2000, a difference that is statistically significant. In more recent years, quality has become a concern. After the long upward trend in outcomes, the trend has turned decidedly negative. The share of Banksupported operations exiting the portfolio in FY08-10 whose outcomes were moderately satisfactory or better was 74 percent, compared to 79 percent in FY05-07, a decline that is statistically significant at 95 percent confidence.1 This decline is more evident for investment projects, which are typically prepared six to seven years earlier, than for development policy operations. Indeed, partial results for FY102 were the worst in an individual year since FY99 (although year-to-year variations should be interpreted with caution). This suggests that the Bank’s quality assurance system deserves a closer look. For project outcomes by year of approval, the most significant improvement in the share of Bank-financed operations with moderately satisfactory or better outcomes occurred before the introduction of the matrix. The share of operations with outcomes rated moderately satisfactory or better was 74 percent for those approved in FY91-96, and 76 percent for those approved in FY1997-2005; the difference is not statistically significant. The slight gain for projects approved during the early years of the matrix (FY1998-2001) was lost by projects approved after FY01. A downward trend commenced after that peak (Figure 4.1b). Using a three-year moving average, the share of operations approved in FY01-03 rated moderately satisfactory or better was 79 percent, compared to 70 percent among operations approved during FY04-05, a drop that is statistically significant with 95 percent confidence. This trend needs to be monitored carefully.
Delivering Quality Services
69