about the excessive focus on fiduciary and safeguard matters and relative inattention to other aspects of quality in regional review processes.
Quality Assurance Systems Need to be Strengthened In line with the recently observed decline in development outcomes, most managers believe the quality of the portfolio has not improved since the introduction of the matrix system. Half of the sector managers and country directors/managers interviewed for this evaluation said that quality of lending and AAA is not significantly better than during the pre-matrix period, possibly reflecting recent pressures on quality. Among the remaining managers interviewed, more think quality has worsened, particularly for AAA, rather than improved. This may be a reflection of the growing volume of NLTA, which has had weaker quality review procedures than ESW. The majority of staff whose Bank careers began prior to the matrix system also do not report a noticeable improvement in the quality of supervision and AAA, while 39 percent feel it has worsened. While these perceived changes may not be due to the matrix reforms, at the very least it indicates that quality assurance systems are under strain and have not prevented quality deterioration, particularly in recent years. Available tools for quality enhancement are not being used effectively by sector units. The two main tools available to sector units are independent peer reviews and QERs. Sector managers value QERs much more than peer reviews for improving quality. Peer reviews in regional operations are not well managed. For the most part, task team leaders select their own peer reviewers, sometimes even from their own unit. In contrast, peer reviewers in DEC and IEG are selected by managers to ensure high quality and global expertise. In regional operations, due to delivery deadlines, peer reviewers are aware that substantive criticisms at the decision meeting or at meetings of the Operations Committee—where operations receive internal approval to move forward with appraisal and to request Board approval—are neither welcomed nor likely to be addressed. QERs have the potential to be more effective because they are considered a nonthreatening venue to obtain high-quality technical advice. But they are now considered optional, they are often undertaken too late, and their use is uneven across Regions and sectors. More systematic and timely use of QERs would significantly enhance quality of lending operations. The potential of the matrix is not being fully exploited to enhance operational quality. In most Regions, the concept review meetings—where teams are authorized to prepare an operation—and decision meetings for investment projects and ESW are chaired by the country director. Development Policy Loan (DPL) review meetings are held at the Regional Operations Committee or Bank-wide Operations Committee. Only in East Asia and the Pacific have both sides of the regional matrix been involved: the country director chaired the concept review meeting to ensure strategic relevance and client demand for the lending or nonlending product, and the sector director chaired the decision meeting to ensure quality. This good practice could easily be replicated in other Regions but ironically, to harmonize procedures across Regions, it has been done away with, as of July 2011.
Delivering Quality Services
79