3. MONITORING SPSP UNDER ROM - CONCEPTUAL ISSUES 3.1. The ROM system for monitoring SPSPs
The European Commission is concerned with whether an SPSP is contributing successfully to a Sector Programme (SP). A well-designed SPSP does not necessarily guarantee a good SP (and the other way round). Monitoring of the SPSP under ROM will help in answering this question, but such monitoring of the SPSP should not lead to an additional SP monitoring exercise carried out by the Commission. That would contradict with the objectives of the Paris Declaration as regards harmonisation and alignment (see box at bottom of this page). Europekd decided therefore, at the end of 2005, to extend the existing ROM system, so far applied for classic projects only, towards monitoring SPSPs. Using ROM for monitoring SPSPs should provide reliable, comprehensive and comparable information to both the Commission and the partner governments, complementing the information provided in SP progress reports, joint review reports and tranche release assessment reports. The ROM system should be able to capture the more qualitative aspects of an SPSP and the sector programme it is contributing to, i.e. reform processes, policy context and changes, support to decentralisation processes, progress in terms of harmonisation and alignment, predictability of funding, coordination among donors, etc. Care should be taken that the ROM system does not add undue transaction costs for partner governments and other donors. Extensive involvement of the partner government and other donors in monitoring the SPSP under ROM should therefore be avoided. Only when and where strictly necessary additional information, data and views could be collected at the level of the partner government and other donors. In cases where monitoring of the SP includes an Annual Review, the monitoring of the SPSP should preferably be carried out during or soon after that Annual Review (see also section 3.3).
DRAFT FINAL Guidelines for Monitors to assess Commission's Sector Policy Support Programmes under Results Oriented Monitoring System
ROM of SPSPs can be considered as an EC-internal monitoring activity, executed by independent external experts from the Consortia executing the ROM for SPSPs on behalf of their contracting authority. The ROM system is meant to complement the internal monitoring carried out by the EC Delegation itself, by providing additional information and insights. As already indicated in section 1.2 the ROM methodology for monitoring SPSPs differs from ROM of classic projects. Emphasis will be laid on analysis of existing information produced by the partner country, the Delegation and other donors, in order not to intervene with the harmonisation, alignment and ownership principles of the SP and the Paris Declaration. Another difference with monitoring projects is the greater complexity (in terms of policies, strategies and institutions involved) and the larger financial size of SPs and SPSPs compared to projects. ROM of SPSPs will consist of two phases: a desk-based phase in Europe and a field phase carried out in the partner country. Whether or not the field phase will be carried out will be decided at the end of the desk phase. During the desk-based phase the monitor will analyse the contribution of the SPSP to the SP's performance on the basis of existing reports, complemented, where required, with Email correspondence with the staff from the Commission. In most cases the desk phase will be followed by a field phase, in order to get a complete, accurate and updated picture of the progress made with implementing the SPSP. During the field phase, the monitor will apply triangulation techmques in order to cross check and verify the hypotheses and findings of the desk-based phase. Contacting institutions and persons outside the EC Delegation will be closely coordinated with the Delegation staff. In some cases it might be decided to abstain from carrying out a field visit, for instance when the desk phase has already provided sufficient information or when the desk phase has revealed that the field phase will not provide much more additional information. Further details on the process of monitoring an SPSP are given in chapter four.
3.2. Using a BCS for monitoring SPSPs - uniformity and consistency Just as in the case of monitoring projects, the monitor of SPSPs will produce a Monitoring Report (MR), supported by a Background Conclusion Sheet (BCS). A specific BCS has been designed for monitoring SPSPs, which ensures uniformity and consistency in the monitoring approach and coverage (see annexe 4). In contrast to the BCS for classic projects, the BCS for SPSPs puts a lot of attention on capturing policy changes and sector reform processes, which are central to most SPSPs and sector programmes.
The monitor might observe a major difference between the quality of the SPSP and the quality of the SP, andlor h s assessment of the SP might differ considerably from the conclusions of the local monitoring of the SP. In those cases the monitor should add a note to the MR, in which helshe explains the background of these differences in assessment. Just like monitoring projects, monitoring of the SPSPs is focussed on the five DAC evaluation criteria (relevance, efficiency, effectiveness, impact and sustainability), which is reflected in the structure of the BCS. As regards each criterion a number of questions (issues) have been identified, whch are meant to help and steer the monitor in assessing the SP and the SPSP as regards that criterion. These lists of questions are neither exhaustive nor exclusive; other questions can be added depending on circumstances and the characteristics of the SP and the SPSP being monitored. Many results and outcomes of an SPSP are closely connected with results and outcomes of the SP. Thus, monitoring the SPSP implies monitoring the contribution of the SPSP to the design, programming and implementation of the SP. An important part of the monitoring of the SPSP will therefore rely on the findings of the country-led monitoring of the SP. Nevertheless, in presenting monitoring conclusions, a clear distinction has to be made between views as regards the SP and conclusions as regards the SPSP. A strict division is therefore made in the BCS for SPSPs between DRAFT FINAL Guidelines for Monitors to assess Commission's Sector Policy Support Programmes under Results Oriented Monitoring System
sets of questions referring to the SP and sets of questions referring to the SPSP. For the SPSP a conclusion is drawn per DAC evaluation criterion, summarised in a grade ranging from a (very good) to b (good), c (problems) and d (serious deficiencies). The SP is not graded, because that is not the aim of this monitoring system. In section 2.3 the seven Key Assessment Areas (KAA), as defined in the "Guidelines for EC Support to Sector Programmes", were presented. These areas have to be assessed during identification and formulation of the SPSP, in order to allow the EC to take a decision on whether or not to support the SP. It is also envisaged that these assessments would be updated regularly during implementation of the SPSP. These KAAs thus play a central role in decision making and monitoring of the SPSP and are therefore given substantial attention in the monitoring questions. This ROM system for SPSPs includes also a rapid appraisal of the extent to whch an SPSP contributes to achieving the objectives of the Paris Declaration. A special module thereto has been developed. Furthermore, various elements of the Paris Declaration (e.g. harmonisation, alignment, ownershp, programme based approaches, etc.) are integral part of the questions in the BCS. The funding modality of an SPSP may be either (i) sector budget support, (ii) pooled funding or (iii) support using EC procurement and grant award procedures. A combination of these modalities is also possible. T h s ROM system for SPSPs is designed in such a way that it can be used no matter whch one of the above mentioned funding modalities is used. However, in the case of particular projects designed for helping to prepare an SP and SPSP, such projects could possibly better be monitored on the basis of the ROM system for projects, notwithstanding the fact that such projects have already certain characteristics of an SPSP. A decision as regards which ROM system to be used has to be taken on a case by case basis.
Testing phase task: Review the text and monitoring questions of the draft BCS (see annexe 31, in particular as regards effective ROM and guaranteeing the uniformity of the ROM approach.
3.3. When could Monitoring of SPSPs under ROM take place? As far as possible, monitoring missions should be coordinated with and llnked to already planned sector reporting activities. In cases where the monitoring of the SP includes an Annual Review or any other major review, the monitoring of the SPSP under ROM could be carried out at the same time, or soon after it; depending on the preference of the Delegation in consultation with Headquarters in order to minimise additional transaction costs. If alignment to an Annual Review would not be possible, then an alignment to one of the monitoring activities of the Delegation for the SPSP could be considered (see section 2.4.3). It should be noted that, particularly in the early phases of an SP, the monitoring systems of the SP are not yet well established. In those cases the possibilities for aligning the ROM with local monitoring and for making use of the results of local monitoring of the SP will be limited. Appropriate solutions have to be sought on a case by case basis.
Testing phase task: verifv the appropriate timing of ROM Testing phase task: assess how long an SPSP should be running before ROM becomes meaningful? Is it sector dependent?
3.4.
What are the benefits of monitoring SPSPs under ROM?
ROM of SPSPs is expected to produce, inter alia, the following added value: Providing safeguards and checking as to whether key issues in SPSPs are receiving sufficient attention, including 'early warning' for timely remedial action; DRAFT FINAL Guidelines for Monitors to assess Commission's Sector Policy Support Programmes under Results Oriented Monitoring System