TRIAD COMES FULL CIRCLE
Member states comprising IFAD’s governance play a pivotal role in bolstering the utility of evaluations, which ensure that the organization performs optimally, and in safeguarding the full intellectual, methodological and communicative independence of IOE. In this regard, while the first edition of Independent Magazine presented IOE’s new strategic vision and the second captured the perspectives of senior management vis-à-vis the role of evaluation in embedding reflective milestones in the IFAD development journey, this third edition completes the triad by discussing the governance stream.
IFAD’s Executive Board and Evaluation Committee play a pivotal oversight role. Review of independent evaluation reports, follow-up with IFAD senior management vis-à-vis implementation of report recommendations, approval of IOE workplan and budget, and appointment of the IOE Director – these are but some of the direct responsibilities and sole prerogatives of IFAD’s member states, which help entrench the learning, transparency and accountability themes threaded in the first two editions of Independent Magazine.
Undoubtedly, engagement between IOE and IFAD member states is essential. The Director and staff of IOE have the authority to communicate and interact directly with members of the Executive Board, with in-country partners, and with others outside the Fund, as appropriate, for the undertaking of evaluations. Building on this, IOE’s new vision and strategic direction re-enhances the Office’s drive to engage with all member state groups in an ongoing and systematic fashion.
To ensure that we present an honest, balanced and inclusive picture on this important topic, Independent Magazine has drawn insights from esteemed representatives of all member state groups of the Executive Board. We are most grateful to Dr Yaya Adisa Olaitan Olaniran, Permanent Representative of Nigeria to Rome-based UN agencies; Ms Michelle Winthrop, Development Policy Director, Department of Foreign Affairs and Trade, Ireland; Dr Bommakanti Rajender, Minsiter (Agriculture) and Alternate Permanent Representative of India to Rome-based UN agencies; and Ms Sandra Paola Ramirez Valenzuela, First Secretary, Mulitlateral Affairs, Rome-based UN agencies, Mexico, for generously taking the time to sit down with us, and for kindly sharing their experiences, insights and perspectives.
Excellencies, ladies and gentlemen, good morning.
Good morning, Alexander.
Could you briefly describe your institutional role as it relates to oversight and evaluation?
Ms Michelle Winthrop
In terms of oversight and evaluation, it is my responsibility to ensure that IFAD delivers on Ireland’s stated objective of ‘reaching the furthest behind first’. This is the principle that guides all of Ireland’s oversees development assistance. IFAD has a good track record in this regard, and we see the evaluation function as being a really important element in ensuring that IFAD continues to do so. In this regard, IOE is uniquely positioned to ensure that IFAD is equipped with the necessary information and the evidence, whilst also to provide the first early warning flags that signal if IFAD institutionally starts to stray away. That is why Ireland put itself forward to be on the Evaluation Committee (EC).
Dr Yaya Adisa Olaitan Olaniran
The oversight function that I am involved in is in the context of the Evaluation Committee. As members of the Committee, we review IOE’s proposed work plan, budget, various products, such as the IOE multi-year strategy, IFAD evaluation policy and evaluation manual. Upon review, we make suggestions, we discuss with IOE, and eventually recommend the documents to the Executive Board (EB) for approval. Furthermore, as part of the Evaluation Committee, it is part of our responsibility to ensure the independency of IOE. We ensure that IOE can look at what it wants to look at, without interference by management. This includes ensuring that management implements evaluation recommendations as best it can. Where we feel that management is not proactive enough, we urge them to act. In the event of complex situations, we invite IOE and management to sit down and iron out the problems.
Ms Sandra Paola Ramirez
I am the delegate in charge of IFAD affairs at the Permanent Mission of Mexico. As such, I am in charge of preparing Mexico’s participation at IFAD’s Evaluation Committee’s and Executive Board’s meetings. On the one hand, I study and analyse the evaluations produced by the Independent Office of Evaluation (IOE). I assess their pertinence and the way its results could help IFAD to be more efficient and effective. On the other hand, I also have to monitor that new policies and decisions always respect the independency of the IOE and that IFAD management utilizes IOE deliverables to enhance its work.
Dr Bommakanti Rajender
I have been in Rome for the last three years, including as a member of the IFAD Evaluation Committee. We have been members of the Executive Board since its inception, and India’s Governor is a member of the Governing Council. These are our institutional roles.
Could you kindly explain the processes and dynamics of the evaluation function within the structure of the IFAD governance architecture?
Ms Michelle Winthrop
At its most basic level, the evaluation function provides the raw material for strategic decision-making by the IFAD Executive Board (EB). As EB members, we need to use this information to encourage management when it makes the right decisions, and equally to challenge management when we see decisions that are contrary to what the evidence is telling us. The role of the EC is to be an intermediary of that evidence, of that raw material. The EC makes sure that Board members see what they need to, filtering the evidence and steering the work of IOE based on the EB’s needs. It’s a supply and demand relationship, with the commodity being evidence. EC members are the traders making transactions in both directions, helping to articulate both supply and demand. I think, in a more general sense, we also set the tone for how IFAD management responds to IOE evaluations, and how IOE is empowered to challenge management. This is important. We need to help ensure that this relationship is a constructive one. It’s not always a friendly one – and that’s OK –, but it needs to be a constructive working relationship. We have a responsibility to make sure that this is the case. If we see management not responding appropriately, we need to call them out. Likewise, if we see IOE being a little unfair or subjective – which, by the way, never happens –, it’s our job to call that out. I am very struck by the fact that IOE has become a victim of its own success, during the last year. By generating high quality evaluations, and by ensuring good processes, we are seeing a big surge in demand. Everyone wants a nice IOE evaluation done on their area of work, country or region. This means that at every turn, IOE management have a set of challenging choices to make. The EB and EC need to help equip IOE to make the right choices about which evaluations to prioritize, and how to maximise efficiencies across multiple evaluations.
Dr Yaya Adisa Olaitan Olaniran
The IFAD governance architecture is set-up to ensure that the Fund is able to help the poorest of the poor. The programme management department designs programmes to this end. These programmes are discussed with national governments, including on the basis of an analysis of past projects. This is where IOE comes into play. IOE’s analysis dovetails the design of country strategy programmes, including by capturing the perspectives of government officials on-the-ground vis-à-vis the issues that they see affecting the implementation of past programmes. For this reason, IOE’s evaluations are very, very important in order to get the right results. Equally important are IOE’s corporate level evaluations. For example, issues like organizational decentralization, and the effectiveness of this process vis-à-vis staffing levels, government dialogue and outputs are issues that IOE is and will need to look at closely.
Ms Sandra Paola Ramirez
IOE is the one in charge of promoting accountability and learning through independent, credible and useful evaluations of IFAD’s work. In this regard, its evaluations should contribute to promote accountability and learning to produce more efficient operations to reduce rural poverty. The Evaluation Committee is a sub-committee of the Executive Board which performs reviews of selected evaluation issues, seeking to enhance the effectiveness of the Executive Board.
Dr Bommakanti Rajender
As per IFAD’s mandate, the main function of evaluation is to look at whether the organization is meeting its goals and objectives. The EB and EC play a pivotal oversight role in this regard. The EB discusses proposed country strategies and programmes, and does so also in light of past analysis carried out by IOE. In terms of the dynamics, there is a distinction between programme and donor countries. Donor countries contribute their tax payers’ money to the Fund, and thus want to understand the results and impacts that their pledges have made with reference to the targets set by IFAD. Evaluation obviously plays a big role in this context. Regarding the schemes, there are institutional mechanisms to ensure that members of the EB have the opportunity to witness first hand select programmes on the ground. These missions also relate to the work carried out by IOE.
Do you believe that the evaluation function helps to ensure that IFAD performs optimally as an organization? If so, why and to what extent?
Ms Michelle Winthrop
Absolutely yes, it does. The management response is the measure of whether IOE evaluations have been applied. Often, the quality of the management response is determined by the relevance of the IOE evaluation. Those two metrics, relevance and application, are really important. I think we have seen a real surge in the quality and relevance of IOE evaluations. There are two examples that I would point to, which are indicative of the importance of this relationship. Nutrition is one of the mainstreaming themes that Ireland believes in very strongly. At this stage, IOE has a very important role in helping to generate the evidence to help us see if IFAD management is being sincere in its commitment to embed nutrition across its work. The other piece is IFAD’s work in fragile and conflict-affected states. We are seeing some really interesting evidence emerging from IOE about how effective IFAD actually is in fragile states, and some of the real financial costs of working in these contexts. This is essential information for us. We would be lost without the evaluation function. In the absence of other forms of triangulation, which we often cannot find in these complex environments, we are completely reliant on the IOE evidence to guide our discussions about what IFAD should be doing, where and with what resources.
Dr Yaya Adisa Olaitan Olaniran
Evaluation is indicative of IFAD’s ability to function. However, many things are necessary for IFAD to function optimally, things which are beyond the control of IOE. What IOE can do, is to point out what these things are, find reasons for shortcomings vis-à-vis envisaged results, and recommend actions for improvement. One aspect of IOE’s work that I personally appreciate is its attention to understanding why projects may be delayed. This is usually due to lengthy internal government processes. In more recent times, through dialogue with management, IOE has been able to capture issues also during the programme design stage. Results may not be perfect, but collaboration between and IOE and management has certainly improved programme design and results. Moreover, it is the learning loop that marks the difference between a successful project and one that fails. I believe that the interaction among all IFAD departments and IOE are central to IFAD’s added value compared to other IFIs.
Ms Sandra Paola Ramirez
Yes, it does contribute to perform in an effective and efficient way. To what extend depends on how much recommendations are taken into account and implemented.
Dr Bommakanti Rajender
Absolutely, there is no denying that evaluation is the sine qua non of any organization. Evaluation is like a doctor, who is there to diagnose whether your body is functioning properly or not, and to give remedial by prescribing medicines. We may not like what the doctor tells us, but we have to face the reality if we want to continue living. Likewise, an organization without evaluation will become ‘sick’. There will be a lack of direction; it will be a ship that is going without a radar. Evaluation thus plays a fundamental role in correcting problems, where necessary. For this reason, evaluation gives member states confidence in the work that IFAD is carrying out. Evaluation also shows us where the successes are, and help us identify those initiatives that can and should be scaled-up and replicated in other countries. IOE is doing a great job in this regard, I am convinced – this is my experience over the past three years.
Do you believe that member states comprising IFAD’s governance structure play a pivotal role in bolstering the utility of evaluations? If so, what are the key elements that help ensure that this is the case?
Ms Michelle Winthrop
We, the member states, should always endeavor to have the views that we express at the EB supported by evidence. Using that evidence is probably the most useful thing that we can do. It is good governance, and it is also a means to reinforce the importance of the evaluation function institutionally, as both an accountability mechanism and as a learning tool. As EC members, we also have a role in championing evaluation, encouraging discussions of them and accessibility and communications. In this regard, I think it’s important to remember that EB members also have an onus to not politicize the results of evaluations, where evaluations can be slightly critical of government performance or commitment. I know that IOE works really hard on objectivity and independence. Personally, I have a lot of faith in the independence of the evaluation function. To be honest with ourselves, we should be able – as an Executive Board – to take bad news on the chin and be mature about it, and process those lessons for the broader benefit of IFAD.
Dr Yaya Adisa Olaitan Olaniran
I find the IFAD Board unique. All representatives come from different backgrounds, which is something that helps us to forge a consensus on the best way forward. However difficult a problem may appear, with consultation and discussion we always get things done and come to an amicable agreement. In this context, member states do their best to ensure that recommendations given by IOE are always followed. Moreover, we must keep in mind that one thing is getting results, something very different is getting the right impacts. We may get results, but if these are not making a positive impact on the lives of people, then these are not the results that we should be looking for. For this reason, it is so important to ensure that members of the EB join IFAD management and IOE in going to the field to see where money is being spent, and to interact with the people that are doing the work in the field, to understand what their thinking is.
Ms Sandra Paola Ramirez
Yes, the IOE responds to member states requests. So, if member states identify the need of assessing a particular policy, aspect or performance, IOE would work on a deliverable that satisfies such a request. The key elements to consider in such a case are the identification of evaluation needs and the request of member states to the IOE.
Dr Bommakanti Rajender
Engagement between IOE and IFAD member states is essential. This has improved over the years, and there is scope for further growth. We must recognize that there can be significant political changes in countries, every two-to-four years. This may result in turnaround of government staff deployed internationally, which in turn has consequences for institutional memory. Understanding the organizational dynamics of IFAD, and the role that evaluation plays within it, takes time. For this reason, there should be institutional mechanisms to help newly appointed staff to come to grips with the evaluation function. Visual materials and briefing packages are what is needed. Traditional approaches are not sufficient. In order to successfully compete for the attention of member states, the information provided will need to be useful, visually attractive and easy to digest.
Looking ahead, what do you see as being the main opportunities for evaluation to further permeate an evidence-based learning culture across IFAD, and for this culture to trigger life-changing impacts for the organization’s stakeholders in the field?
Ms Michelle Winthrop
The accessibility of findings is really important. Getting the balance right between presenting the rigour of the process and demonstrating all the amazing data gathered, on the one hand, and distilling this down to key lessons that can help guide decisions, on the other, is a really important part of the evaluation function. I know that Indran has invested a huge effort in comms, including the launch of the new website, to ensure that evaluations are as easily accessible as possible. This is the most important investment, really. The decentralization push within IFAD is something that we are supportive of, and we need to find ways to make sure that lessons are embedded in operations in the field, and communicated effectively to IFAD staff in the field. Outreach and dissemination must be done in a way that reflects the needs of outposted staff, to foster a reflective culture on evaluation findings. In addition, IFAD has a very important role to play in helping to build a foundation of good data around rural and agricultural development, particularly in Africa. I would like to see – with the limited resources IOE has – more work to foster community practice, so that it’s not just the people sitting in IFAD offices who are reflecting on the lessons, but also the broader set of stakeholders, many of whom IFAD will be working with ten years down the line on different projects. Lastly, the recent evaluation of government performance was a really good example of how a large, highly complex evaluation can be distilled into some really compelling lessons. Despite our respective experience and intellectual capabilities, we are all being bombarded on a regular basis with lots of information. For this reason, I like to say that it is possible to be both rigorous and compelling – that is how you embed institutional lessons and change. This work that IOE is doing around good communications, synthesising in different ways, coming up with snappy solutions to present findings, it is all extremely positive as far as I’m concerned.
Dr Yaya Adisa Olaitan Olaniran
There is no crystal ball, but I believe a lot has been done in the last 5-6 years in IFAD that has given us a new understanding of the problems we are grappling with. The governance structure of countries affects what IFAD is able to do. It is easy for governments to say “let’s look at what results we can get over the next four-to-five years working with IFAD”. However, the truth is that the impact of IFAD interventions cannot be really felt in four years. So, governments must not see programmes through the lens of their office’s tenure, but rather give all the support necessary to achieve long-term impacts. Furthermore, with the ODA shrinking, it is important for IFAD to show countries the value-for-money of its intervention, and to do so by demonstrating the possibility of scaling-up the Fund’s interventions in a sustainable manner. These are the areas where IFAD needs to focus its efforts, and where IOE needs to increasingly focus its attention.
Ms Sandra Paola Ramirez
The evaluation of the implementation and impact of recently adopted policies is a key opportunity. Also, the decentralization process in all its aspects would require constant evaluation in order to monitor its implementation and impact on the ground.
Dr Bommakanti Rajender
In my experience, seeing is believing – especially when we are talking about the use of tax payers’ money, and the impact of these resources on people’s lives and livelihoods. This needs to be evaluated properly. Field visits are very important in this regard, not only for evaluation specialists, but also for member state representatives. This is a trend that should increase. The work carried out by IOE should continue to support these field visits. We need to see for ourselves where the bottlenecks are in order to fully appreciate how these relate to evaluation recommendations. In turn, these recommendations must increasingly be backed by a sound, robust and scientific methodology, which must be grounded in the evidence-based and context-specific nature of evaluation work. One-size-fits-all solutions are not credible. While generic recommendations may be easier to ‘digest’, and people may not always appreciate very specific recommendations, it is this level specificity that will increasingly give value and credibility to the evaluation function.
