Independent Magazine - Issue 9, 2024

FROM EVALUATION PHOBIA TO EVALUATION EMBRACING:

STORY OF A MINDSET SHIFT

Recognizing the traditional tension between evaluators and evaluands, steeped in stereotypes, IOE has put in place a drive to perfect its communication craft, including by horning-in neuroscience-based principles. The result has been a mindset shift among stakeholders and evaluands alike. Co-creation and enhanced engagement are now a daily reality in the work of IOE, which maintains its full independence whilst thriving on milestone publications. The roots of these results run deep and date back to the beginning of 2021, when IOE embarked on its ‘revolutionary’ journey.

This approach has been one of the keys in the rise of appreciation towards IOE’s work, which has ensured a rapid surge in uptake on evaluation recommendations, as confirmed by the recent MOPAN Review. This trend echoes the words of the UN Secretary-General, António Guterres, who emphasized the importance of using high quality evaluative evidence to improve efforts on the ground in his message to the UNEG EvalWeek 2024.

The UNEG EvalWeek 2024 took place from 29 January to 2 February 2024, in Málaga. Hosted by the World Intellectual Property Organization (WIPO) and the United Nations Office on Drugs and Crime (UNODC), in collaboration with CIFAL Málaga. On 30 January, Dr Naidoo and Dr Alexander Voccia, Coordinator of the Evaluation Communication Unit, delivered a joint presentation in the seminar titled ‘What does it take to build a credible, independent evaluation function?’, which was chaired by Judit Jankovic, Senior Evaluation Specialist at the International Criminal Court.

Drawing from his over thirty-year long career, Dr Naidoo explained that there tends to be an assumption that evaluators can affect change and enhance programme quality. Unfortunately, that would appear to not be always the case. Cognizant of this, IOE’s new approach builds on a dynamic and engaging process of triangulation that is based on independence, credibility and utility. This has resulted in enhanced opportunities for dialogue and mutual learning throughout the evaluation process.

“Evidence, on its own, does not drive change. As a matter of fact, 70 per cent of initiatives that attempt to trigger change fail. Instead, organizations that develop initiatives geared towards changing mindsets are two times more likely to succeed in achieving the progress desired. We must be empathetic listeners. People must look forward to an evaluator coming, because we all share the common goal of making programmes better. Disagreement is also fine, it’s part of a healthy process of interaction”, clarified the IOE Director.

To spearhead this effort, IOE adopted a ground-breaking neuroscience-based approach to its communication efforts which resulted in the production of a pioneering online training, and a communication module within the IFAD Evaluation Manual, in addition to a video series knows as the ‘evaluation pills’, among other outputs. This undertaking is bearing notable fruits, as IOE has refined its interactions with Management, whilst continuing to engage in global dialogues and networks.

Dr Voccia explained that a large suite of innovative communication products provides enhanced visibility to these efforts and achievements. These include a state-of-the art fully independent website, and a first-of-a-kind online magazine, which have already reached over a hounded thousand users across all countries in the world, since their launch.

In addition to contributing experiences vis-àvis its state-of-the-art approach to communication, during EvalWeek, IOE also stimulated thought-provoking conversations on some of the core methodological issues pertaining to evaluations. On 30 January, Fabrizio Felloni, IOE Deputy Director, Dr Kouessi Maximin Kodjo, Lead Evaluation Officer, and Massiel Jiménez, Evaluation Research Analyst, hosted a session titled ‘The crisis of linear thinking - Combining theory of change and systems thinking in complex strategic and policy evaluations’, in which they also delivered a joint presentation.

Mr Felloni, Dr Kodjo and Ms Jiménez explained that a theory of change (ToC) does not necessarily have to be linear, despite the linear thinking that it normally entails. A linear ToC risks over-simplifying reality. As a result, it is inadequate for humanitarian programmes, due to the dynamic and unpredictable natures of crises, as well as for complex development programmes.

These considerations notwithstanding, a ToC has the merit of helping to understand intended results, causal links and main assumptions. Therefore, ToCs should not be abandoned. Instead, they should be enriched by incorporating additional frameworks, such as a system thinking approach that can help better capture and unpack complexity.

Mr Felloni provided further points of reflection in his presentation during the session titled ‘Evaluating Policy Influence’. In particular, the IOE Deputy Director used the example of IFAD’s support to the Central America Strategy for Territorial Development, approved in 2010 by the Summit of the Central America Integration System. In his presentation, Mr Felloni showed several steps of policy development that can be evaluated with limited resources through desk reviews, virtual interviews and brief in-country work. Instead, an assessment of socio-economic effects of a