FURTHER READING

Tailored approaches to evaluation design in fragile settings

In fragile settings, including those affected by conflict and violence, the approach and methodology to design an evaluation needs to be tailored to specific local circumstances. Moreover, evaluation – like development aid itself – can unintentionally exacerbate tensions in ways that can negatively affect people and institutions if care is not taken in designing and approaching it with conflict sensitivity. This argument is at the centre of ‘Evaluation in Fragility, Conflict and Violence’, a book published by the International Development Evaluation Association, and prepared under the leadership of Hur Hassnain, Senior Evaluation Advisor at the European Commission. The book was launched during two live events, on June 29th and July 8th.

Drawing on the insights and personal experiences of many evaluation professionals from around the globe, the book presents real time case examples and practical guidance that show a diversity of perspectives and experiences on sensitive issues when measuring change in unpredictable, complex, and violent situations. Mitigating these challenges can entail significant effort, and can sometimes require redefining the direction, purpose, or scope of the evaluation.

Fabrizio Felloni, Deputy Director of IOE, provided introductory comments during the launch event on July 8th, alongside Gonzalo Hernández Licona, Director of the Multidimensional Poverty Network at the University of Oxford, and member of IOE’s Evaluation Advisory Panel. Mr. Felloni’s remarks centred on how the principles and approaches presented in the book could be applied in a real evaluation scenario in rural development, and what could be the challenges and the way to overcome them.

Simona Somma, Evaluation Officer, who is among the co-authors of this volume, joined other evaluation experts for a panel discussion during the launch event on June 29th. To emphasize the importance of context-specific evaluation approaches, in the book Somma shared her experiences in designing, piloting and implementing surveys and focus group discussions in Yemen and in fragile areas of India and Mozambique. Somma also contributed to the discussion on the evaluation learning function, with special focus on the dissemination of evaluation results to beneficiaries.

To learn more about Simona’s contribution to the book, and her motivation for getting involved in this ambitious initiative, Independent Magazine caught up with IOE’s Evaluation Officer.

Good morning, Simona.

Good morning, Alexander

Simona Somma

Congratulations on the publishing of this important volume; a complex effort no doubt. What pushed you to accept the challenge of collaborating on the preparation of this ‘how to’ guide to conduct evaluation in fragile settings?

During conflicts and pandemics and in fragile circumstances in general, monitoring and evaluation matter more than ever, and yet very often they are disregarded or perceived as a burden rather than as a way to build evidence and ensure accountability and learning. This is why I felt it was important and timely to contribute my knowledge of evaluation approaches, methods and processes, including in conflict affected contexts, to the preparation of this book.

I assume that the importance of designing a valid approach to evaluations in fragile settings – which you stress – is directly proportionate to the difficulty of doing so successfully, correct?

Yes, precisely. Especially in countries affected by conflicts and crisis, an in-depth analysis of the context is of the utmost importance in the design phase. Likewise, in these situations, the selection of appropriate evaluation methods and tools (including the innovative ones) for primary data collection become even more critical and fundamental for the robustness and validity of the evaluation.

Does this mean that ready-made, cookie-cutter methodological approaches will not suffice in these circumstances?

This is a key point that I felt strongly about addressing in the book. When you design an evaluation to be done in fragile settings, you cannot adopt the same approach and methodology that you would have used in a ‘normal’ context. You need to tailor the approach to the specific circumstances in which the evaluation will be conducted. Elaborating this in the book was something that required a lot of work among the co-authors, because we wanted to make sure to bring to the table something meaningful and useful to evaluation practitioners.

In addition to being involved in the chapters that focus on evaluation design, did you also contribute to those that address the implementation steps?

I am pleased to say that I was able to capitalize on my experience not only in designing, but also in piloting and implementing surveys and focus group discussions in Yemen and in fragile areas of India and Mozambique. Moreover, I worked on the part of the book that concerns the organization of the field missions and the collection of data, as well as the part that is entitled ‘close the evaluation learning loop’ – a step in the evaluation process very close to my heart as it concerns the dissemination of the evaluation results to the beneficiaries.

Looking now at the volume, are you satisfied with how the final product has shaped out?

Yes, very much, especially since this book is the result of a collaborative effort, to which many evaluation experts provided substantive inputs. This endeavour entailed coordination work and frequent online meetings, especially in the final stages of the drafting. Despite the challenges of embedding in the final product and editing so many different inputs and perspectives, this was for me an extremely enriching learning experience and an excellent example of collaboration across institutions and colleagues with different backgrounds.