Operational Research

The Reproductive, Maternal and Neonatal Health Innovation Fund (RIF)

Policy brief June 2018

RIF (2013-2018) has been funded by the UK’s Department for International Development (DFID) and managed by Ethiopia’s Federal Ministry of Health (FMOH) with the assistance of a Technical Assistance Supplier (Pathfinder International).

Operational Research The Reproductive Maternal and Neonatal Health Innovation Fund (RIF) has aimed to improve the health outcomes of women, newborns and young people in the pastoralist areas of Ethiopia. With RIF funding, FMOH has administered grants to both government and non-governmental implementing partners. These implementing partners have been tasked with identifying innovative solutions to the barriers that prevent pastoralist populations (especially women and young people) from accessing reproductive, maternal and neonatal health (RMNH) services.

Learning by doing “… I had three things in mind: innovation, evidence and health in other sector polices. On evidence, we might have high impact interventions that work, but the important thing is to know how and why so we can replicate or scale them up elsewhere – that is why I was very pleased to see an OR component had been included in RIF. …We need to prioritise OR using our research organisations. ” (Dr Ephrem Tekle, Director, Maternal and Child Health Directorate) RIF has used an innovative funding modality to encourage tailored, flexible and innovative projects to respond effectively to different contexts and needs. Against a background of scarce evidence related to Ethiopia’s diverse pastoralist settings and cultures, RIF has also funded Operational Research (OR) to promote programme learning and contribute to the wider evidence base.

Through the OR component, RIF has sought to generate evidence on ‘how’ to better reach and serve women and girls, what works and what does not, and ‘for whom’, in order to inform both programme implementation and wider national and regional strategy, policy and planning.

2

From the beginning, the programme was designed to maximise ‘learning by doing’ at programme theory and practice levels, not only through OR, but also through a theory of change approach, technical guidance and documentation of lessons learnt.

Rather than discuss the individual findings from the studies, this Policy Brief is concerned with whether the learning process was successful or not, and what lessons emerged.

How did it function?

Local research organisations were invited to bid for OR funding through the same transparent and competitive process used for implementation grants, which resulted in the contracting of four universities/research centres to cover each pastoralist region (although not all were based in the region where the OR took place): • • • •

Afar: Mekelle University (Round 1 and Round 2 grants) Oromia: Jimma University (Round 1 grant) SNNP: Addis Ababa University (Round 2 grant) Somali: St. Pauls’ Hospital Millennium Medical College/Jijiga University (Round 2)

The RIF evaluation team was asked to provide technical and quality assurance support. In order not to compromise the independence of the M&E function, a separate OR Research Resource Team was established.

Challenges Conducting OR in pastoralist areas has not been without challenges, some of which resulted in additional costs. Both reaching and gaining the trust of communities in pastoralist areas posed particular challenges, requiring researchers to cover huge distances and spending time with clan/religious leaders to refine their approach. The teams also faced unexpected logistical constraints including drought and security issues. Round 2 grantees suffered from time constraints, as they received their grants only at the end of 2016. Nevertheless, a positive outcome of the process was that some universities managed to use their RIF OR grants as ‘seed funding’ to attract additional funds, and make RMNH a new research focus.

OR outputs The OR partners conducted over 20 studies in the pastoralist zones of Afar, Somali, Oromia and SNNP regions. The studies used mixed methods to support depth of analysis. Findings from ethnographic investigations and gender analyses provided rich data for adapting interventions to local context. Innovative methods included integrated Trials of Improved Practices (TIPs), design thinking and deviant (outliers) approaches. The studies were varied in size, with the largest having a sample size of over 2,600 women.

Overview of emerging themes The studies provide evidence and metrics from each region on the barriers and other social, economic and cultural factors affecting the utilisation of RMNH services in pastoralist areas. They tend to focus on women of reproductive age, leaving young people as an outstanding OR gap. While the studies often reinforce what has already been documented, they also uncover lesser known challenges. One such important challenge revealed is the contradictory attitude towards family planning among health providers in the Somali region, which has significant operational implications (these findings are discussed in a separate Policy Brief).

There is a strong thread throughout the studies on the importance of understanding the values and practices of local pastoralist cultures and the dynamics of community networks - including drivers of decision-making, gender relations and concepts of

3

health. These aspects were key in tailoring interventions to bridge communities and the formal health system. More than one study, for example, remarks that there is scope for more cultural adaptation to the pastoralist context of Maternity Waiting Homes to increse their acceptability and use (this topic is covered in a separate Policy Brief).

Within the overall pastoralist context, a study from SNNPR highlights that there is also an underlying need for better skills by health service planners and implementers in identifying the gender- and age-differentiated needs/priorities of different sections of the population (e.g. women, men, youth) and their implication for service delivery, as well as, crucially, a need for planners to be empowered to actually implement contextualised services.

OR and adaptive programming: did it work?

Timeliness and programme alignment The most significant lesson of the RIF OR process is the importance of appropriately sequencing OR studies and programme implementation – and the resource flows that come with them. The RIF OR findings are generally coming too late to be usefully applied within the programme.

The studies tend to focus on barriers to RMNH utilisation (i.e. ‘the problem’ in RIF’s Theory of Change) rather than actual interventions, and their findings have largely come at the end of the programme’s lifetime, during which around 20 sub-recipients have delivered interventions. Although the contribution of the studies to the broader evidence base is undeniable, by the time that the preliminary findings were shared at a meeting convened by the FMOH in early 2018 , the implementing sub-recipients were well beyond the ‘analysis of barriers’ stage. This has been an understandable cause for frustration.

Linkages between OR and implementing partners RIF evaluators have remarked on the existence of a lively dialogue between the programme leadership and academic partners on observing the principles of research relevance, partner engagement and research uptake.

The FMOH and M&E Supplier have played an important role in this, supporting exchanges between the OR researchers and the implementers. A key lesson from this is that it may take active facilitation by an intermediary to ensure that academic research remains relevant and supports applied programme learning and adaptation.

Matching research partners and implementers from the same type of environment (where possible) also adds credibility to the research, and facilitates both engagement and a shared sense of ownership.

Pastoralist Forums have been widely acknowledged as important forums for showcasing progress, sharing lessons, research data and findings, and addressing common challenges. They have also been an important vehicle for building programme identity, along with a unity of purpose across diverse sectors and partnerships. Although they can be costly and logistically challenging to convene, there remains a strong case for including such forums in future programmes of this type.

Dissemination Similarly, there is a case for an intermediary function to lead on documenting and sharing programme lessons, OR findings and promising practices (e.g. relating to the grant making mechanism, working with non governmental partners and provision of technical assistance support) in a timely manner (i.e. coinciding with key programme dates), in an appropriate format, and through suitable communication channels (the Pastoralist Forums being one key opportunity).

4

While publication in academic journals remains the gold standard for academic institutions (and indeed many of the RIF OR studies are in the process of being published), this process is lengthy and the format may not always be appropriate for communication to programme stakeholders.

Policy recommendations •

• •

•

•

•

In future programming, give more attention from the outset to the alignment and sequencing of OR studies with programme implementation, to ensure the findings remain relevant and useful for programme contexts.

OR studies could be useful for formative research and for testing sub-recipient innovations through focused M&E within an overarching research agenda.

Test the Promising Practices identified by the RIF evaluators (e.g. those relating to male involvement, provider attitudes and for scaling up family planning services) in and across different pastoralist settings, without the credibility of findings being compromised by time constraints.

Ensure the intervention-based OR studies are completed as designed so they produce good evidence for future action. If necessary, pair OR partners with new programmes and funding sources to build a ‘virtuous circle’ of context-specific research and adaptive programming.

Consider long-term collaborative working between Regional Health Bureaus, regional universities and contracted local NGOs to commission, research and implement evidence-based interventions.

Ensure OR research publications and policy briefs are accessible and relevant to programme implementers, and include opportunities for professional dialogue. This can be done by combining publications with effective communication and dissemination strategies and face-to-face meetings (such as Pastoralist Forums).

Photo credits Page 1: Creative commons licensed (CC BY-NC-ND 2.0) flicckr photo: Jessica Lea/Department for International Development https://www.flickr.com/photos/dfid/14727787123 Page 3: Rod Waddington (CC BY-SA 2.0) https://www.flickr.com/photos/rod_waddington/9727694723/

This Policy Brief was prepared by the RIF Evaluation Team.