unsatisfactory practice was mostly in the setting of a clear methodology for data analysis, and the highlighting of concerns/limitations of the research process (though grantees were not required to do these). AgroTech, TICmbay and Digital Integration came out as the strongest performers, while MODES and UPTAKE performed fairly well with some shortcomings. EMM’s performance appeared fair, although it was unsatisfactory in terms of timeliness and completeness. Grantees’ relatively good performance as assessed in the DQA occurred despite the limited support and guidance in the collection and analysis of data at the start of the NA-ICT programme, lack of quality review by the MEL contractor, budgetary constraints, and the start of the MEL contract when the programme was already in its implementation. The results of the DQA are in line with the findings of the ET overall. • To what extent are the other public outputs of the programme (e.g. webinars, press releases etc.) suitable and of good quality? Public outputs of the programme were minimal. Some of the MEL contractor deliverables (i.e. the landscape analysis) were not approved by USAID to be published due to quality issues. USAID published two blogs on the USAID Learning Lab69 about the programme. The MEL contractor was working on a gender infographic and a cross-country case study prior to completion of its contract at the end of September 2018. Both document review and interviews have indicated that producing public outputs has not been a priority in any of the six NA-ICT countries; internal learning lab briefs were presented well (though there are some inaccuracies in the one about EMM, Mozambique) but it is not clear if they have been made public; and case studies70 are of good quality, although it is unclear how and to whom they were disseminated outside the NA-ICT community on the USAID Learning Lab. At country level, some documents were published. For example, CABI (Tanzania) has been uploading the technical briefs developed under UPTAKE on to the publicly accessible African Soil Health Consortium database. These technical briefs are clear and comprehensive according to the ET. This is illustrated in a technical brief of cassava, a 35 message SMS campaign from UPTAKE in the African Soil Health Consortium database: http://africasoilhealth.cabi.org/materials/cassava-35-message-sms-campaign/. FRI published several blogs on their Barzawire platform about the projects in Ethiopia, Ghana, Mozambique and Tanzania. • What lessons can be learned about the challenges in establishing common indicators and collection of data? The idea of having common PIRS indicators across all six countries was rather new, both for the MEL contractor and the grantees, but not for USAID. It took time to make grantees aware of the benefits of common indicators and of shared definitions and to train them to use them accordingly. All grantees agreed that more time should have been allocated for face-to-face capacity building. Clear tools, such as data collection sheets and definition guidelines, are helpful to grantees. Common indicators are hard to create but are necessary to compare countries with each other. SSTP had many more indicators than NA-ICT, and assistance to grantees in reporting was provided by M&E experts in each SSTP country office, which worked well. USAID was not satisfied with the MEL contractor’s ability to implement, or support, the use of common indicators across countries. It did not help that the MEL contractor was contracted late. However, the face-to-face workshop run by the MEL contractor in Tanzania had a day dedicated to data collection practices and plans, monitoring techniques and a discussion of the challenges grantees face in collecting and reporting quality data. The FRI contact responsible for three of the countries where FRI was USAID’s Learning Lab is an interactive community where members can access and contribute to a growing collection of tools and resources on integrating collaborating, learning, and adapting (CLA). USAID established for NA-ICT CF a closed subcommunity to collaborate and learn from each other. 70 Four-page briefs about NA-ICT CF implementation in five of the six countries, produced by the MEL contractor. 69
37