Appendices
Table 21. Effectiveness summary Issues investigated MOPAN has worked hard to improve the quality of its assessments, to enhance their use, including its methodological approach and the assessment process. To what extent did these efforts pay off in terms of higher quality and use of MOPAN assessments?
Information used Collected MOPAN stakeholders’ concerns about quality and relevance Comparison of MOPAN processes and practices with best knowledge-to-use practices
To what extent does the effort of enhancing use contribute to improving quality? To what extent have member administrations of MOPAN used assessments in the following ways (among others) to: • help fulfil accountability criteria? • inform budget allocation? • monitor or gain intelligence about MOs? • support a strategy of influence? • learn about other members' multilateral policies? • • • •
• •
To what extent have MOPAN assessments been used by MOs in the following ways (among others)? To help identify organisational issues? To reinforce partnerships among MOs? To what extent have MOPAN assessments been used to contribute to change across the multilateral system in the following ways (among others)? Inform discussion about significant issues across the MS (including cross-cutting themes? Inform system-wide reflection about organisational performance measures and accountability?
Lines of evidence Survey of members and MO focal points Interviews with MOPAN members, Secretariat staff and consultants Within case studies: interviews with MOs Review of key documents, especially the 2018 methodological review
Extent to which each CC is confirmed across case studies.
Interviews with MOPAN members, Secretariat staff, and consultants
Extent to which assumptions underlying each CC are found to be valid.
Survey of former ILs and MO focal points
Role of other drivers contributing to expected changes alongside MOPAN assessment Significance of MOPAN assessments relative to other sources of influence in contributing to identified uses. Identification of other potential uses noted by case study interlocutors and other stakeholders.
Case study interviews with selected MOs, member representatives, Secretariat staff and consultants and review of selected internal documents. Case studies selection criteria: Evidence of use from interviews with MOPAN members and Secretariat staff Open text comments from the survey of former ILs and MO focal points Review of publicly available documents from MOs
Relevance: To what extent is MOPAN’s work relevant for an evolving multilateral context and support efforts to promote coherence and improve the performance of the Multilateral System? This question will be addressed by reconstructing MOPAN's ToC and considering its underlying assumptions. On the one hand, we will continue to clarify expectations of MOPAN along the lines expressed above. On the other, we will discuss the assumptions about the relationship between MOPAN and the MOs and how MOPAN can contribute to the overall performance of the multilateral system.
113
