Issuu on Google+

Integrated Assessment Approach M&E and IE

6 Purposes of Impact Planning, Assessment, Reporting & Learning (IPARL) •

Improve projects

Strengthen relationships

Demonstrate impact

Inform strategy

Sustain credibility and legitimacy

Educate society

Integrated Assessment Approach M&E and IE •

Baseline Data/Evidence

KPI Scorecards

Evidence of Change Journals

Comparative Constituency Feedback (CCF)

Rapid Advocacy Learning Activity Reports

Comparative Case Studies

KPIs Scorecards

Evidence of Change Journals

Constituents’ perceptions of CEPA’s performance and impact

Quality of relationships within CEPA

Comparative analysis

• • • • • •

New insight on the campaign’s effects Information on key relationships Inform assessments and course correct Amplify voice and participation Legitimacy Advocacy tool

• • •

Partners Allies Media

• • • • • •

National level

UNAIDS PEPFAR Global Fund UNITAID CHAI Unicef Global level

To what extent has your association with CEPA met your expectations? How would you rate the quality of the communications within CEPA?


How would you rate CEPA’s impact in the field of paediatric HIV/AIDS? In which ways is CEPA contributing to the elimination of paediatric HIV/AIDS in your country/globally? Impact

Oct-Dec 2009

Design and implementation of global level GAA/CEPA baseline survey

Oct-Dec 2009

Design of country level CEPA baseline survey

Jan- March 2010

Implementation of the country level baseline survey

Q4 2010

Implementation of global and country level constituency surveys

Q1 2012

Repetition of global and country level constituency surveys

Rapid Advocacy Learning Tools •

Bellwether Methodology

Policymaker Ratings

Advocacy Pre-mortem

Intense Period Debriefs

Rapid Advocacy Learning Tools

Intense Period Debriefs •

Occur shortly after policy windows or intense activity periods • Gathers in-depth and real-time information Convening of either a focus group or the conducting of individual interviews with advocacy stakeholders to capture data about advocates’ recent experiences. Captures: • •

• •

The public mood and political context during the policy window; What happened and how the campaign members responded to eve nts, especially as related to actions that occurred behind closed door s; Perspective on the outcome(s) achieved or not achieved; How strategies might be adjusted in hindsight.

Intense Period Debriefs Sample Questions 1. What events triggered this intense period? 2. How was the organization’s response determined? Who w as responsible for that decision? How was that decision co mmunicated to partners and allies? 3. Which elements of the organization’s response worked well ? Which elements could have been improved? 4. What was the outcome of the intense period? Was the resu lt positive or negative? 5. What insights will you take away from this experience that might inform your strategies going forward?

Comparison and Triangulation “A Platinum Standard of Rigor” For Impact Evaluation Comparison and Triangulation of: • • • • •

Alternate theories of action (casual pathways) Before and after, counterfactual Contexts (location, time, scale) Multiple types of information and data (Competing) stakeholder perspectives

Rigorous Comparison and Triangulation Advances in Case Study Designs Case studies still a central “method” and can be made more rigorous with relatively small investments Case studies essential for analysis of: • • •

context – location, time, scale, scope descriptive inference: construct validity causal processes: internal and external validity

(simple, complex, complicated, equifinality, etc..) • • • •

multiple results (outputs, outcomes, impacts) unintended results/unexpected consequences triangulate qualitative and quantitative data useful communication and reporting tool

Rigorous Comparison and Triangulation Advances in Case Study Designs Purposes and types of Case Studies:

• Generating theory of change – “inductive” • Elaborating TOC – “plausibility probes” • Evaluating TOC – “least likely” and “most likely” “tough tests”, “counterfactual thought experiments” • Innovating theory of change – “deviant case studies” of exceptional “outliers”

Rigorous Comparison and Triangulation Advances in Case Study Designs Case Study Methods and Tools: • Process-Tracing • Analytic Induction • Structured-focused comparison • General (and Specific) Elimination • Typological Theorizing • Increasing Observations • Data-bases of Case Studies … and so many more…