Page 1

Evaluation of  the   ICCO  Alliance  Programmatic  Approach               ANNEXES INCLUDING THE TERMS OF REFERENCE AND THE THREE SUB-REPORTS  


TABLE OF CONTENT Annex  1  -­‐  Terms  of  Reference  Advancement  of  ICCO  Alliance’s  working  with  the   Programmatic  Approach  2009  –  2012       Annex  2  –  Work  plan  ICCO  Evaluation  Programmatic  Approach     Annex  3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach       Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  -­‐  Survey   about  the  development  and  performance  of  coalitions     Annex  5  –  Terms  of  Reference  Advancement  of  ICCO  Alliance’s  working  with  the   Programmatic  Approach  2009  –  2012,  phase  3,  field  study     Annex  6  -­‐  Report  of  the  field  (third)  phase  of  the  Evaluation  Programmatic  Approach          


Annex 1  -­‐  Terms  of  Reference  Advancement  of  ICCO  Alliance’s  working  with  the   Programmatic  Approach  2009  -­‐  2012   1.  Introduction   This  terms  of  reference  concerns  the  second  evaluation  of  The  ICCO  Alliance  Programmatic   Approach.  A  first  evaluation  has  been  carried  out  by  two  external  consultants,  Erica  Wortel  and   Jouwert  van  Geene,  and  was  mainly  based  on  a  desk  study.  The  evaluation  had  a  strong  focus  on   learning  directed  to  provide  input  to  further  strengthen  the  implementation  of  the  Programmatic   Approach.  Many  of  the  recommendations  have  been  taken  up  in  the  new  Business  plan  2011  –  2015.   The  Programmatic  Approach  still  uses  very  much  a  ‘learning  by  doing’  approach,  with  changes  being   induced  by  experiences  in  the  ‘field’  and  reflections  on  the  way  of  working  with  the  approach.  After   another  3  years  of  working  with  the  PA,  the  ICCO  Alliance  wants  to  take  a  more  in-­‐depth  look  again   and  see  what  the  issues  are  at  this  moment,  how  the  PA  is  working,  and  if  tangible  results  can  be   seen  already?     2.  Background   The  ICCO  Alliance  is  implementing  a  Programmatic  Approach  (PA)  since  the  start  of  its  first  Business   Plan  2007  -­‐2010  in  2007.  PA  is  both  part  of  ProCoDe,  the  change  agenda  of  ICCO  focussing  on   decentralisation  and  co-­‐responsibility,  and  an  important  supporting1  instrument  in  this  change   agenda.  The  organization  has  chosen  to  implement  the  PA  in  a  ‘learning  by  doing’  and  ‘continuous   adaptation’  approach,  so  step  by  step  and  with  changes  induced  by  own  learning  on  the  use  of  the   PA  in  practice.     Theory  of  Change  of  the  Programmatic  Approach   The   Programmatic   Approach   is   essentially   about   the   way   in   which   the   ICCO   Alliance2   promotes   cooperation  between  organizations  in  developing  countries  in  order  to  reach  development  results.     As  ICCO  Alliance  we  recognize  that  poverty  and  injustice  are  always  related  to  complex  problems  in   which   many   people   have   a   stake   and   where   organizations   represent   certain   interests.   All   are   embedded   in   larger   systems   that   often   maintain   existing   inequalities.   The   combination   of   different   systems   makes   up   societies.     The   ICCO   Alliance   aims   at   changing   the   systems   that   embed   and   maintain  inequalities  in  such  a  manner  that  poverty  is  ended,  justice  is  guaranteed  and  rights  of  all   individuals  and  communities  are  respected.    To  be  able  to  do  so  we  propose  to  work  in  an  approach   that   will   support   actors   with   different   stakes   in   systems   to   come   together   and   develop   a   shared   agenda  for  change.  We  call  this  approach  the  Programmatic  Approach  which  is  defined  as  follows:     A   multi   stakeholder   process   that   leads   to  organizations   working   together   based   on   a   joint   analysis,   shared  vision  and  objectives  and  clear  perspective  on  the  results  of  the  cooperation.  In  such  a  process   all  actors  can  do  different  things,  work  at  various  levels  and  use  their  own  strengths  for  the  common   purpose   and   objectives,   as   well   as   share   some   activities   and   in   particular   share   and   participate   in   the   linking   and   learning   processes.   The   programmatic   approach   does   not   only   address   single   problems   but  aims  at  change  in  systems3     The  Programmatic  Approach  is  not  only  a  new  strategy  for  the  ICCO  Alliance.  It  is  also  a  very  specific   form  of  Programmatic  Approaches.  Programmatic  Approaches  are  mainly  interpreted  as  the  bringing   1

The International Advisory Group phrased this support of PA in 2007 as stimulus for: complementary and cooperative roles between stakeholders; a common goal; a participatory process and co-owned decision making 2 The ICCO Alliance is formed by: ICCO, Edukans, Prisma, KerkinActie, Share People, ZeisterZendingsgenootschap, Oikocredit, Yente 3 A system is a set of interacting or interdependent entities forming a larger whole. These systems may include organisational systems, may have geographical boundary, often has multiple levels and actors. Systems have the capacity to change, to adapt when it is necessary in response to internal or external stimulus. Complex Adaptive Systems, Heather Baser and Peter Morgan, Complex Adaptive Systems Theory, ECDPM 2004

Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

3 date: April 2013


together of  relevant  projects  and  programs  in  one  planning  container  such  as  the  Sectoral  Approach.   For  the  ICCO  Alliance  the  core  of  the  Programmatic  Approach  is  that  we  support  cooperative   processes  of  multiple  stakeholders  aiming  at  creating  systemic  change  because  we  think  that  we   need  the  strength  of  cooperation  to  be  effective  in  realizing  difficult  and  complex  changes.  It  is   therefore  not  mainly  a  planning  approach  for  the  ICCO  Alliance  but  a  strategy  for  realizing   fundamental  change  with  our  partner  organizations  and  other  stakeholders  in  the  areas  in  which  we   work.     This  angle  is  offered  by  Systems  Thinking.  Systems  are  defined  as  interactions  among  diverse  agents   that  persist  and  evolve  as  a  coherent  whole.  Systems  Thinking  looks  at  the  ‘whole’  first  and  examines   how  parts  of  the  wider  whole  influence  each  other,  or  change  as  result  of  their  relationship  to  their   environment.  Attention  to  the  various  elements  of  the  system  is  secondary  to  attention  to  the   whole4  5.  Systems  thinking  states  that  changes  in  parts  of  a  system  will  always  cause  the  whole   system  to  change.  This  change  will  however  not  have  a  predictable  result  nor  can  it  be  planned  in  a   linear  fashion.  The  ICCO  Alliance  takes  systems  behavior  into  account  in  its  Programmatic  Approach.   The  insecurity  that  is  implied  by  the  unpredictability  of  changes  needs  to  be  reflected  in  the   monitoring  and  evaluation  systems  that  we  use.  In  addition  to  measuring  expected  changes,  we  need   to  be  able  to  capture  the  unexpected  and  ‘notice’  emergent  change  as  well.     This  line  of  thinking  has  resulted  in  the  following  theory  of  change  underpinning  our  Programmatic   Approach:   • Development  problems  are  the  result  of  complex  systems  of  interlinked  actors,  structures,   institutions  and  processes   • Complex  problematics  demand  an  approach  that  can  deal  with  and  work  in  the  complexity.   Therefore  a  Multi  Stakeholder  Process  (MSP)  is  needed   • MSPs  lead  to  joint  learning  and  cooperation  between  the  actors  involved   • The  MSP  represents  the  system  involved  in  the  problematic.  Cooperation  between  actors  and   organizations  leads  to  added  value:  greater  effectiveness  in  change  at  the  institutional  level  and   whole  system  change.   • The  ICCO  Alliance  will  support  existing  cooperative  processes  and  initiate  the  cooperative   process  if  none  exists  yet.     • Coalitions  of  cooperating  actors  have  (and  adhere  to)  ownership  in  the  programmatic   cooperation  (the  program).     • This  also  implies  that  a  coalition  can  identify  possibilities  for  diversification  of  funding  sources  to   assure  sustainability  of  the  cooperation  and  independence  from  the  ICCO  Alliance.  It  is   preferable  that  the  cooperative  process  is  not  solely  dependent  on  ICCO  Alliance  funding.     Evaluation   In  2009  an  evaluation  of  the  programmatic  approach  of  the  ICCO  Alliance  was  carried  out.  This   evaluation  –  although  very  relevant-­‐  was  done  in  a  time  that  the  PA  had  not  yet  landed  fully  in  the   organization  (and  the  decentralized  organization  later  on)  and  the  decentralization  itself  was  not  yet   fully  implemented.  Both  factors  were  certainly  contributing  to  confusion  on  the  content  of  this  new   way  of  working.  The  evaluation  yielded  useful  recommendations  to  improve  working  along  the  PA   principles  which  were  taken  up  by  the  organization.  Main  recommendations  were  to:   - ‘recalibrate’  the  Programmatic  Approach  and  give  more  theoretical  underpinning   - close  the  learning  loops  on  the  Programmatic  Approach  in  the  organization   - strengthen  capacity  for  the  Programmatic  Approach  on  all  levels:  ROs  –  GO  and  program   coalitions   4

Definition by Peggy Holman in Engaging with Emergence, page 220, Berrett Koehler 2010 The idea and practice of systems thinking and their relevance for capacity development’, Peter Morgan, ECPDM march 2005 5

Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

4 date: April 2013


- provide instrument  to  support  the  approach  in  MSD  and  instruments     PA  capacity  development     Already  starting  under  the  first  Business  Plan  various  types  of  support  have  been  given  to  various   bodies  in  the  organization  to  draw  PA  to  a  higher  level.  In  this  support  the  lessons  drawn  from   working  in  a  programmatic  way  in  practice  are  being  taken  along.  The  table  below  gives  an  overview   of  the  various  learning  and  capacity  development  activities  around  the  Programmatic  Approach:     Table  Overview  of  PA  activities   Year   Activity   For  whom     by  whom   Follow-­‐up   2008-­‐ Training  of  staff  of   Staff  ICCO   CDI   Open  Space   2009-­‐ ICCO  in  Utrecht  and   Staff  ICCO  Alliance   sessions   2010   staff  of  ICCO  Alliance   member   partners   organisations  NL   2008-­‐ Open  Space  learning   Staff  ICCO  Alliance   P&D   Appreciating  and   2009-­‐ sessions   NL   on-­‐going  training   2010   programmes   2008-­‐ Learning  History   Programmes  and   Hester  Pronk   Appreciating  and   2009   development   staff  ICCO  Alliance   on-­‐going  training   programmes   2009-­‐ Appreciating  the   6  programmatic   Appreciating   Synthesis  document   2010   Programmatic   Development   team:  James   recommendations,   Approach   processes   Taylor   developing  Pscan   Kaustuv   Bandhopadyay   Meas  Nee   Domien  Bruinsma   Lisette  Caubergs   Charly  Buxton   Fons  van  der   Velden   Hettie  Walters   2011-­‐  Regional  training  on   POs,  partner   Hettie  Walters   Recommendations   2012   PA  in  Central  Asia,   organizations   to  staff  and  partners   Central  America,  South   Africa  and  Southeast   Asia   2012   5  Action  Researches  on   Programmes  and   Junior  action   Reports  for  RO;  film   state  of  affairs  of  5   staff  ICCO  Alliance   researchers   of  feed  back   programmes   session,  films  on  the   use  of  participatory   video  processes  in  2   programmes   2012   Participation  in  PSO   Two  programs:   Penpen  Libres  (   Reports,  filmed   TLP  on  Power  in  Multi-­‐ organisations  and   Philippines   interviews,  joint   Stakeholder  Processes   stakeholders   Seeweeds  VCD)   publication   and  Job   Blijdenstein   (Forestry  program,   Guatemala)   2011-­‐ 3  Continental  Training   ICCO  staff  and   Hettie  Walters  and   Strengthening  ToC   Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

5 date: April 2013


2012

of Trainers  on  the   Programmatic   Approach    

Training ROs  on  PA   Kirgystan,  South  –East   Asia   Training  on  ToC  

2012-­‐ 2013

Partner organisations  40-­‐ 60%  >100   participants   PO  

CDI

Approach in  PA  

Hettie Walters  

Ongoing, linked  to   many  other  CoL   and  training/   workshops  where   we  meet  staff   ICCO  and  partners   and  in  WASH   Alliance  

Hettie Walters  

In some  Programs   ToC  are  being   developed.  

The  lessons6  learned  of  both  the  Action  Research  and  the  Thematic  Learning  Programme  have  been   merged  by  the  Capacity  Development  Coordinator  in  several  synthesis  documents  and  in  the   Guidance  on  the  Programmatic  Approach  note.  Processes  have  been  documented  and  final  products   have  all  been  placed  on  the  Compart  wiki:  Programmatic  Approach.  The  Action  Research  has  led  to  a   final  document  that  was  shared,  such  as  a  video  of  the  final  presentation  and  a  blog  on  the   development  of  ToC  in  one  of  the  programmes  involved.  These  products  and  other  recent  products   of  learning  processes  will  be  posted  on  the  new  ICCO  portal  Programmatic  Approach  community.     After  all  these  inputs  on  strengthening  the  PA  and  another  3  years  working  in  practice  with  this   approach  the  ICCO  Alliance  wants  to  undertake  a  new  evaluative  study.  This  is  fully  in  line  with  the   intention  to  become  a  learning  organization.  The  outcomes  of  the  evaluation  will  serve  as  input  for   further  learning  on  working  with  the  PA  as  methodology  and  possibly  leads  to  changes  and   adaptations  in  the  ways  of  working  for  the  coming  years  till  2015  and  possibly  beyond.     3.  Purpose   3.1.  Purpose   Purpose  of  this  evaluation  is  to  further  deepen  our  understanding  of  the  Programmatic  Approach  in   practice  and  its  contribution  to  the  implementation  of  effective  programs  reaching  their  (change)   objectives.  The  main  focus  is  on  critical  aspects  of  the  Programmatic  Approach  like  ownership,   stakeholder  diversity  and  sustainability,  and  to  grasp  the  factors  related  to  these  issues.  This   evaluation  purposefully  builds  on  the  various  (internal)  studies  done  on  the  Programmatic  Approach   by  giving  a  comprehensive  external  view.     With  the  results  of  the  evaluation  the  ICCO  Alliance  will:   -­‐   have  deeper  understanding  in  the  development  of  the  Programmatic  Approach,  both  in   relation  to  the  development  and  functioning  of  the  programmatic  coalitions  and  the  way  in   which  program  coalitions  work  towards  reaching  their  development  objectives.   -­‐   get  pointers  for  further  consolidation  of  the  Programmatic  Approach    

6

Main issues from the Action Research and the Thematic Learning Programme are: How are partners cooperating, and how has their joint ‘programme’ been developed, how is monitoring and joint learning taking place? What is the role played by the ICCO Alliance in the Programmatic cooperation? In the action research the final recommendations all concerned the creation of clarity: about purpose of the cooperation, of envisaged results of the cooperation, of roles played by all actors involved, clarity in developing a Theory of change, clarity on responsibilities of all actors involved, and in particular about ownership in relation to funding.

Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

6 date: April 2013


-­‐ -­‐   -­‐  

gain insight  in  the  contribution  of  the  Programmatic  Approach  to  the  realization  /   implementation  of  ESP3.   gain  insight  in  the  role  the  ICCO  Alliance  plays  in  strengthening-­‐  hindering  the  development   of  fruitful  cooperative  processes.   gain  insight  in  particular  in  the  relation  between  the  Programmatic  Approach  and  the  other   crosscutting  approaches  :  Gender  and  the  Rights-­‐based  approach:  are  programmes  and   coalitions  working  in  an  inclusive  manner  as  to  gender  and  human  rights?  

3.2  Objective     The  objective  of  the  evaluation  of  the  Programmatic  Approach  is  to  get  insights  on  some  more   qualitative  and  some  quantitative  questions  regarding  the  programmatic  approach.  The  outcomes  of   the  evaluation  will  help  to  further  consolidate  the  Approach  for  the  remaining  period  of  the  business   plan  2011  -­‐  2015  and  for  policy  development  beyond  2015.       The  most  urgent  questions  on  this  moment  are:  “What  works  well  and  what  lags  behind  or  doesn’t   work,  for  what  reason  in  relation  to  the  Programmatic  Approach?”;  Other  urgent  questions  are   questions  around  emerging  partnerships  and  their  development:  “How  has  the  programmatic   coalition  taken  shape?”;  “Who  are  the  stakeholders  in  the  program  coalition?”  and  ”Who  has  the   ownership  of  the  programmatic  coalition?”       Four  topics  are  identified  around  issues  which  are  considered  to  be  crucial  for  further  consolidation   of  the  Programmatic  Approach.  These  are;   Actors/  organisations  involved  in  programme  coalitions  and  their  responsibilities   (research  question  a)   Several  aspects  of  the  development  of  a  ‘joint  programme’  (how  partners  are   cooperating,  purpose;  clarity  on  developing  a  Theory  of  Change,  role  of  monitoring  and   joint  learning,  relation  to  other  IA  principles)  (research  question  b,  c  and  e)   Roles  of  ICCO  staff  and  external  advisors  (research  question  a)   Development  Results  of  programmes  (research  question  d)     4.  Research  questions   a.  program  coalitions  (quantitative)   • number  of  (functional)  programmes  per  [date    ]   • Thematic  orientation  of  program  coalitions   • Regional  division   • Role  ICCO  Alliance  in  the  PA   • Funding  modalities   b.  ESP  3  aspects:  systemic  change   • Who  are  the  members  of  the  program  coalitions?    Are  members  stemming  from  the  pool  of   existing  ICCO  Alliance  partners;  are  members  stemming  from  other  civil  society  partners  or   other  actors  (  like  government,  knowledge  institutions,  private  sector,  etc)   • In  what  ways  and  directions  are  program  coalitions  developing?   • Roles,  responsibilities  of  the  various  members  of  the  coalitions   • to  what  extent  does  the  use  of  the  programmatic  approach  as  methodology  strengthen  the   capacity  to  result  in  sustainable  forms  of  cooperation  between  different  actors  in  Southern   countries   c.  ESP  3  aspects:  co-­‐creation   • To  what  extent  does  the  programmatic  approach  contribute  to  the  ability  of  co-­‐creation  of   different  stakeholders  in  civil  society?   • Are  there  differences  in  the  application  of  the  Programmatic  Approach  in  the  6  thematic   fields  and  how  do  they  ‘show’?   Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

7 date: April 2013


What are  ‘institutional’  opportunities  and  constraints  within  the  ICCO  Alliance  regarding  the   Programmatic  Approach?  

d. assessing  progress  in  the  programmes  to  date   • What  have  been  the  most  significant  results  of  working  with  the  programmatic  approach  to   date?   e.  coherence  with  cross  cutting  principles   • How  are  gender  equality  and  human  rights  being  integrated  /  taken  up  in  programs?     5.  Methodology   The  evaluation  will  be  carried  out  in  the  form  of  a  desk  study  (synthesis  and  analysis  of  existing   documents,  available  studies  and  other  relevant  material  within  ICCO  /Alliance),  in  combination  with   interviews/  conference  calls  and  a  survey  among  ICCO  Alliance  staff  and  selected  partner   organizations  working  in  a  programmatic  way.  The  latter  will  be  used  for  triangulation.  In  the  analysis   stage  the  various  data  and  findings  will  be  compared  and  analysed  to  answer  the  research  questions.     Based  on  the  analysis  of  the  data  gathered  the  evaluators  will  formulate  a  specific  set  of  questions   which  still  have  to  be  answered  and  can  only  be    answered  by  field  work  research.  They  develop  a   motivated  proposal  for  a  phase  2  of  additional  field  work.  The  field  work  has  to  be  designed  to  find   answers  on  these  questions  by  visiting  a  maximum  of  three  of  the  5  programmes  studied  in  the   action.     In  the  whole  evaluation  process  the  evaluators  should  be  open  to  ‘surprises’’  and  unintended  effects   (both  positive  and  negative)  of  working  with  the  Programmatic  Approach.       Methods  of  data  collection  and  analysis:   • Secondary  data  review  of  the  various  reports  on  programmatic  approach  mentioned  under   point  10,  bibliography     • Conduct  semi-­‐structured  interviews  with  approx.  35  ICCO  Alliance  staff  (group  interviews   with  ICCO  KerkinAktie  specialists  in  Utrecht;  staff  of  Edukans  and  Prisma),  and  in  the  ROs  (by   Skype).  The  selection  should  be  made  in  such  a  way  to  be  able  to  answer  the  evaluation   questions  b  -­‐  d.   • Telephone  interviews  (by  Skype)  with  approx.  10  programmatic  coalitions  and  the  actors   involved  in  those  coalitions  (see  PSO  evaluation)  in  the  South  who  are  working  in  a   programmatic  way.   • Online  anonymous  survey  for  ICCO  staff  and  partners  in  the  South  on  working  with  and  the   results  of  the  programmatic  approach,  and  simple  statistical  analysis.   and  under  the  condition  of  a  well  substantiated  proposal     • Field  research  (interviews  with  various  stakeholders  and  beneficiaries),  focus  group   discussions  with  beneficiaries),  network  analysis  and  power  analysis)  to  complement  and   consolidate  findings  from  the  other  evaluation  questions  (especially  on  question  d  and  e).     6.  Deliverables   A  draft  report  in  English;  to  be  submitted  December  31,  2013  latest.     A  final  report  in  English  (max  25  pages),  excluding  annexes.  The  format  of  the  report  will  be  in   electronic  version  and  in  hard  copy  (5  copies).  The  final  report  should  be  submitted  within  7  days   after  receiving  ICCO’s  comments  on  the  draft  report.     7.  Planning   Part  of  the  evaluation   time   Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

8 date: April 2013


Phase 1     Document  study   5  days   Group  interview  (5,  with  5-­‐8  Alliance  staff)   6  days   Skype  interviews  RWO  staff  (16)   8  days   Individual  interviews  (5,  with  RMs,  R&D  staff)   3  days   Survey  (including  preparation,  distribution  and  processing)   5  days   Interviews  with  coalition-­‐  partner  organizations  (10)   5  days   Report  writing  phase  I   5  days       Subtotal     37  days       Phase  2     Field  visit  (to  max.  3  programme  coalitions,  5  days  +  travel)   21  days   Report  writing  phase  II  +  overall  report   3  days   Finalisation   2  day       Subtotal   25  days       Total     62  days       The  evaluation  report  will  be  finalized  by  of  2013.     8.  Profile  evaluator     -­‐   background  in  sociology,  cultural  anthropology,  human  geography,  development  studies  or   similar  field   -­‐   knowledge  of  working  in  a  programmatic  way  (and  ICCO’s  interpretation  of  this  concept)   -­‐   experience  with  social  survey  methods  (data  collection,  entry,  analysis),  semi-­‐structured   interviews  and  focus  group  discussions   -­‐   analytical  skills   -­‐     at  least  5-­‐10  years’  experience  working  in  the  South     -­‐     English,  French,  Spanish  and  possibly  Portuguese  language  skills   The  possibility  exists  to  carry  out  the  evaluation  in  a  small  team  of  2  evaluators.     9.  Budget   The  evaluation  PW  is  part  of  the  budget  set  aside  for  PMEL.   The  maximum  budget  available  for  the  evaluation  is  €  55.000  (VAT  included).     Payments:   The  payment  procedure  is  the  following:   30%  at  acceptance   30%  at  presentation  draft  report   40%  after  receipt  of  approved  final  report  and  financial  justification     10.  Bibliography   Will  be  available  at  the  start  of  the  evaluation.  

Subject:ToR Programmatic approach status:draft Author: Dieneke de Groot; Hettie Walters

2009 - 2012

9 date: April 2013


Annex 2  –  Workplan  ICCO  Evaluation  Programmatic  Approach  

ICCO Evaluation  Programmatic  Approach   WORK  PLAN     THE  PURPOSE  AND  OBJECTIVE  OF  THE  EVALUATION   In  the  words  of  the  ToR  for  this  evaluation,  the  purpose  of  this  evaluation  is  to  further  deepen  the   understanding  of  the  programmatic  approach  in  practice  and  its  contribution  to  the  implementation   of  effective  programs  reaching  their  (change)  objectives.  The  main  focus  is  on  critical  aspects  of  the   programmatic  approach  like  ownership,  stakeholder  diversity  and  sustainability,  and  to  grasp  the   factors  related  to  these  issues.  This  evaluation  purposefully  builds  on  the  various  (internal)  studies   done  on  the  programmatic  approach  by  giving  a  comprehensive  external  view.   The  objective  of  the  evaluation  of  the  programmatic  approach  is  to  get  insights  into  some  more   qualitative  and  some  quantitative  questions  regarding  the  programmatic  approach.  The  outcomes  of   the  evaluation  will  help  to  further  consolidate  the  approach  for  the  remaining  period  of  the  business   plan  2011  -­‐  2015  and  for  policy  development  beyond  2015.       THE  MAIN  EVALUATION  QUESTIONS   For  the  ICCO  Alliance,  the  most  urgent  questions  for  now  are:   With  regards  to  the  programmatic  approach:  what  works  well  and  what  lags  behind  or  doesn’t  work   with  regards  to  the  introduction  and  implementation  of  the  approach  and  the  learning  thereon,  and   what  factors  contribute  to  this?   With  regards  to  programmatic  coalitions  and  programmes7:  how  have  these  programmatic  coalitions   and  programmes  taken  shape,  who  are  their  stakeholders,  and  who  has  their  ownership?     In  dealing  with  these  overall  questions,  the  evaluators  will  consider  four  specific  dimensions  in  their   analysis:  actors  (who  is  involved),  roles  and  responsibilities  (who  does  what  why),  process  (who  does   what  when  and  how),  and  results  (to  what  does  all  of  it  lead).       MORE  SPECIFIC  RESEARCH  ISSUES   During  their  research,  the  evaluators  will  treat  all  the  following  specific  issues  to  enable  appreciation   of  the  abovementioned  questions:   a) Programme  coalitions  and  programmes  –  initial  sketch   • Evolution  of  the  number  of  (functional)  programme  coalitions  and  programmes  over  time;   • Thematic  orientation  of  programme  coalitions  and  programmes;   • Members  of  the  programme  coalitions  (origin,  type);   • Geographical  location  of  programme  coalitions  (regions  and  countries);   • Funding  modalities  of  the  programme  coalitions  and  programmes;   • The  instruments  put  in  place  to  promote  the  emergence  of  programmatic  coalitions  and  to   manage  the  programmes.   During  the  preparatory  desk  study  phase,  the  evaluators  will  further  clarify  and  define  the  way  in   which  they  will  work  with  the  terms  and  concepts  ‘programme  coalition’,  ‘programme’,  ‘   functional’,  ‘  themes’,  ‘origin  &  type’,  and  ‘funding  modalities’  mentioned  above.  They  will  do  so   based  on  the  documents  at  hand  and  in  close  collaboration  with  ICCO  staff  concerned.   b) Programme  coalitions  and  programmes  –  more  in  detail  (systemic  change  and  co-­‐creation8)   7

Note: the phenomena ‘programme’ is added to the scope as it seems at times to be a starting point for coalitions as well as at times its outcome and means to impact Annex 2  –  Workplan  of  the  Evaluation  Programmatic  Approach  

10


• • •

Evolution of  the  programme  coalitions  and  variations  thereof  in  various  thematic  and   geographical  settings;   Roles  and  responsibilities  of  the  various  members  of  the  coalitions  and  the  ways  in  which   these  have  evolved  (including  ICCO  staff  and  external  advisors);   The  sustainability  of  the  cooperation  within  the  programme  coalitions.  

c) The programmatic  approach   • The  evolution  of  the  programmatic  approach,  and  the  geographical  and  thematic  variations   therein.     • The  effectiveness  of  the  programmatic  approach  as  a  methodology  to  strengthen  sustainable   forms  of  cooperation  between  different  actors  in  Southern  countries  (or:  the  degree  to   which  the  programmatic  approach  has  contributed  to  the  emergence  of  sustainable   programme  coalitions  –  see  above);   • The  extent  to  which  the  programmatic  approach  contributes  to  the  ability  of  co-­‐creation  of   different  stakeholders  in  civil  society;   • The  ‘institutional’  opportunities  and  constraints  within  the  ICCO  Alliance  regarding  the   programmatic  approach.   d) Assessing  progress  in  the  implementation  of  the  programmatic  approach9  to  date   • The  most  significant  results  of  working  with  the  programmatic  approach  to  date.   e) Coherence  with  cross  cutting  principles   • The  ways  in  which  and  the  extent  to  which  gender  equality  and  human  rights  are  being   integrated  /  taken  up  in  programmes.   METHODOLOGY   The  evaluation  will  be  executed  during  a  number  distinct  phases:   A  document  study  phase  during  which  the  evaluators  will  analyse  existing  documents,  available   studies  and  other  relevant  material  in  order  to  get  to  grips  with  the  issues  at  stake.  It  will  generate   the  first,  very  rough  overviews  of  the  emergence  and  evolution  of  the  various  key-­‐concepts  with   regards  to  the  programmatic  approach.  It  will  result  amongst  others  in  an  unpolished  timeline   showing  the  evolution  of  the  programmatic  approach.  Some  short  interactions  are  foreseen  with  key   IA  staff  for  clarification  purposes.  At  the  start-­‐up  of  this  phase,  the  evaluators  will  draft  a  detailed   ToR  for  a  junior  who  will  browse  through  all  IA-­‐systems  and  contact  relevant  staff  to  generate  initial   tabular  overviews  with  quantitative  data  with  regards  to  programme  coalitions  and  programmes.   The  evaluators  will  guide  the  junior  during  the  process.   These  very  first  insights  generated  during  this  phase  will  also  result  in  the  formulation  of  a  series  of   questions  that  will  form  the  basis  for:   An  interview  and  data-­‐gathering  phase  during  which  the  evaluators  will  engage  with:   • 35  ICCO  Alliance  staff  (semi  structured  group  interviews  with  Dutch-­‐based  thematic  staff   from  ICCO  KerkinAktie,  Edukans  and  Prisma);   • 16  ICCO  Alliance  staff  from  the  regional  offices  (through  semi-­‐structured  Skype-­‐interviews);   • Representatives  of  10  programmatic  coalitions  (through  semi-­‐structured  Skype-­‐interviews);   • Regional  managers  and  R&D  staff  (though  inquisitive  conversations);   • A  broader  group  of  ICCO  Alliance  staff  and  partners  in  the  South  on  working  with  and  the   results  of  the  programmatic  approach  (through  a  semi  structured,  anonymous  online   survey).   The  outcomes  of  this  phase  will  be  used  to  adapt  and  further  fine  tune  the  insights  gathered  during   the  desk  study  phase  and  to  add  first  and  indicative  appreciations  of  results  generated  through   8 9

The concepts systemic change and co-creation did not seem to cover the issues mentioned thereunder Note: the original research questions mentioned ‘programmes’ here

Annex 2  –  Workplan  of  the  Evaluation  Programmatic  Approach  

11


working with  the  programmatic  approach  and  programme  coalitions.  The  first  timeline  will  be  fine-­‐ tuned  and  critical  factors  during  the  evolution  of  the  programmatic  approach  will  be  highlighted.   A  reflective  and  initial  report  writing  phase  during  which  the  evaluators  will  analyse  the  undoubtedly   varied  picture  that  emerged  from  the  inquisitive  reading  and  listening  phases  and  produce  a   synthesis  capturing  the  key  elements  with  regards  to  the  evaluation  questions  and  the  underlying   dimensions.  During  this  phase,  the  evaluators  will  have  at  least  one  encounter  with  key  ICCO  staff  to   sharpen  the  synthesis  as  it  emerges.  As  this  synthesis  might  most  probably  be  enriched  and   deepened  by  first-­‐hand,  real  life  experiences  and  stories  directly  from  field-­‐level,  the  evaluators  will   produce  a  detailed  and  substantiated  proposal  to  finalise  the  evaluation  of  the  programmatic   approach  with  a  field  research.     A  field  research  phase  during  which  the  evaluators  will  conduct  interviews  and  focus  group   discussions  with  various  stakeholders  in  programme  coalitions  and  beneficiaries  of  programmes   implemented,  to  complement  and  consolidate  findings  from  the  earlier  evaluation  phases.     A  final  reporting  phase  during  which  the  evaluators  will  finalise  the  initial  report  and  add  insights   gathered  during  the  field  research  phase.  During  this  phase,  the  evaluators  will  include  a  final   iteration  with  key  ICCO  staff  in  order  to  incorporate  their  comments  in  the  final  document.     RESULTS   The  results  of  the  evaluation  will  allow  the  ICCO  Alliance  to:   • have  deeper  understanding  in  the  evolution  of  the  programmatic  approach,  both  in  relation   to  the  emergence  and  functionality  of  the  programmatic  coalitions  as  well  as  in  the  way  in   which  programme  coalitions  work  towards  reaching  their  development  objectives  through   joint  programmes;   • get  pointers  for  further  adaptation  and  consolidation  of  the  programmatic  approach;   • gain  insight  in  the  contribution  of  the  programmatic  approach  to  the  realisation  and   implementation  of  the  current  strategic  plan;   • gain  insight  in  the  role  the  ICCO  Alliance  plays  in  strengthening  or  hindering  the  development   of  fruitful  cooperative  processes;  and  to   • gain  insight  in  the  relation  between  the  programmatic  approach  and  the  other  crosscutting   approaches  :  Gender  and  the  Rights-­‐based  approach.   TIME  PLANNING  AND  BUDGET   The  initial  report  will  be  delivered  before  the  end  of  July.  The  end  report  (including  the  outcomes  of   the  field  visits)  will  be  delivered  as  soon  as  possible  after  the  field  visits  before  the  end  of  December   of  this  year.   The  number  of  days  presented  in  the  table  below  deviate  from  those  indicted  in  the  ToR.  The   evaluators  believe  that  for  certain  activities  more  time  is  required,  e.g.  document  study  in  view  of   the  number  of  documents  provided;  certain  activities  (group  interviews)  will  partly  involve  the  input   of  both  the  evaluators;  while  the  coordination  with  IA  key  staff  in  both  phases  was  initially  not   foreseen.   The  junior  indicated  for  the  document  study  phase  is  not  included  in  the  budget.  His/her  assignment   will  most  probably  take  between  5-­‐10  days,  depending  on  the  presence  of  readily  available  data   and/or  the  challenges  to  further  unearth  these.          

Annex 2  –  Workplan  of  the  Evaluation  Programmatic  Approach  

12


Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach      

ICCO Evaluation  Programmatic  Approach   INITIAL  SYNTHESIS  OF  FINDINGS     INTRODUCTION   The  evaluation  of  the  programmatic  approach  as  shaped  and  implemented  by  the  ICCO  Alliance  (IA)   is  subdivided  in  two  phases:   • a  phase  during  which  the  evaluators  read  and  analyse  all  material  available  on  the  launch,   emergence,  and  implementation  of  the  programmatic  approach,  including  material  on  the   subsequent  learning  thereon,  gather  additional  data  in  dialogue  with  key  players  in  the  ICCO   Alliance,  and  share  their  first  thoughts  with  them;   • an  optional  phase  of  field  visits  to  three  countries  to  better  understand  and  document  the   conditions  and  factors  that  contribute  to  the  emergence  of  successful  coalitions.   The  online  survey  directed  towards  IA  programme  officers  and  representatives  of  coalitions  that  was   initially  foreseen  to  be  undertaken  during  the  first  phase  of  the  evaluation  will  be  held  during  the   month  of  August.   This  paper  presents  the  initial  findings  and  some  first  recommendations  based  upon  the  first  phase   of  the  evaluation.  The  outcomes  of  the  online  survey  will  be  included  in  the  final  report  on  this   evaluation.  The  current  paper  also  includes  a  first  series  of  thoughts  and  ideas  on  the  optional  phase   of  field  visits.  As  per  explicit  request,  this  paper  will  be  crisp  and  brief  to  enable  its  practical  use   during  the  upcoming  gatherings  and  strategic  deliberations  of  the  ICCO  Alliance  leadership.  As  a   consequence,  this  paper  does  not  present  the  extensive  evidence-­‐base  on  which  the  evaluators  have   grounded  their  findings  and  recommendations  and  does  not  always  follow  the  classical  rules  for   evaluative  studies.  The  evaluators  are  however  available  to  provide  for  detailed  explanations  to  and   share  their  coded  data-­‐base  with  all  interested  parties.  Although  all  findings  and  recommendations   are  based  upon  the  material  provided  and  the  conversations  held,  the  evaluators  are  solely   responsible  for  their  formulation.       THE  MAIN  EVALUATION  QUESTIONS     In  the  words  of  the  ToR  for  this  evaluation,  the  purpose  of  this  evaluation  is  to  further  deepen  the   understanding  of  the  programmatic  approach  in  practice  and  its  contribution  to  the  implementation   of  effective  programmes  reaching  their  (change)  objectives.  The  main  focus  is  on  critical  aspects  of   the  programmatic  approach  like  ownership,  stakeholder  diversity  and  sustainability,  and  to  grasp  the   factors  related  to  these  issues.  This  evaluation  purposefully  builds  on  the  various  (internal)  studies   done  on  the  programmatic  approach  as  well  as  on  all  conversations  held  by  giving  a  comprehensive   external  view.   The  objective  of  the  evaluation  of  the  programmatic  approach  is  to  get  insights  into  some  more   qualitative  and  some  quantitative  questions  regarding  the  programmatic  approach.  The  outcomes  of   the  evaluation  will  help  to  further  consolidate  the  approach  for  the  remaining  period  of  the  business   plan  2011  -­‐  2015  and  for  policy  development  beyond  2015.     THE  INTRODUCTION  OF  THE  PROGRAMMATIC  APPROACH  AND  THE  LEARNING  THEREON   In  introducing  the  programmatic  approach,  the  IA  has  followed  a  ‘learning  by  doing’  approach.  Since   2006  a  rich  series  of  documents  has  seen  the  light  in  which  the  ins  and  outs  of  ‘the  programmatic   way  of  working’  have  been  indicated  and  explained.  Parallel  to  these,  and  often  in  strong  interaction   with  these,  the  IA  has  invested  heavily  in  organising  all  kinds  of  learning  trajectories,  appreciations,  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

13


action researches  and  training  exercises  to  guide  and  support  the  IA  staff  as  well  as  partner   organisations  in  further  defining  and  working  with  the  programmatic  approach.  There  is  an   abundance  of  papers,  notes,  workshop  debriefings  and  syntheses  indicating  the  evolution  of   thoughts  and  experiences  on  introducing  and  working  with  the  programmatic  approach.  (See  the   Annex  es  B    and  C  to  this  report  for  a  full  overview  of  documents  and  peoples  consulted.)     Findings   1. The  choice  for  a  ‘learning  by  doing’  approach  was  necessary  as  the  programmatic  approach   was  described  in  bold  and  general  terms  only  at  its  launch  and  as  the  IA  staff  and  partners  had   no  prior  experience  in  working  in  this  way.   2. The  ‘learning  by  doing’  approach  seems  furthermore  justified,  as  working  in  a  programmatic   way,  through  its  very  nature,  simply  requires  such  an  approach.   3. All  documentation  on  the  different  learning  processes  present  a  clear  and  very  rich  picture  on   the  factors  contributing  to  the  approach’s  success  as  well  as  to  the  bottlenecks  it  faces.   4. Although  the  documentation  contained  quite  some  critical  remarks  on  the  introduction  of  the   programmatic  approach  right  from  the  start,  the  management  responses  to  these  seem  much   less  obvious  and  clear.  This  justifies  the  finding  that  learning  has  definitely  taken  place,  but   also  that  most  often  the  learning  loops  did  not  seem  to  have  been  closed.   The  introduction  of  the  programmatic  approach  went  hand  in  hand  with  the  ‘co-­‐responsibility’   process  (i.e.  bringing  the  design  and  decision-­‐making  on  the  processes  of  change  closer  to  its   partners  and  context)  and  the  ‘decentralisation’  process  of  the  operations  to  the  regions  (meant  to   better  execute  different  roles  indicated  in  the  IA  business  plans).  These  three  major  innovations  have   become  known  as  the  PROCODE  process.   It  is  clear  to  the  evaluators  that  these  three  major  change  processes  are  highly  interrelated  and  that   their  advantages  seem  obvious  (including  partners  in  decision-­‐making,  better  context  knowledge  and   closer  interaction  between  IA  staff  and  IA  partners).  All  documentation  however  reveals  that  the   parallel  launch  of  these  three  major  change  processes  has  complicated  the  introduction  of  the   programmatic  approach  and  might  even  have  set  it  off  in  other  directions  than  initially  intended.     Findings   5. In  the  eyes  of  many  partner  organisations  (and  IA  staff)  the  introduction  of  the  programmatic   approach  was  linked  to  the  decentralisation  process  and  associated  directly  with  the  new   thematic  focus  of  the  IA  indicated  in  various  business  plans.  The  changes  were  perceived  to  be   all  directed  by  ICCO,  changing  the  terms  of  the  cooperation  with  its  partners.  This  was  in  many   cases  not  perceived  as  ICCO  intending  to  share  power,  but  as  ICCO  taking  the  reigns  of  the   cooperation  stronger  in  its  own  hands.   6. Introducing  three  major  changes  at  the  same  time  has  certainly  put  severe  stress  on  the  IA   staff  and  systems.  Many  a  document  indicates  that  IA  staff  simply  did  not  and/or  could  not   invest  sufficient  time  in  the  often  delicate  processes  required  by  the  programmatic  approach   and  that  the  PMEL  systems  were  not  fully  geared  towards  supporting  the  introduction  of  the   programmatic  way  of  working.   Next  to  the  ‘learning  by  doing’  approach  and  the  launch  of  three  major  changes  at  the  same  time,  a   third  factor  has  had  a  heavy  bearing  on  the  ways  in  which  the  programmatic  approach  has  actually   materialised  in  the  IA  practise.   Right  from  its  inception  in  2006,  all  notes  and  guidance  on  the  programmatic  approach  have  stressed   the  significance  of  local  ownership  of  the  problems  and  the  solutions  thereto,  and  of  the  importance   of  tuning  interventions  to  the  local  context,  to  the  dynamics  between  local  stakeholders  and  to  their   agenda’s,  ambitions,  roles  and  opportunities.  Flexibility  in  timing  and  planning  and  true  local   ownership  over  the  process  are  key  ingredients  of  successful  implementation  of  the  programmatic   approach.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

14


The programmatic  approach  was  however  not  launched  in  splendid  isolation,  but  as  a  way  of  working   to  implement  the  IA  business  plans  2006-­‐2010  and  2011-­‐2015.  These  business  plans  present  specific   thematic  focus  areas,  and  detailed  objectives  and  results  to  be  achieved.       Findings   7. All  documents  and  experiences  reflect  the  true  balancing  act  between  learning  by  doing  with   an  approach  that  requires  flexibility  and  a  focus  on  local  actors  and  dynamics  on  the  one  hand   and  implementing  the  IA  business  plans  with  predefined  themes,  fixed  result  frameworks  and   procedure  driven  planning  and  reporting  formats  on  the  other.   8. As  a  matter  of  course,  the  balance  has  often  tipped  in  favour  of  implementing  the  IA  business   plans,  resulting  in  what  became  known  as  ‘programme-­‐building’  with  the  IA  in  the  driver’s  seat   of  programmes  and  coalitions.     CONCEPTUAL  CLARITY  ON  KEY  CONCEPTS   Appreciating  ‘what  works  well  and  what  lags  behind  or  doesn’t  work  with  regards  to  the  introduction   and  implementation  of  the  programmatic  approach  and  the  learning  thereon’  (one  of  the  evaluation   questions),  requires  clarity  on  the  key  concepts  of  the  approach  and  an  appreciation  of  the  degree  to   which  this  clarity  is  indeed  shared  amongst  the  rank  and  file  of  the  IA  staff  and  its  partners.  In  other   words:  “What  is  the  programmatic  approach?”  And:  “Do  all  concerned  understand  it  in  the  same   way?”  To  this  end,  the  evaluators  have  first  analysed  all  relevant  documents  and  established  a   timeline  indicating  the  evolution  of  the  various  key  concepts  with  regards  to  the  programmatic   approach.  They  have  secondly  added  on  this  timeline  various,  often  critical,  remarks  and   observations  on  the  concepts  that  emerged  from  earlier  evaluations  and  the  different  learning   histories,  appreciations,  action  researches  that  the  IA  undertook  during  the  2007-­‐2012  period.  A   summarised  overview  of  the  timeline  is  presented  in  Annex    A  to  this  report.     Findings   9. The  document  review  and  interviews  so  far  have  revealed  that  there  is  no  common   understanding  amongst  IA  staff  and  partners  of  what  is  to  be  understood  by  the  programmatic   approach.  Descriptions  vary  from  ‘cherishing  and  facilitating  local  ownership  of  development’,   to  ‘a  programme‘,  to  ‘a  blue-­‐printed  approach  with  the  IA  in  the  lead’,  to  ‘a  quest  for  the  holy   grail’,  to  ‘a  belief-­‐system’.   10. The  recent  studies  and  the  MTR  have  revealed  such  a  broad  variety  of  programmes  and   coalitions,  of  typologies  thereof,  of  organisational  models,  of  governance  structures,  of   contracting  arrangements,  and  of  funding  modalities  that  it  is  hard  to  distinguish  a  common   denominator  under  which  these  can  all  be  labelled  ‘fruits  of  working  with  the  programmatic   approach’.   11. There  are  a  number  of  key-­‐traits  of  the  current  IA  way  of  working  that  are  cherished  by  most   people  interviewed  and  that  these  wish  to  further  exploit  and  take  on  board  in  the  formulation   of  future  strategy  and  policy  currently  underway  (see  below).   12. Document  analysis  did  not  reveal  any  specific  attention  to,  or  development  and  introduction  of   practical  instruments  on  gender  equality  and  human  rights.   As  the  timeline  clearly  indicates,  the  key-­‐concepts  on  the  programmatic  approach  have  evolved  over   time.  The  description  most  consistently  in  use  is  the  one  introduced  in  2010:   The  programmatic  approach  is  defined  by  ICCO  as  a  process  that  leads  to  organisations  working   together  based  on  a  joint  analysis,  shared  vision  and  objectives  and  clear  perspective  on  the  results  of   the  cooperation.  In  such  a  process  all  actors  can  do  different  things,  work  at  various  levels  and  use   their  own  strengths  for  the  common  purpose  and  objectives,  as  well  as  share  some  activities  and  in   particular  share  and  participate  in  the  linking  and  learning  processes.  The  programmatic  approach  is   an  approach  that  does  not  only  address  single  problems  but  aims  at  change  of  systems.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

15


The programmatic  approach  is  described  as  a  process.  In  the  earlier  years  during  which  the  most   programmes  and  coalitions  were  shaped,  descriptions  of  the  approach  however  most  often  referred   to  ‘programmes’,  ‘interventions’,  ‘combinations  of  the  four  IA  roles  with  the  business  plan  themes’,   and  ‘stakeholders  collaborating  in  a  programme’.  The  most  recent  paper  in  2013  on  the   programmatic  approach  again  linked  the  approach  to  the  realisation  of  results  in  ‘identified  thematic   domains’,  without  clarifying  how  that  was  to  relate  to  the  alleged  key  importance  of  ‘local   ownership’.     Findings   13. Comments  on  the  apparent  lack  of  clarity  and  perceived  inconsistencies  on  the  key-­‐concepts   of  the  programmatic  approach  have  been  circulating  right  from  the  launch  of  the  approach  in   2006.  Despite  massive  efforts,  this  lack  of  clarity  continues  to  date.     14. The  absence  of  quality  standards  on  ‘good  programmatic  practice’  has  not  contributed  to   creating  the  necessary  conceptual  clarity.   Understanding  why  this  ‘experienced  confusion’  and  ‘perceived  inconsistency’  has  been  so  persistent   throughout  the  entire  2006-­‐2013  period  under  consideration  remains  one  of  the  big  challenges  of   the  current  evaluation.   The  simplest  explanation  seems  to  be  that  the  confusion  is  built  right  into  the  name  ‘programmatic   approach’  and  its  positioning  as  a  way  of  working  to  achieve  the  ambitious  results  in  the  six   programmes  that  are  indicated  in  the  IA  business  plans.  Simply  put:  the  bulk  of  the  IA  staff  seems  to   have  understood  the  approach  as  a  means  to  shift  from  a  ‘one  partner-­‐one  project-­‐one  contract’  way   of  working,  to  a  ‘coalition  and  programme  building’  approach,  in  which  they  have  taken  the  lead  and   indeed  started  to  create  coalitions  and  programmes  on  the  six  IA  themes  (or  programmes!).     Findings   15. IA  guidance  on  the  programmatic  approach  has  not  sufficiently  clarified  the  major  strategic   question  whether  the  IA  core-­‐business  is  about  ‘brokering  locally  owned  solutions  to  locally   owned  problems  within  local  settings  and  dynamics  following  local  agendas  and  rhythms’,  or   about  ‘facilitating  multi-­‐stakeholder  processes  to  support  joint  action  to  realise  the  results  on   programmes  as  indicated  in  the  IA  business  plans’.   16. The  apparent  lack  of  a  common  and  shared  understanding  on  the  key  concepts  of  the   programmatic  approach  has  given  room  to  the  organisations  participating  in  the  IA,  to  the   regional  managers  and  to  the  thematic  departments  ‘to  fill  in  the  blanks  themselves’  which  has   not  helped  in  creating  a  consistent  story  on  the  IA  way  of  working  and  an  alluring  and   unambiguous  corporate  profile.       THE  PROGRAMMATIC  APPROACH  AS  ‘EFFECTIVE  SELLING  POINT’   During  discussions  with  the  IA  leadership  (directors  and  regional  managers)  it  became  clear  that  the   IA  way  of  working  is  hardly  ever  explained  to  outsiders  (or  sold  to  them  for  that  matter)  using  the   label  ‘programmatic  approach’.   In  this  context  is  seems  fair  to  say  that  the  programmatic  approach  is  neither  new  nor  innovative.   The  approach  was  introduced  in  development  cooperation  in  the  eighties  and  nineties,  under  the   label  ‘process  approach’  and  also  ‘programmatic  approach’.  The  approach  lost  traction,  as  many  a   donor  appeared  not  to  be  able  to  handle  the  long-­‐term,  open-­‐ended  and  flexible  nature  of  the   interventions  that  resulted  from  working  with  it  (just  to  mention  one  of  the  reasons).   A  number  of  elements  of  the  programmatic  approach  are  however  much  cherished  by  most  people   interviewed,  and  indeed  often  mentioned  in  explaining  the  IA  way  of  working  to  partners  and   outsiders  alike.  These  elements  are  considered  to  constitute  the  ‘pearls  of  the  IA  practice’  that  are   most  definitely  to  find  their  way  into  the  future  formulations  of  the  IA  way  of  working:  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

16


• •

• • •

for change  processes  to  be  relevant,  they  are  to  be  grounded  in  a  thorough  and  continued   understanding  of  the  ever-­‐changing  context,  issues  at  stake,  power  relations  amongst  the  major   actors  and  the  dynamics  on  how  these  actors  relate  to  one  another  and  to  the  systems  of  which   they  are  part;   for  change  to  happen,  working  at  multiple  levels  with  multiple  stakeholders  is  necessary;   for  stakeholders  to  indeed  join  forces  effectively  and  enter  into  some  form  of  joint  and   emergent  action,  a  shared  vision  on  the  change  and  transformation  to  be  promoted  and  about   how  to  go  about  that  (in  other  words  a  mutual  consensus  on  a  ‘theory  of  change’)  seems  crucial;   for  change  to  be  sustainable,  is  seems  crucial  to  address  ‘the  big  issues’,  the  systems  that   generate  poverty  and  injustice;   for  change  to  be  effective,  learning  on  what  happens,  on  what  works  and  what  does  not  and   consequently  flexibly  adapting  one’s  set  of  activities  is  key;   relations  amongst  the  cooperating  parties  are  based  upon  trust  and  mutual  accountability.    

In a  simple  picture  the  main  results  of  the  IA  way  of  working:   joint analysis, shared vision and a clear perspectives on the results to be achieved organisation (NGO) action

organisation (govt.) learning big issue organisation (private)

organisation (FBO) cooperation organisation 'coalition'

Findings   17. The  ‘ideal  type’  IA  programmatic  approach  is  more  about  ‘facilitating  local  change  processes’   than  about  ’formulating  and  implementing  log-­‐frame  type  programmes’.   18. The  major  challenge  seems  to  be  on  how  to  couple  the  ‘local  ownership  over  development   processes’  as  advocated  by  the  programmatic  approach,  with  the  evident  own  institutional   interests  and  development  agenda  of  the  IA.   It  is  remarkable  to  note  that  these  key-­‐traits  very  much  resemble  the  original  IA  way  of  working  as   presented  in  the  2006  business  plan  (see  Annex  A).  Then  again,  why  did  this  way  of  working  not   materialise  in  a  consistent  way  everywhere  in  the  IA  practice?     Apart  from  the  argument  and  explanation  introduced  above  that  led  to  programme  and  coalition   building,  a  number  of  other  factors  have  contributed  to  an  all  but  full-­‐fledged  introduction  of  the   programmatic  approach.  These  are  linked  to  the  IA  roles  and  the  IA  management  instruments.     ROLES  OF  IA  STAFF   As  indicated  in  all  major  policy  documents  and  business  plans,  staff  of  the  IA  in  general  and  the   programme  officers  more  in  particular  are  to  play  a  role  as  strategic  funder,  broker  (knowledge  and   relations),  supporter  of  capacity  development  (of  the  cooperative  initiatives  and  of  its  partner   organisations)  and  acts  as  lobbyist  and  advocate  for  change  in  the  global  and  the  Dutch  polity.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

17


Of these  four  roles,  the  brokering  role  and  the  support  to  capacity  development  role  are  of  key   importance  to  a  successful  application  of  the  programmatic  approach.     Findings   19. The  various  documented  experiences  of  programme  officers  in  working  with  the  various  roles   indicate  a  substantial  tension  between  the  funder  role  one  the  one  hand  and  the  broker  and   capacity  builder  role  on  the  other.  Experience  apparently  shows  that  the  broker  and  capacity   builder  roles  are  compromised  by  the  fact  that  the  same  institution  that  brokers  and   strengthens  capacity  is  also  the  one  that  is  willing,  under  certain  conditions,  to  fund  ‘those   brokered  and  capacitated’.   20. Brokering  as  well  as  supporting  capacity  development  are  time-­‐consuming  and  delicate   processes  that  require  specific  skillsets.  Despite  the  massive  investment  in  trainings,  not  all  IA   staff  appears  to  have  the  necessary  competencies,  and  those  who  do  often  have  too  little  time   available  to  do  a  proper  job  on  the  two  roles.  Recent  studies  have  indicated  that  the  bulk  of   time  available  (±  80%)  is  invested  in  ‘dossier’  management  (part  of  the  funder  role?).   21. The  IA  managers  and  leaders  appear  not  to  steer  on  a  balanced  application  of  the  various  roles   (nor  on  a  successful  application  of  the  programmatic  approach  for  that  matter).  Appreciating   the  quality  of  work  is  done  on  indicators  like  number  of  contracts  signed  and  managed,   adherence  to  financial,  administrative  and  reporting  procedures,  and  amount  of  financial   resources  mobilised.   In  many  a  case,  external  consultants  are  hired  to  play  the  broker  role  and  the  supporter  of  capacity   development  role.  The  documentation  indicates  that  most  partners  are  very  satisfied  with  the  quality   of  work  done  by  these  external  parties.       Findings   22. Two  of  the  four  roles  in  the  core  business  of  the  IA  staff  seem  to  deliver  the  best  results  (both   quality  and  time  wise)  when  executed  by  external  consultants.     Recently  a  fifth  role  has  been  added  to  the  four  roles  mentioned  here  above:  the  fundraiser  role.   Explicit  attention  for  this  role  is  highly  understandable  in  view  of  the  substantial  decrease  of  central   level  subsidy  from  Dutch  Government  to  the  IA  that  is  foreseeable  in  the  not  too  distant  future.     Findings   23. From  the  various  conversations  it  did  not  become  clear  if  the  fundraising  role  concerns  mainly   fundraising  for  the  IA,  and,  if  so,  whether  or  not  that  role  is  compatible  with  fundraising  to   enhance  the  financial  sustainability  and  autonomy  of  partners  and  coalitions.   During  the  various  exchanges  with  IA  leadership,  a  sixth  role  surfaced  that  will  most  definitely  have   its  bearings  on  the  IA  way  of  working:  the  co-­‐implementer  role.  In  this  role,  the  IA  becomes  an  active   stakeholder  in  development  programmes  and  activities  and  will  bear  more  direct  responsibilities  for   the  production  of  development  outputs  and  outcomes.  The  driving  force  behind  wanting  to  play  this   role  seems  to  be  that  it  might  increase  the  IA  opportunities  for  fundraising.     Findings   24. The  fundraiser  and  co-­‐implementer  roles  might  result  in  a  direct  competition  between  the  IA,   its  partners  and  other  local  actors  on  the  decreasing  ‘development  funds  market’.       25. The  co-­‐implementer  role  bears  the  risk  of  redirecting  energy  and  attention  from  ‘supporting   local  development  processes’  to  ‘formulating  bankable  proposals  that  serve  the  donor’s   agenda’.    

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

18


MANAGEMENT TOOLS  FOR  THE  PROGRAMMATIC  APPROACH    The  IA  has  introduced  management  information  systems  (PME  tools  and  procedures,  financial  and   administrative  systems  and  formats)  that  are  especially  geared  towards  the  management  of  the   implementation  of  the  IA  business  plans.  Regular  information  is  gathered  on  the  status  of  contractual   and  financial  obligations  on  partners  and  the  IA  programmes  and  on  the  indicators  for  success  as   formulated  in  the  monitoring  protocol  agreed  upon  with  the  Dutch  Ministry  for  Development   Cooperation.     Findings   26. The  current  systems  do  not  provide  for  valid  and  regular  management  information  on   numbers  and  qualities  (and  the  evolution  thereof)  of  programmes  and  coalitions.  Timely   introduction  of  the  P-­‐Scan  would  have  improved  the  current  lack  of  quantitative  and   qualitative  management  information  on  entities  crucial  to  monitor  progress  on  the   implementation  of  the  programmatic  approach.   27. The  current  push  for  one-­‐year  contracts  does  not  help  in  creating  the  right  conditions  for  the   further  emergence  of  multi-­‐stakeholder  dynamics  and  action.     INITIAL  RECOMMENDATIONS   The  evaluators  have  had  the  honour  of  having  a  brief  look  in  the  kitchen  of  the  IA.  They  have   encountered  and  exchanged  ideas  with  very  dedicated  and  highly  professional  staff  and  leadership.   They  have  engaged  with  the  team  responsible  for  the  shaping  of  the  programmatic  approach  and  the   training  and  learning  thereon.  They  have  deliberated  with  the  staff  responsible  for  thematic  quality   and  coherence  and  with  the  men  and  women  that  have  to  translate  central  ambitions  to  the  day-­‐to-­‐ day  practice  in  field  reality.  And  they  have  conversed  with  the  four  directors  of  the  IA  that  are  there   not  only  to  inspire  and  motivate,  but  at  the  end  of  the  day  also  to  account  for  the  results  achieved.   The  challenges  are  vast:  realising  the  IA  ambitions  and  preparing  for  the  future  in  a  rapidly  changing   development  sector.   The  evaluators’  findings  and  recommendations  are  to  be  seen  in  this  light:  observations  and  thoughts   of  outsiders  that  fully  realise  the  different  roles  that  everyone  plays  in  a  large  organisation  as  the  IA   and  that  fully  recognise  that  under  pressure  many  a  thing  becomes  liquid.  Observations  and  thoughts   that  are  meant  to  support  the  IA  in  reflecting  on  strategic  issues  and  on  future  positioning.   In  summary,  the  introduction  of  the  programmatic  approach  in  the  IA  has  not  reached  its  full   potential  as  a  result  of:   • an  apparent  lack  of  a  common  and  widely  shared  understanding  on  the  meaning  of  the  key-­‐ concepts  of  the  approach;   • a  serious  and  unresolved  strategic  tension  between  implementing  an  ‘ideal  type  programmatic   approach’  and  realising  the  IA  business  plan  results;  or  to  rephrase:   • the  fact  that  at  conceptual  level  the  IA  has  not  been  able  to  properly  balance  ‘local  ownership’   with  the  alliance’s  own  interests  and  development  agenda;   • the  difficulties  (both  strategically  as  well  as  quality  and  time  wise)  in  fully  realising  two  of  the   four/six  IA  roles  (brokering  and  support  to  capacity  development);   • the  fact  that  the  IA  management  tools  and  HR  systems  have  not  been  geared  towards   information  on  and  steering  on  a  successful  implementation  of  the  programmatic  approach.   The  IA  leadership  is  currently  engaged  in  a  ‘multi  annual  strategic  planning’  process.  During  this   process,  the  IA  leadership  will  most  definitely  spend  some  quality  time  together  to  thoroughly  deal   with  a  number  of  key  issues  and  questions  with  regards  to  the  future  IA  positioning  and  issues  that   all  have  to  do  with  the  question  who  you  are  and  what  you  want  to  be:   • Are  the  IA  vision  and  mission  statements  still  up  to  date?  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

19


• • •

What are  the  defining  elements  or  key  traits  of  the  IA  as  a  Dutch  civil  society  organisation  with   its  roots  in  the  protestant  community?   Are  the  ways  and  means  in  which  the  IA  members  relate  to  and  communicate  with  their   constituencies  still  up  to  date?   What  is  the  added  value  and  specific  niche  of  the  IA  in  development  cooperation?  In  what  does   the  IA  distinguish  itself  from  the  numerous  other  INGO  active  in  (implementing)  development   (programmes)?   What  is  the  most  suitable  business  model  to  bankroll  the  above?  Do  we  follow  the  money  or  do   we  mobilise  money  for  what  we  want  to  do?  

Depending on  the  outcome  of  the  conversations  mentioned  above,  it  is  highly  recommendable  to:   1.

Deal with  the  two  major  strategic  issues  mentioned  in  this  report:   a. how  to  couple  the  ‘local  ownership  over  development  processes’  with  the  evident  own   institutional  interests  and  development  agenda  of  the  IA?   • define  ‘local  ownership’  and  indicate  what  the  IA  considers  to  be  within  the   boundaries  of  ‘local  ownership’;   • define  ‘partnership’  and  indicate  what  the  IA  considers  to  be  the  qualities  of  ‘true   partnership’;   • define  and  make  explicit  the  institutional  interest  of  the  IA  in  terms  of  branding,   national,  regional  and  global  presence,  minimum  staff  and  budget  levels;   • define  and  make  explicit  the  IA  development  agenda  in  terms  of  themes  and/or   processes  to  be  supported  and  in  terms  of  types  of  results  to  be  achieved.   b. is  the  IA  core-­‐business  about  ‘brokering  locally  owned  solutions  to  locally  owned  problems   within  local  settings  and  dynamics  following  local  agendas  and  rhythms’,  or  about   ‘facilitating  multi-­‐stakeholder  processes  to  support  joint  action  to  realise  results  on   programmes  as  indicated  in  the  IA  business  plans’?  

2.

Rephrase the  term  ‘programmatic  approach’  using  internationally  accepted  concepts  and  terms.   In  doing  so,  use  could  be  made  of  the  ‘pearls  of  the  IA  practice’  mentioned  in  this  report  and  to   firmly  anchor  these  pearls  in  a  widely  shared  understanding  of  ‘local  ownership’.  The  ensemble   of  these  ‘pearls’  could  be  simply  labelled  as  the  ‘IA  way  of  working’.  Clarifying  furthermore  once   and  for  all  the  concepts  ‘local  ownership’,  ‘programme’  and  ‘coalition’  and  how  these  are  to   relate  to  the  IA  themes  and  result  frameworks  is  of  key  importance.  

3.

Depending of  the  outcomes  of  the  conversations  indicated  above,  it  seems  advisable  to  redefine   the  IA  roles.  The  current  six  roles  position  the  IA  as  ‘a  jack  of  all  trades’,  in  practice  having   difficulties  to  deliver  high  quality  services  on  a  number  of  them.  The  suggestion  here  is  to  focus   on  the  roles  crucial  for  the  IA  core-­‐business.  And  that  could  be  either  in  for  example  the  co-­‐ funding  business  (the  IA  channelling  funds  from  its  constituencies  in  Europe  to  partner   organisations  in  the  South),  in  the  brokering  business  (the  IA  brokering  local  solutions  to  local   problems),  or  in  the  co-­‐implementation  business  (the  IA  as  one  of  the  key  actors  in  managing   the  implementation  of  development  programmes).  In  focussing  on  the  roles  crucial  for  the  IA   core-­‐business,  it  is  important  to  realise  that  the  ‘pearls  of  the  IA  way  of  working’  are  all   specifically  designed  for  and  do  indeed  all  favour  the  IA  in  broker  and  capacity  developer  roles.    

4.

Systems, tools  and  procedures  are  to  support  the  IA  in  implementing  its  core-­‐business,  in   maintaining  quality  standards  and  control  and  in  assuring  accountability  over  the  operations.  In   view  of  the  above,  it  is  highly  advisable  to  adapt  the  IA  management  information  systems,  the   PM&E  systems  and  the  HR  systems  to  fully  support  the  IA  way  of  working.  Formulating  quality   standards  for  the  IA  way  of  working  seems  a  prerequisite  for  this.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

20


5.

For the  second  phase  of  the  current  evaluation  assignment,  it  is  advised  to  undertake  three   fieldtrips  to  coalitions  (in  the  broadest  sense  of  the  word)  that  are  to  be  considered  interesting   and  successful.  The  main  task  for  the  evaluators  would  be  to  engage  with  all  concerned   (especially  the  participating  organisations,  but  also  IA  staff)  and  to:   a. identify  why  these  coalitions  are  considered  successful;   b. describe  the  process  of  their  genesis;   c. describe  what  they  do,  why  they  do  it  and  how  they  go  about  in  doing  it;   d. analyse  the  various  factors  that  have  contributed  to  their  success  (external  environment,   nature  and  composition  of  cooperating  partners,  type  and  quality  of  support  provided,   governance  arrangements,  planning  and  learning  dynamics,  funding  flows  and  mechanisms,   personal  qualities,  happenstance);   e. chart  the  factors  that  might  influence  the  future  success  and  the  viability  and  sustainability   of  the  coalition;   f. document  the  above  in  crisp,  brief  and  clear  papers  highlighting  best  practices  in  the  IA  way   of  working.  

It would  be  most  interesting  to  visit  three  coalitions  that  have  not  already  participated  in  the   appreciations  and  action  researches  or  have  been  otherwise  documented.       September  2013,  Verona  Groverman  &  Kees  Zevenbergen      

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

21


Annex A.   Timeline  Evolution  Key-­‐Concepts  Programmatic  Approach   Statements  on  key-­‐concepts     Critical  remarks  (all  from  documentation)   Business  Plan  2006-­‐2010   2006     ICCO  Alliance  way  of  working:   • context  analyses  country  &  theme   • stakeholder  &  drivers  of  change  analysis   • stakeholders  collaborate  in  a  programme   and  strive  for  change  in  power  relations   • stakeholder  agree  upon  monitoring,   learning  and  evaluation   Start  Paper  Programmatic  Cooperation     Start  Paper  Programmatic  Cooperation   The  programmatic  way  of  working  implies   The  concepts  ‘programme’,  ‘project’,  ‘theme’   combining  the  four  roles  (strategic  funding,   and  ‘programmatic  approach’  are  often  used   brokering,  capacity  development  and   interchangeably,  which  does  not  help  in   advocacy  &  lobby)  within  the  business  plan   clarifying  matters.   themes.   A  programme  is  a  consistent  combination  of   the  four  roles  in  a  specific  area.   The  Programmatic  Approach  in  the  ICCO   2007   Programmatic  Way  of  Working   Alliance   • give  more  time  for  introduction   By  a  programmatic  approach  we  mean  all  of   • improve  guidance  on  creation  of   the  interventions  during  a  certain  period  of   programmes   time  and  within  a  particular  geographical   • improve  linkage  of  PRO-­‐DE-­‐CO   area.  Organisations  with  different  areas  of   • increase  communication  on  PA   expertise  and  experience  participate  in  such  a   • improve  learning  on  PA   programme.  Together  they  try  to  reach   agreement  on  its  vision,  objective,   implementation  strategy  and  the  roles  played   by  the  various  organisations.   Programmatic  Approach  at  Work   2008   Programmatic  Approach  at  Work   The  word  “programme”  is  often  confusing  as   • The  programmatic  approach  influences  the   it  can  mean  many  things  and  therefore  can   dynamics  of  change  in  a  society   be  understood  in  many  different  ways.   • Local  ownership  is  a  crucial  and  decisive   The  separate  programmatic  departments  in   element!   ICCO/KIA  are  not  so  good  for  initiating  and   • Different  actors  join  forces  and  decide  to   support  to  multi-­‐thematic  programmes.   engage  into  something  new   • The  programmatic  approach  changes  the   way  in  which  partners  relate  to  their   society   • The  ICCO  Alliance  supports  the  initiation  of   southern  programme  coalitions   The  Programmatic  Approach   2009   The  Programmatic  Approach   The  main  organisational  strategy  for   Experience  shows  that  the  process  of  coming   cooperation  with  partners  that  ICCO  (and  to  a   to  a  joint  vision  and  objectives  and  the   different  degree  some  of  it’s  alliance   process  of  motivating  actors  to  become   partners)  have  adopted  is  the  programmatic   involved  in  a  programmatic  cooperation   approach.  Through  the  programmatic   process  is  lengthy  and  not  easy.     approach  the  ICCO-­‐Alliance  seeks  to  promote   The  ICCO  Alliance  does  not  intend  to  “build”   and  support  multi  stakeholder  co-­‐operation   ICCO  Alliance  programmes  but  intends  to   processes  needed  for  sustainable  reduction   promote  and  support  programmatic  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

22


of poverty  and  injustice.    

cooperation between  relevant  actors  in  a   context.   Appreciating  the  Programmatic  Approach   “My  overall  assessment  is  that  for  southern   partners  to  really  take  ownership  of  these   emergent  ‘organisations  of  organisations’  will   take  many  years  and  very  high  quality   support  and  accompaniment  by  those  who   have  initiated  them”  (JamesTaylor)   Evaluative  Study  ICCO  Programmatic   Approach   There  evolved  a  clearer  notion  of  a  difference   between  a  ‘programme’  (content)  and  the   Programmatic  Approach  (process).  While  in   some  cases  there  are  programmes  that  are   just  administrative  clusters  of  activities  and   contracts,  others  are  the  result  of  a  process   of  partners  defining  some  joint  strategies   with  actions.  All  the  variations  carry  the  title   ‘programme’,  which  therefore  has  made  the   concept  -­‐  as  a  common  denominator  -­‐  rather   useless.  

Evaluative Study  ICCO  Programmatic     Approach   Under  a  programmatic  approach  different   organisations  agree  to  work  towards  one   concrete  common  goal.  They  work  together   on  the  basis  of  a  common  vision,  core  values,   goal  and  a  common  strategy.  Though  every   actor  may  play  a  different  role  using  its  own   strength  and  networks,  the  end  result  of  their   collective  intervention  reaches  a  greater   impact  in  society.   The  four  key  principles  of  the  Programmatic   Approach  are  therefore:  1)  working  towards   a  common  goal;  2)  local  ownership  of  the   programme  and  its  governance;  3)  multi-­‐ stakeholder  involvement;  and  4)  added  value   of  the  different  actors  involved.   Synthesis  Paper  2010   2010   Synthesis  Paper  2010   The  Programmatic  Approach  is  defined  by   The  Programmatic  Approach  is  seen  as  an   ICCO  as  follows:  A  process  that  leads  to   element  of  the  decentralisation  process  and   organisations  working  together  based  on  a   associated  to  the  new  thematic  focus  of   joint  analysis,  shared  vision  and  objectives   ICCO.   and  clear  perspective  on  the  results  of  the   ICCO  took  the  role  of  initiator  of   cooperation.  In  such  a  process  all  actors  can   programmatic  cooperation.   do  different  things,  work  at  various  levels  and   This  has  led  to  a  great  variety  of  initiatives   use  their  own  strengths  for  the  common   under  the  heading  ‘programmatic  approach’   purpose  and  objectives,  as  well  as  share   or  ‘programme  building’.   some  activities  and  in  particular  share  and   participate  in  the  linking  and  learning   processes.  The  programmatic  approach  is  an   approach  that  does  not  only  address  single   problems  but  aims  at  change  of  systems.   Synthesis  Paper  2011   2011     The  current  practice  of  the  Programmatic   Approach  in  ICCO  shows  a  wide  variety  of   programmes  with  different  characteristics  on   the  content  (type  of  issue,  scope,  level),  the   composition  (partners,  non-­‐partners,   networks),  role  of  ICCO  during  the  process,   funding  and  governance.  Various  models  for   structure  and  governance  have  been   identified.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

23


Strategy Paper  Effective  Selling  Points   To  realise  its  mission  and  to  collaborate  with   different  actors,  a  programmatic  approach  is   basic  to  the  work  of  the  IA.  Programmatic   working  is  defined  by  the  IA  as  a  process   leading  to  multiple  stakeholders  working   together  on  joint  analyses  of  given   challenges,  shared  visions  and  clear   perspectives  and  purposes  on  the  results  of   the  collaboration.  The  IA  believes  that   bringing  together  joint  and  diverse   perspectives  is  the  only  way  to  alter   underlying  systems  in  society.   Guidance  Note  for  the  Programmatic   Approach   The  ICCO  Alliance’s  Programmatic  Approach   is  an  approach  that  is  about  emerging  forms   of  organisations  that  come  together  to   collaborate.  These  organisational  forms  are   known  by  different  names  such  as  coalitions,   alliances,  networks,  partnerships,  joint   ventures  or  federations.     In  this  paper  we  will  use  the  term  coalition  

Appreciating the  Programmatic  Approach  -­‐   debriefing  note   The  danger  of  PA  becoming  too  controlled,   linear  and  driven  by  predefined  results   should  be  avoided  and  continuously   monitored.   The  changes  within  the  organisation  have   increased  administrative  demands  on  the   programme  staff.   As  in  the  past  lack  of  clarity,  communication,   coaching  and  training  on  PA  has  caused  some   uncertainty  and  insecurity,  the  Global  Office   could  make  a  public  statement  recommitting   to  PA  and  programme  cooperation.  Clarity  on   the  approach  will  help  the  partners  to  see   the  added  value  of  cooperation  and  instil   motivation  among  the  ICCO  staff  on  the   ground.   Appreciating  the  Programmatic  Approach  -­‐   workshop  report   Initially  there  was  no  clear-­‐cut  guideline  for   PA,  which  caused  some  insecurity  and   uncertainty  among  the  partners,  and   ROs/POs.     Communications  about  the  PA  was  neither   continuous  nor  coherent.  Different   interpretations  of  PA  caused  differential   understanding  among  the  partners  and  POs.    

2012 Guidance  Note  for  the  Programmatic   Approach   In  the  ICCO  Alliance  we  have  initially  called   them  program  coalitions  or  even  shorter:   programmes.  This  last  term  is  however   confusing  because  it  is  also  used  for  the  ICCO   Alliance  policy  level,  for  a  set  of  objectives,   results  and  activities  (projects)  related  to  a   thematic  domain  and  for  the  cooperation   between  stakeholders  on  a  problematic.  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

24


for the  associative  form  of  organisations   working  together  for  the  realisation  of  a  joint   purpose.    

Synthesis Paper  2013   In  2012  emphasis  in  the  support  of  the   implementation  of  the  Programmatic   Approach  was  on  strengthening  the  capacity   of  the  actors  involved  to  cooperate  within  a   Programmatic  Approach  so  that  the   envisaged  and  joint  development  results  are   being  realised  within  the  identified  thematic   domains.        

Working Better  Together   The  different  roles  of  ICCO,  i.e.  being  a   broker,  capacity  developer  and  funder,  often   created  tension  between  ICCO  and  the   coalition.  The  inherent  ambiguity  that  is  the   consequence  of  an  international  donor   organisation  starting  coalition  initiatives,   while  at  the  same  time  aspiring  for  these   coalitions  to  become  independent  leads  to   several  challenges.  These  are  related  to   ownership,  coherence,  clarity  on   complementarity  and  roles,  and  the  relation   with  ICCO.  In  particular  the  funding  role  of   ICCO  creates  an  unequal  power  relation  with   the  coalitions.   2013   Synthesis  Paper  2013   Problematic  issues  on:   • maturity  of  coalitions   • learning  within  coalitions   • brokering  and  facilitation   • ownership   • funding  and  fundraising   • governance  of  coalitions  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

25


Annex B.   Documents  reviewed    (mostly  chronological)     DOCUMENTS  RELATED  TO  THE  DEVELOPMENT  OF  THE  PROGRAMMATIC  APPROACH     -­‐ HD.  12  Maart  2007.  Programmatisch  werken  in  de  ICCO  Alliantie.  Opzet  en  achtergronden     -­‐ No  author.  Spring  2007  (22-­‐6-­‐2007  last  saved).  Programmatic  Approach  of  the  ICCO  Alliance  (ppt)   -­‐ Derksen,  Harry.  4  juni  2007.  Voorstel  1.  Programmatisch  Werken   -­‐ Klarenbeek,    Mildred  &  Ooijens,  Machteld  2007?  Bijlage  III.  Systematisering  van  de  ervaringen   met  programmatisch  werken  in  de  Ruta  del  Sol,  een  onderwijsprogramma  in  Peru   -­‐ Derksen,  Harry.  14-­‐4-­‐  2008.  Programmatic  Approach  at  work.  A  second  update  spring  2008     -­‐ Walters,  Hettie.  14-­‐5-­‐2009.  The  Programmatic  Approach     -­‐ State  of  the  Art  PA  Aug  2009  v2   -­‐ Wortel,  Erica  &  Jouwert  van  Geene.  December  2009.  Consolidating  Experiences  of  ICCO’s   Programmatic  Approach,  An  evaluative  study  of  the  Programmatic  Approach  of  the  ICCO  Alliance     -­‐ No  author.  March  19,  2010.  How  the  U  Process  can  be  used  to  strengthen  a  Programmatic   approach  to  Development  Cooperation.  An  exploratory  conversation   -­‐ Walters,  Hettie.  28-­‐3-­‐2010.  Synthesis  paper  :  Findings  and  recommendations  gained  from  the   Evaluative  study  and  the  Appreciating  the  Programmatic  Approach  processes   -­‐ Walters,  Hettie.  December  2010,  Proposal  for  the  prolongation  of  the  Learning-­‐working  Path   (LWP)  of  the  ICCO  (Alliance)  Jan  2011-­‐  Dec  2012   -­‐ Walters,  Hettie.  31.1.2011.  Synthesis  paper  :  Findings  and  conclusions  from  the  Evaluative  study   and  the  Appreciating  the  Programmatic  Approach  processes.   -­‐ No  author.  3.2.2011.  Appreciating  the  programmatic  approach.  How  far  have  we  come?  Where   do  we  need  to  go?   -­‐ Walters,  Hettie.  2011.  Guidance  note  for  the  Programmatic  Approach  of  the  ICCO  Alliance   -­‐ Walters,  Hettie.  11-­‐4-­‐2012.  Monitoring    and  progress  meeting    report  based  on  the    Planning   Framework   -­‐ ICCO.  170912.  PROGRAMMATIC  COOPERATION  SCAN  (P-­‐SCAN)   -­‐ No  author.  May  2013?  Synthesis  of  observations    and  feedback  to  reporting  about  the  progress   with  the  implementation  of  the  Programmatic  Approach  in  2012   -­‐ Walters,  Hettie.  8-­‐5-­‐2013.  Prog.  Approach  rapportages  with  comments     -­‐ P&D,  June  2013.  Synthesis  of  observations  and  feedback  to  reporting  about  the  progress  with  the   implementation  of  the  Programmatic  Approach  in  2012     EVALUATIONS  AND  APPRECIATIONS    (by  type  of  evaluation/appreciation,  chronological)   -­‐ Pronk,  Hester  &  Hettie  Walters,  2009.  ICCO’s  Implementation  of  the  Programmatic  Approach:   What  can  Learning  Histories  Tell  Us?   -­‐ Ramírez    Eduardo,  2009.  Assessment  Report  for  the  Paraguay  Organic  Programme   -­‐ Taylor,  James.  November  2009.  Appreciating  the  programmatic  approach:  A    systematisation  of   experience.  Report  of  the  learning  from  ICCO  /  Kerk  in  Actie  Conflict  Transformation  Programme   Uganda   -­‐ Taylor,  James.  January  2011.  Appreciating  the  programmatic  approach:  A  systematisation  of   experience.  Second  Report  of  the  learning  from  ICCO  /  Kerk  in  Actie.  Conflict  Transformation   Programme  Uganda   -­‐ CB  =??  20.9.2010.  Report  for  ICCO:  draft  conclusions  and  recommendations   -­‐ Ramírez,  Eduardo  November  2010.  Report  on  Second  Round  of  Interviews  –  Paraguay  Orgánico   -­‐ Bruinsma,  Domien.  January  2011.  Report  on  the  2010  Appreciation  process  experiences  within   the  West  African  Organic  &  Fair  Trade  Cotton  programme     -­‐ Context,  international  cooperation.  February  28,  2011.  Appreciating  the  Programmatic  Approach;   a  systematisation  of  experiences.  Report  of  the  Workshop  January  31  –  February  4,  2011,  Utrecht  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

26


-­‐ No author.  4.3.2011.    Appreciating  Programmatic  Approach:  Systematisation  Of  Experiences.   Debriefing  Note  From  The  Workshop  Held  On  31  January  To  4  February  2011  At  Utrecht   -­‐ Bandyopadhyay  Kaustuv  K  .  February  2011?  Bangladesh  Water  and  Food  Security  Partnership   Programme  (BWFSP).  Appreciating  Programmatic  Approach  of  ICCO  Alliance,  the  Netherlands   -­‐ Caubergs,  Lisette.    Juin    2011.  Rapport    d’appréciation    de  «  l’Approche  Programmatique»  :   systématisation  de  l’expérience  pour  la  période  09/09  –  12/10   -­‐ Karssemeijer,  Ward.  14-­‐6-­‐2012.  End  report  Junior  Action  Researcher  Burkina  Fasso   -­‐ Mulder,  Hanske  &  Kees-­‐Jan  Mulder,  15-­‐06-­‐2012.  Final  Report.  Recherche  et  renforcement  des   champs  de  collaboration  entre  les  membres  entités  de  la  coalition  LIFE   -­‐ Jong,  Elja  de.  June  2012.  End  Report  Junior  Action  Researcher.  Programmatic  Approach  –  Agro   ecological  Consortium  Peru   -­‐ Punt,  Wievenlien.  June  2012.  Final  Report  Action  Research  Nepal  Coalition  on  Food  Security  and   Water     -­‐ Kruiter,  Ingrid.  14  June  2012.  Final  report  Action  Research  Programmatic  Approach  Youth  and   Security  Programme  Central  America  (Programa  de  Seguridad  Juvenil  Centroamérica)   -­‐ Jong,  Elja  de,  Ward  Karssemeijer,  Ingrid  Kuiter,  Hanske  Mulder,  Kees-­‐Jan  Mulder,  Wievenlien   Punt.  July  2012.  Working  better  Together.  Action  Research:  The  Programmatic  Approach  from  the   Perspective  of  the  Actors  Involved.   -­‐ No  authors.  2013.  Evaluation  of  PSO  Learning  Trajectories  Jan  2011  –  Dec  2012   -­‐ No  author.  May  2013?  Results  for:  Survey  for  Evaluation  of  the  Learning  and  Working  Process   activities  2011-­‐2012   TRAINING  REPORTS  (by  type  of  training,  chronological)   -­‐ General  Programme  Regional  Facilitators  ICCO.  16  –  20  November  2009.  Guest  House  ICCO   -­‐ Report  of  the  workshop  on  the  Programmatic  Approach  for  staff  of  ICCO  Alliance.  Programmatic   way  of  working  –  Programmatic  cooperation  –  Multi  Stakeholder  processes.  Block  1  (of  3)    17-­‐18   February  2011   -­‐ Report  of  the  workshop  on  the  Programmatic  Approach  for  staff  of  ICCO  Alliance.  Programmatic   way  of  working  –  Programmatic  cooperation  –  Multi  Stakeholder  processes.  Block  2  (of  3)  –  17-­‐18   March  2011   -­‐ Report  of  the  workshop  on  the  Programmatic  Approach  for  staff  of  ICCO  Alliance.  Programmatic   way  of  working  –  Programmatic  cooperation  –  Multi  Stakeholder  processes.  Block  3  (of  3)  –  13-­‐14   April  2011   -­‐ Report  of  the  workshop  on  the  Programmatic  Approach  for  regional  facilitators  of  ICCO  Alliance.   Programmatic  way  of  working  –  Programmatic  cooperation  –  Multi  Stakeholder  processes.   Utrecht,    8  -­‐  12  November  2010   -­‐ CDI.  May  2011.  A  series  of  training  on  methods  and  approaches  for  a  Programmatic  Approach   within  ICCO  Alliance.  A  proposal  for  the  ICCO  Alliance   -­‐ Walters,  Hettie.  Report  of  the  workshop  on  the  Programmatic  Approach  for  regional  facilitators  of   ICCO  Alliance.  Programmatic  way  of  working  –  Programmatic  cooperation  –  Multi  Stakeholder   processes.  Wageningen,    10  -­‐  14  October  2011   -­‐ Capacitación  de  Facilitatadores  Alianza  ICCO  –  Enfoque  Programático.  Lima  (Perú),  7  –  11   Noviembre  2011   -­‐ Walters,  Hettie.  Report  of  the  workshop  on  the  Programmatic  Approach  for  regional  facilitators  of   ICCO  Alliance.  Utrecht,  16  –  20  November  2011   -­‐ Brouwer,  Herman,  Hettie  Walters,  Dete  Aliah.  Programme  Overview  ICCO  Alliance  Programmatic   Approach.  Training  of  Trainers  Sanur  Bali,  Indonesia  27  February  –  2  March  2012   -­‐ Report  of  the  training  on  the  Programmatic  Approach  of  ICCO  Alliance.  Programmatic  way  of   working  -­‐  Programmatic  cooperation  -­‐  Multi  Stakeholder  Processes.  Bali.    27  February    –  2  March   2012   -­‐ Report  of  the  training  on  the  Programmatic  Approach  of  ICCO  Alliance.  Programmatic  way  of   working  -­‐  Programmatic  cooperation  -­‐  Multi  Stakeholder  Processes.  Kampala,  23  -­‐27  April  2012  

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

27


-­‐ No Author.  Induction  process  for  new  staff  of  RWO’s  1-­‐1-­‐2009  to1-­‐1-­‐2010   ADDITIONAL  DOCUMENTS  REVIEWED  (by  focus/item,  chronological)   -­‐ ICCO  Alliantie.  April  2006.  Bedrijfsplan  2007  -­‐  2010     -­‐ Derksen,  Harry.  16  december  2008.  Opzet  jaarverslag  2008   -­‐ Ooijens,  Machteld.  18  december  2009.  Opzet  jaarverslag  2009   -­‐ ICCO  Alliance.  Grant  Application  Phase  1  MFS  II   -­‐ ICCO  Alliance.  Grant  Application  MFS  II  Phase  2.  From  Aid  to  Entrepreneurship   -­‐ ICCO  Alliance.  27  January  2011.  Framework  letter  Reporting  &  Planning  2010  /  2011   -­‐ ICCO  Alliance.    27  January  2012.  Framework  letter  Reporting  &  Planning  2011  /  2012   -­‐ ICCO  Alliance.  28  November  2012.  Framework  letter  Reporting  2013   -­‐ Context,  international  cooperation  &  Strategic  Connections.  June  10,  2012.  Evaluation  of  the   ProCoDe  Pilot  in  Central  and  East  African  Region.  Draft  report  &  Annexes   -­‐ Phlix,  Geert.  2013  MTR  ICCO  Programme  2011-­‐2015.  Assessment  of  the  progress  made  and   results  reached  in  the  period  2011-­‐2012.  Draft  Reports  Southern  Africa,  Western  Africa,  Central  &   Eastern  Africa,  Central  and  South  Asia,  South  East  Asia,  Central  America,  Latin  America   -­‐ No  author.  7.6.2013.  Strategy  Paper  Effective  Selling  Points  ICCO/KerkinActie   -­‐ No  author,  no  date.  Manual  Monitoring  Tool   -­‐ Also   about   monitoring:   two   untitled   and   undated   notes   about   the   monitoring   protocol   and   one   ppt  (Decision  tree  for  existing  projects  –  how  to  use  the  Monitoring  Tool)     -­‐ 1558_Compte  rendu  21  25  aout  2009_EN_def  21  and  25  August  2009     Annex  C.   People  interviewed   Aart  van  de  Broek  –  Edukans;  Linking  pin  –  ICCO  Cooperation     Mariecke  van  der  Glas  -­‐  Regional  Manager  Nicaragua       Dieneke  de  Groot  -­‐  Coordinator  PME  unit,  FACTS  department,  ICCO/Kerk  in  Actie     Wim  Hart  -­‐  Executive  Board  ICCO  Cooperation     Helmke  Hofman  -­‐  Specialist  Education,  Edukans     Henk  Jochemsen  -­‐Directeur  Prisma       Kees  de  Jong  -­‐  Directeur  Edukans     Machteld  Ooijens    -­‐  Head  Policy  and  Development  department,  ICCO/  Kerk  in  Actie   Anke  Plange-­‐van  Well  -­‐Specialist  Health  and  HIV,  PRISMA     Piet  Postuma  -­‐  Specialist  CT&D,  ICCO/  Kerk  in  Actie   Pepijn  Trapman  -­‐  Regional  Manager  South  and  Central  Asia   Elly  Urban  –  Prisma;    Linking  pin  –  ICCO  Cooperation     André  Vording  -­‐  Coordinator-­‐  Specialist  FED,  ICCO/Kerk  in  Actie     Gerrit  de  Vries  -­‐    Regional  Manager  Southern  Africa     Marinus  Verweij    -­‐  Chairman  of  the  Executive  Board  ICCO  Cooperation   Hettie  Walters  -­‐  Coordinator  Capacity  development.  Policy  and  Development  department.  ICCO   Cooperation      

Annex 3  –  Report  of  the  first  phase  of  the  Evaluation  Programmatic  Approach

28


Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  -­‐  Survey   about  the  development  and  performance  of  coalitions   In  the  context  of  the  evaluation  of  the  programmatic  approach  of  the  ICCO  Alliance  a  survey  was   developed  to  gain  insight  in  the  stage  of  development  of  programmatic  cooperation  through  the     formation  of  coalitions.  We  followed  the  broad  definition  of  coalitions  given  in  the  Guidance  Note  for   the  Programmatic  Approach:  ‘all  formal  or  informal  forms  of  cooperation,  alliances,  networks,  etc.’,   which  we  indicated  as  ‘coalitions’.  The  survey  was  meant  for  programme  officers  of  the  ICCO  Alliance   (ICCO/Kerk  in  Actie,  Edukans  and  Prisma)  and  representatives  of  ‘coalitions’.  The  survey  was  made  up   of  statements  to  which  the  respondent  could  (fully)  agree,  to  some  extent  or  (fully)  disagree,  posed   in  three  languages:  English,  French  and  Spanish.  In  Box  1  you’ll  find  the  email  which  invited  POs  and   representatives  of  a  coalition  to  fill  out  the  survey  as  well  as  the  introduction  section  to  the  survey.   In  this  report  we  present  the  findings  in  nine  sections  following  the  sections  in  the  survey.  At  the  end   (in     section  X)  we  attempt  to  answer  the  relevant  questions  of  the  TOR.  Attached  to  the  report  is  a  pdf   with  the  overall  answers  and  remarks/  clarifications  of  the  respondents  and  a  pdf  which  shows  a   differentiation  between  the  5  themes.  For  the  analysis  use  is  made  of  three  cross-­‐tables:  status  of   the  coalition  at  its  start  (‘based  on  an  existing  network,  coalition,  alliance  and  the  like’,  versus   ‘formation  on  instigation  of  ICCO  Alliance’);  the  theme  the  coalition  focuses  on  (FED,  FNS,  Basic   Box  1:  Email  inviting  POs/  representatives  of  a  coalition  to  fill  out  the  survey  and  introduction  section  of  the   survey   Email   Dear  program  officer/  representative  of  a  coalition  supported  by  the  ICCO  Alliance,     On  request  of  the  ICCO  Alliance  we  are  evaluating  the  Programmatic  Approach  (ICCO/Kerk  in  Actie;  Prisma;   Edukans)  to  help  the  Alliance  in  its  policy  development.  We  try  to  involve  as  m any  people  as  possible  to  learn   about  the  implementation  of  the  Approach.  One  of  the  tools  we  use  is  a  survey.    Since  you  are  key  actors  in   the  application  of  the  Approach  we  very  much  appreciate  to  hear  your  views.  Therefore,  we  kindly  invite  you   to  fill  out  the  online  survey.  We  estimate  that  it  takes  not  more  than  20  minutes  to  complete  the   questionnaire.     Please  click  on  the  following  link    …  to  access  the  survey  and  SUBMIT  THE  QUESTIONNAIRE  BEFORE  ..   AUGUST.     We  understand  that  we  ask  quiet  some  input  from  your  side  and,  therefore,  we  thank  you  very  m uch  for  your   cooperation!   The  evaluation  team,   Verona  Groverman   Kees  Zevenbergen     Introduction  section  of  the  survey   Background  to  the  survey   is  survey  is  part  of  the  evaluation  of  the  programmatic  approach  of  the  ICCO  Alliance.  Presently,  the  ICCO   Alliance  does  not  yet  have  a  good  overview  of  the  stage  of  development  of  the  coalitions  it  supports.  By   ‘coalition’  we  mean  formal  or  informal  forms  of  cooperation,  alliances,  networks,  etc.       The  survey  is  meant  for  program  officers  and  representatives  of  coalitions.  It  is  composed  of  9  sets  of  mostly   short  multiple  choice  questions.  Please  note  that  a  higher  score  does  not  automatically  mean  ‘better’  or  ‘more   advanced’!       We  estimate  that  it  will  not  take  you  more  than  20  minutes  to  complete  the  questionnaire.  Please  submit  the   questionnaire  by  clicking  the  Send/Submit  button  at  the  end  BEFORE  15  AUGUST.  We  will  respect   confidentiality.  The  results  of  the  questionnaire  will  be  anonymised  so  that  no  individual  or  organisation  will   be  identified.   Thank  you  very  much  for  your  time  and  energy,  we  much  appreciate  your  contribution  to  this  survey!  

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

29


health and  HIV  AIDS,  Conflict  transformation  and  Democratization,  Basic  Education10);  and,  lastly,   member  of  the  ICCO  Alliance  (ICCO/  Kerk  in  Aktie,  Prisma  and  Edukans).    Where  relevant,  interesting   or  striking  differences  taken  from  these  cross-­‐tables  have  been  added  to  the  findings.    However,  it   appeared  difficult  to  draw  hard  conclusions  about  the  themes  because  both  the  total  number  of   ‘coalitions’  per  theme  and  the  number  of  scores  given  for  certain  statements  per  theme  were   generally  too  small.         Section  I:  General  information     The  survey  was  sent  out  to  59  POs  and  103  representatives  of  ‘coalitions’,  162  persons  in  total11,  on  6   August  and  could  be  filled  out  up  to  24  August  (a  reminder  was  sent  to  those  who  had  not  reacted   yet  on  16  August).  These  162  persons  are  involved  in  95  coalitions  totally.     Overall  41  persons  submitted  the  survey,  121  did  not  react12.  The  response  was  as  follows13:   • 18  representatives  of  coalitions  responded,  involved  in  17  coalitions.     • 13  POs  filled  out  the  survey  concerning  12  coalitions.   • In  five  (5)  cases  both  a  representative  and  a  PO  filled  out  the  survey  for  the  same  coalition.     Concluding,  the  responses  of  the  survey  covered  34  coalitions.  The  names  of  these  coalitions  can  be   found  in  annex  as  well  as  those  coalitions  about  which  the  survey  did  not  receive  information.  In   terms  of  received  questionnaires  the  response  rate  is  25,3%  (30,5%  of  the  POs;  23,3%  of  the   representatives  submitted  the  survey).  More  important  though,  is  the  response  rate  in  terms  of   coalitions  involved  since  we  are  specifically  interested  in  the  development  and  performance  of   coalitions:    35,8%.  The  picture  that  can  be  sketched  based  on  the  outcomes  of  the  survey  can  be   considered  reasonably  representative  of  the  coalitions  supported    by  the  ICCO  Alliance.       Main  features  of  responses:   1.  ‘Coalitions’  are  a  mix  of  organisations  (respondents  could  tick  more  than  one  option).  Over  half   of  the  actors  involved  in  the  ‘coalitions’  concerned  are  partners  of  the  ICCO  Alliance  as  shown  in   figure  1,  followed  by  other  NGOs,  not  being  partners  of  the  ICCO  Alliance  (20%)  and  government   institutions  (12%).  Table  1  shows  the  responses  per  category  of  actors.   Table  1  -­‐  Actors  involved  in  the  ‘coalition’   reactions  (%)     Partners  of  members  of  the  ICCO  Alliance  (IA   92,7%   Other  NGOs,  not  being  partners  of  the  ICCO  Alliance   36,6%   Private  sector/business  organisations   14,6%   Government  institutions     22,0%   Other  (such  as  knowledge  centres,  etc.   17,1%   question  answered    

reactions (#)   38   15   6   9   7   41  

10

We did not included the theme Fair Climate in the cross table and the analysis of the themes because only one respondent dealt with this theme. 11 The figure is excluding the email address that bounced. 12 One respondent answered 9 of the 15 questions only and it not included in the 41. 13 These figures are not based on the responses in the questionnaire (question 1) but on the list of respondents provided by Survey Monkey. Taking this list made it possible to trace the coalitions in which the representatives and POs were involved and to find out whether or not a PO and representative had assessed the same coalition. Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

30


Figure 1 - Actors involved in the coalition

Other (such  as   knowledge  centres,   etc)   9%   Government   ins•tu•ons   12%   Private  sector/ business   organisa•ons   8%   Other  NGOs,  not   being  partners  of  the   ICCO  Alliance/     20%  

Partners of  members   of  the  ICCO  Alliance   (IA)   51%  

2. The  34  ‘coalitions’  (some  of  which  regional  ‘coalitions’)  concerned  are  mostly  located  in  Africa   (19).  Others  are  active  in  Asia  (9),  Central  Asia  (2),  South  America  (3)  and  in  the  Middle  East  (1)   (See  Annex).     3. Differentiated  according  to  member  of  the  ICCO  Alliance  we  see  the  following  picture  –  figure  2.   ‘Coalitions’  supported  by    ICCO/Kerk  in  Actie  include  private  sector  organizations  and  more   government  institutions  and  other  NGOs  than  the  other  Alliance  members.  In  table  2  the  actors   of  the  ‘coalitions’  concerned  per  theme  are  given.      

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

31


Figure 2  -­‐  Actors  involved  in  the  coalijon  per  member  of  the  IA  

Asjtel

45 40   35   30   25   20   15   10   5   0  

Partners of   Other   Private   Governme members   NGOs,  not   sector/ nt   of  the   being   business   ins•tu•on ICCO   partners  of   organisa•o s   Alliance     the  IA   ns  

Edukans

4

4

0

0

1

Prisma

12

3

0

2

1

ICCO/Kerk in  Ac•e  

23

10

6

7

6

Other (such  as   knowledge   centres,   etc)  

Table 2  -­‐  Actors  involved  in  the  ‘coalition’   Fair   Food  and   Basic   Conflict   Basic   Economic   Nutrition   Health   Transform Educatio reactions   reactions   Develop-­‐ Security   and   ation  and   n   (%)   (#)   ment     HIV   Democrati AIDS   -­‐zation  

Partners of  members  of   8   6   13   4   4   92,1%   35   the  ICCO  Alliance  (IA)   Other  NGOs,  not  being   4   0   3   3   4   36,8%   14   partners  of  the  IA   Private  sector/business   4   0   1   0   0   13,2%   5   organisations   Government   4   0   3   1   0   21,1%   8   institutions   Other  (such  as   4   0   0   1   1   15,8%   6   knowledge  centres,  etc)     4. The  ‘coalitions’  concerned  are  mainly  supported  by  the  following  members  of  the  ICCO  Alliance:   ICCO/  Kerk  in  Actie  (60%  or  24  of  the  responses),  while  30%  (12  responses)  is  supported  by   Prisma  and  15%  (6  responses)  by  Edukans.         5. The  majority  of  the  ‘coalitions’  are  formed  on  instigation  of  (members  of)  the  ICCO  Alliance   (75,6%  of  the  responses)  –  see  table  3.  This  holds  for  all  the  members  of  the  IA,  only  four  of  the   24  ‘coalitions’  supported  by  ICCO/Kerk  in  Actie  are  said  to  be  formed  based  on  existing   coalitions  or  networks,  three  of  which  focusing  on  FED  and    one  on  conflict  transformation.        

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

32


Table 3  -­‐    Origin  of  the    ‘coalition’   reactions  (%)   reactions  (#)     the  ‘coalition’  is  based  on  a  coalition,  network  or  alliance  that   already  existed  before  the  creation  of    the    ‘coalition’  supported   12,2%   5   by  members  of  the  ICCO  Alliance   the  ‘coalition’  has  been  organized  on  instigation  of  the  members   75,6%   31   of  the  ICCO  Alliance   Other     12,2%   5     6. Basic  health  is  the  theme  on  which  most  ‘coalitions’  concerned  focus  as  shown  in  Figure  3,   followed  by  Fair  Economic  development  and,  in  equal  percentage,  Basic  education  and  Food  and   nutrition  security.  The  ‘coalitions’  in  this  survey  supported  by  Edukans  (evidently)  focus  on  Basic   education,  those  of  Prisma  on    Basic  Health  and  HIV  AIDS    while  those  supported  by  ICCO/Kerk   in  Actie  focus  on  a  mix  of  themes  (Table  4).   Other   5%  

Figure 3  -­‐  Theme  of  the  ‘coalijon’s’  acjvijes  

Conflict Transforma•on   and   Democra•za•on   10%  

Basic Educa•on   15%  

Fair Economic   Development   19%  

Basic Health  And   HIV  AIDS   34%  

Fair Climate     2%  

Food and   Nutri•on   Security   15%  

Table  4  -­‐    Theme  of  your  ‘coalition’s’  activities  per  member  of  the  IA   ICCO/Kerk  in  Actie   Prisma   Edukans     Fair  Economic  Development   7   0   0   Fair  Climate     1   0   0   Food  and  Nutrition  Security   6   0   0   Basic  Health  and  HIV  AIDS   4   10   0   Conflict  Transformation  and   4   0   0   Democratization     Basic  Education   0   2   6   Other   2   0   0      

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

33


Section II:  Participant  diversity  and  cooperation  with  other  actors     Table  5  below  shows  the  responses  to  six  statements  concerning  the  diversity  of  the  participants  of   the  ‘coalition’  and  cooperation  with  other  actors.    What  can  we  observe?   1. First,  how  were  the  potential  actors  of  the  ‘coalition’  identified?  According  to  most  respondents   the  initial  actors  of  the  ‘coalition’  carried  out  a  stakeholder  analysis  to  identify  potential  co-­‐ operating  parties  (statement  a:  39,0%  score  fully  agree  +  24,3%    agree  =  63,3%).    A  slightly  lower   percentage  (58,5  %)    states  that  this  analysis  was  used  to  form  the  ’coalition’    (statement  b:   26,8%  fully  agree  +  31,7%  agree).       2. The  growth  of  the  ’coalition’  shows  a  mixed  picture:  some  have  grown  organically,  others  not   (statement  c).    The  survey  did  not  allow  to  dig  deeper  in  this  issue.  Above  (table  3)  we  have  seen   that  most  ‘coalitions’  have  been  formed  on  instigation  of  members  of  the  ICCO  Alliance  but  not   how  it  developed  further  nor  how  existing  networks  developed  further.  In  Box  2  some  brief   examples  of  respondents  are  given  on  how  ‘coalitions’  developed.     Box  2  -­‐  Additional  explanations  of  some  respondents  on  how  the  ‘coalition’  developed   -­‐

-­‐

-­‐

-­‐

The way  in  which  the  coalition  was  formed  was  driven  very  much  from  the  perspective  of  'this  is  a   requirement  in  order  to  receive  funding'.  At  this  stage  of  formation,  there  was  no  understanding  of   what  the  programmatic  approach  was  nor  use  of  Theory  of  Change  or  other  MSPs.  Also  the  funded   partners  are  spread  across  very  big  geographic  areas  and  in  different  provinces  so  collaboration  on   delivery  was  hardly  possible.  So  we  did  the  best  we  could  and  formed  an  alliance  around  learning   which  has  actually  worked  rather  well  and  we  are  set  to  continue  post  2015  and  are  looking  at   expanding  membership  and  funding.  A  number  of  partners  start  working  together  during  the  MFS  1   period,  they  formed  X.  X  grow  and  was  reorganized  (open  network  organization)  into  Y.     The  coalition  in  X  started  with  9  partner  organisation,  over  a  period  of  two  years  the  number  had   dropped  to  5  partners.  however,  the  coalition  has  now  started  linking  up  with  other  networks  of   donor  organisations  and  also  mobilize  the  organisation  that  had  dropped.  The  reason  for  earlier   dropping  includes  end  in  funding  from  ICCO  alliance  for  the  particular  organisation.  The  coalition   partner  organisation  have  a  well-­‐established  relationship  with  the  local  governments  in  their  districts   of  operation.   En  effet,  le  réseau  du  programme  multi  acteur  d'éducation  non  formelle  est  mis  sous  fond   baptismaux  depuis  l'année  2007.  Tout  au  début  les  organisations  travaillaient  avec  le  même   partenaire  mais  de  manière  isolé  et  chacun  son  domaine  d'intervention  et  des  thématiques   similaires.  Sentant  la  nécessité  de  se  regrouper,  les  organisations  de  la  société  civile  sous  l'impulsion   de  ICCO  ont  trouvé  opportun  de  créer  le  programme  multiacteur  dont  l'objectif  est  de  lutter  contre   la  pauvreté  en  s'appuyant  sur  les  leviers  de  l'éducation  non  formelle,  la  promotion  de  filières   porteuses,  la  chaîne  des  valeurs  et  la  promotion  de  la  santé  de  la  reproduction.     With  help  of  ICCO,  the  Coalition  was  initiated  by  its  partners,  who  have  been  working  on  the  theme   of  Basic  Health  and  HIV/AIDS.  Later,  the  linkage/networking  has  been  expanded  with  other  like-­‐ minded  network  together  movement  to  achieve  shared  goal.  

3. According  to  all  respondents  their  ’coalition’  is  composed  of  multi-­‐type  actors  and  multi-­‐level   actors.  This  is  confirmed  by  what  was  shown  earlier  in  figure  1.  The  respondents  have  provided   all  names  or  abbreviations  of  the  actors  involved  (not  included  in  this  report).           4. Concerning  cooperation  between  activities  of  the  ’coalition’  and  other  existing  networks   working  on  the  same  problematic,  most  respondents  to  some  extent  or  fully  agree  that  this   happens  (statement  f,  score  3,  4,  5).      

Statements  

Table 5  -­‐  Participant    diversity  and  cooperation  with  other  actors   1     3   5     not   4   reactions   (fully   2   (to   (Fully   relevant/   (agree)   (#)   disagree)   some     agree)   do  not  

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

34


extent) a.  The  initial  actors  of  the   ‘coalition’  carried  out  a   stakeholder  analysis  to  identify   potential  co-­‐operating  parties   b.  The  ’coalition’    has  been   formed  on  the  basis  of  a   stakeholder  analysis   c.  The  ’coalition’    has  grown   organically,  starting  with  about   two  partners  and  over  time   increasing  the  number  and  type  of   actors   d.  The  ’coalition’  is  composed  of   multi-­‐type  actors   e.  The  ’coalition’  is  composed  of   multi-­‐level  actors   f.  The  ’coalition’  carries  out   activities  in  close  cooperation  with   other  existing  networks,  working   on  the  same  problematic  

know

3

4

7

10

16

1

41

4

5

8

13

11

0

41

8

10

6

8

9

0

41

5

2

5

9

18

2

41

4

4

9

9

13

2

41

2

6

10

12

10

1

41

Section  III:  Implementation  and  implementation  capacity  of  the  ‘coalition’   Through  a  few  statements  we  tried  to  get  an  impression  of  how  the  ‘coalitions’  plan  and  implement   their  activities.  Table  6  shows  the  scores  given  by  the  respondents  on  various  statements,  which   were  mostly  positively  assessed  (score  4).    Our  main  observations  are  the  following.   1. It  appears  that  most  members  of  the  ’coalitions’  organise  their  planning  based  on  a  logframe   (statement  a).  In  three  cases  such  a  planning  does  not  exist  (score  1).  Less  ’coalitions’  have  an   organised  planning  based  on  a  Theory  of  Change  (statement  b).    There  could  be  different   explanations  of  these  outcomes.  For  example,  the  ‘coalitions’  now  use  a  Theory  of  Change  (after   the  trainings  conducted  by  the  Alliance)  while  previously  using  a  logframe.  Or,  they  do  both   following  requirements  of  different  donors.  One  respondent  explained:  Our  work  as  a  coalition   is  very  shaped  and  bounded  by  the  MFS  II  Results  Framework.  However,  we  are  now  starting  to   think  about  and  plan  work  outside  of  this  framework.     2. The  majority  of  the  respondents  (65,8%)  agree  that  the  ’coalition’  reaches  its  targets  and   objectives  (score  4,  statement  c).  Less  (36,6%)  indicate  that  the  ‘coalition’  has  a  monitoring   system  in  place  that  is  used,  while  29,2%  does  have  not  a  proper  system  in  place/does  not  use  it.   Looking  at  the  start  situation  of  the  ‘coalition’  less  ‘coalitions’  that  are  based  on  existing   networks,  coalitions  etc.    indicate  to  have  a  monitoring  system  in  place  that  is  used  than  those   formed  on  instigation  of  the  IA  ,  but  the  total  numbers  are  too  small  to  draw  hard  conclusions  (it   concerns  5  ‘coalitions’  in  total).     3. Most  of  the  respondents  agreed  on  the  statement  staff  of  all  actors  in  the  ’coalition’  has   sufficient  expertise  to  address  the  issues  on  which  the  cooperation  focuses  (statement  e).  It  may   be  well  in  line  with  their  perceptions  on  the  previous  statements  mentioned  under  point  2:   planning  is  well  organised  and  targets  /  objectives  are  reached.          

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

35


4. The statements  on  how  activities  are  being  implemented  appeared  confusing:  both  the   statement  (f)    Activities  planned  by  the  ’coalition’  are  carried  out  by  individual  actors  and  (g)   Most  activities  planned  by  the  ’coalition’  are  carried  out  jointly  got  the  highest  response  in  score   4.  Most  respondents  (58,5%)  were  positive  about  the  flexibility  in  which  the  ‘coalition’  adapts  its   activities  based  on  learning  (statement  h  -­‐  score  5,  3  and  4  in  decreasing  response),  but  still   26,8%  scored  to  some  extent:  3.  In  Box  3  a  few  additions  by  respondents  are  given.     Box  3  -­‐  Additional  explanations  of  two  respondents  on  implementation  issues   -­‐

-­‐

La coalition  est  structurée  de  manière  qu'il  est  mis  en  place  d'un  comité  de  pilotage  et  d'une   commission  technique.  Pour  une  meilleure  coordination  des  activités  de  la  coalition,  le  comité  de   pilotage  a  mis  en  place  une  équipe  chargée  de  mettre  en  oeuvre  la  mission  déclinée.  il  s'agit  de  la   coordination  qui  a  en  charge  de  développer  des  activités  de  plaidoyer  lobbying,    de  renforcement  des   capacités  des  acteurs  et  de  suivi  évaluation  des  activités.  la  coordination  rend  compte  au  comité  de   pilotage  périodiquement  de  ses  activités  qui  fera  l'objet  de  validation.   The  coalition  implements  commonly  agreed  activities  using  the  existing  technical  expertise  among  the   member  organisations.  It  is  also  common  for  partner  organisation  to  share  expertise  e.g.  in  training,   follow  up  etc.  Although  the  individual  organisations  have  their  monitoring  systems,  a  joint  monitoring   mechanism  has  not  been  fully  developed  (indicators  were  developed).  

5. About  half  of  the  respondents  state  that  mechanisms  for  accountability  towards  target-­‐groups,   clients  or  grassroots  representatives  are  in  place,  but  still  26,8%  scored  to  some  extent  (3).    One   respondent  added  that  one  partner  uses  the  Client  Satisfaction  Instrument.   Table  6  -­‐  Implementation  and  implementation  capacity  of  the  ‘coalition’   not   1     5     4   relevant;   (fully   2   3   (Fully     (agree)   do  not   disagree)   agree)   know/   a.  The  ’coalition’    has  a  well   organised  planning  based  on  a   3   6   9   8   14   1   logframe   b.  The  ’coalition’    has  a  well   organised  planning  based  on  a   1   5   13   15   7   0   Theory  of  Change   c.  The  ’coalition’  reaches  its   0   3   5   27   6   0   targets  and  objectives   d.  The  ‘coalition’  has  a  monitoring   2   10   9   15   5   0   system  in  place  that  is  used   e.  Staff  of  all  actors  in  the   ’coalition’  has  sufficient  expertise   1   3   8   18   11   0   to  address  the  issues  on  which  the   cooperation  focuses   f.  Activities  planned  by  the   ’coalition’  are  carried  out  by   6   3   9   15   7   1   individual  actors   g.  Most  activities  planned  by  the   2   8   7   13   11   0   ’coalition’  are  carried  out  jointly   h.  The  ‘coalition’  flexibly  adapts  its   set  of  activities  based  on  learning   1   5   11   10   14   0   on  what  works  and  does  not  work   i.  Mechanisms  for  accountability   towards  target-­‐groups,  clients  or   0   8   11   12   10   0   grassroots  representatives  are  in   Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

reactions (#)  

41

41 41   41  

41

41 41   41  

41

36


place     Section  IV:  Attention  to  gender  equality     The  survey  included  four  statements  to  identify  the  extent  to  which  the  ‘coalitions’  are  paying   attention  to  gender  equality,  a  cross-­‐cutting  theme  of  the  ICCO  Alliance  reflected  in  a  policy.  14  Table   7  gives  the  results.  We    like  to  point  to  the  following  results.     1. Concerning  a  clear  vision  on  gender  equality  of  the  ‘coalition’  the  respondents’  scores  mostly   vary  between  3,    4  and  5:    26,8%,    31,7%,    and  31,7%  respectively  (statement  a).  We  see  the   same  variation  on  the  ‘implementation’  of  such  a  vision  –  efforts  to  decrease  gender  inequality  -­‐   with  only  slightly  different  responses:  24,3%,  31,7%  and  34,1%  (statement  b).    Monitoring  the   impact  of  its  activities  on  gender  equality  is  not  done  by  all  ‘coalitions’:  most  (41,4%)  score  ‘to   some  extent  we  do  so’  (score  3),  and  36,5%  scores  4  +  5:  we  (fully)  agree  that  we  monitor.  In  Box   4  five  reactions  of  respondents  are  added  which  shows  the  variation  between  the  ‘coalitions’.       Box  4  –  Additions  by  five  respondents  on  gender  equality   -­‐ -­‐ -­‐ -­‐ -­‐

-­‐

Cooptation de  X  spécialiste  du  gender  in  value  chain   The  coalition  has  not  yet  reflected  on  the  gender  dimension  as  common  area  of  concern.  Individual   partner  organisations  have  specific  actions  that  are  geared  towards  addressing  the  gender  inequality   La  vision  stratégique  de  la  coalition  repose  essentiellement  sur  l'équité  et  l'égalité  de  genre  en  termes  de   contribution  à  la  stratégie  nationale     pour  l'équité  et  l'égalité  de  genre  au  X.  C'est  pour  cette  raison  que  le  but  est  de  lutter  contre  les   inégalités  sociales  et  la  pauvreté  des  jeunes  et  des  femmes.     One  of  the  weak  points,  but  at  present  a  gender  specialist  is  involved  as  well  as  a  gender  specific   organization  as  a  member  of  the  platform.       Gender  is  a  cross  cutting  issue  for  the  coalition  and  each  coalition  member  has  a  gender  based  policy  to     establish  a  gender  equality  system  in  the  respective  organization.  

2. The  number  of  ‘coalitions’  that  include  actors  with  a  track  record  in  dealing  with  gender   inequality  is  almost  the  same  for  the  score  2  –  5,  about  23%  each,  in  other  words,  there  is  much   difference  between  the  ‘coalitions’.      

14

Unlearning gender. ICCO Alliance gender policy. 2010

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

37


Table 7  -­‐  The  extent  to  which  the  vision,  objectives  and  strategy/  activities  of  the  ‘coalition’   include  attention  to  inequality  between  the  two  genders   not   1     5     4   relevant;   reactions   (fully   2   3   (Fully     (agree)   do  not   (#)   disagree)   agree)   know   a.  The  ’coalition’  has  a  clear   vision  on  equality  between   1   3   11   13   13   0   41   men  and  women   b.  The  activities  of  the   ‘coalition’  include  efforts  to   0   4   10   13   14   0   41   decrease  inequality  between   men  and  women   c.  The  ’coalition’  monitors  the   impact  of  its  activities  on   1   7   17   9   6   1   41   inequality  between  men  and   women   d.  The  ’coalition’  includes   actors  with  a  track  record  in   3   8   10   11   9   0   41   dealing  with  gender  inequality     Section  V:    Use  of  a  rights-­‐based  perspective     Similar  issues  as  for  gender  equality  were  raised  concerning  the  use  of  a  rights-­‐based  perspective.   ICCO  adheres  to  the  APRODEV  formulated  RBA  policy.15  Attention  to  RBA  is  also  paid  in  the  Alliance’s   gender  policy.  The  scoring  on  the  issues  can  be  found  in  table  8.  Some  main  observations:   1. Concerning  a  clear  vision  the  respondents  score  a  bit  lower  than  on  gender  equality:  the  majority   scored  3  and  4:  34,1%  and  31,7%  respectively.  According  to  the  respondents  most  activities  of   the  ‘coalition’  include  efforts  to  strengthen  the  capacities  of  rights  holders,  especially  vulnerable   groups:  78%  for  score  4  and  5  combined.       2. A  number  of  respondents  13  (31,7%)  fully  agree  that  activities  of  the  ‘coalition’  include  efforts  to   hold  duty  bearers  accountable,  which  makes  56%  when  score  4  and  5  are  combined.  But  the   number  that  score  ‘we  do  this  to  some  extent  ‘(score  3)  is  still  high:  15  (36,6%).     3. The  highest  score  concerning  the  ’coalition’s’  monitoring  of  the  impact  of  its  activities  on   vulnerable  groups  is  4,  namely  51%.    

15

http://www.aprodev.eu/files/Development_policy/Dev-RBA/Rights-Position-Paper_E-2008.pdf

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

38


Table 8  -­‐  To  what  extent  does  the  ’coalition’    use  a    rights  based  perspective  

a.  The  ’coalition’  has  a  clear   vision  on  the  obligations  of   duty  bearers  and  the   entitlements  of  rights  holders,   especially  of  vulnerable   groups   b.  The  activities  of  the   ‘coalition’  include  efforts  to   strengthen  the  capacities  of   rights  holders,  especially   vulnerable  groups   c.  The  activities  of  the   ‘coalition’  include  efforts  to   hold  duty  bearers   accountable   d.  The  ’coalition’  monitors  the   impact  of  its  activities  on   vulnerable  groups  

3

4 (agree)  

5   (Fully   agree)  

not relevant;   do  not   know  

reactions (#)  

2

14

13

10

0

41

1

0

8

14

18

0

41

1

2

15

10

13

0

41

1

1

11

21

6

1

41

1   (fully   disagree)  

2

2

Section  VI:  Ownership  and  complementarity  of  the  ’coalition’   In  the  survey  statements  have  been  included  to  better  understand  the  actors’  commitment  to  the   ‘coalition’.  Issues  such  as  decision  making,  feeling  of  ownership  and  responsibility,  the  relation   between  the  own  organization  and  the  ‘coalition’  are  helpful  indicators  in  this  respect.  An  overview   of  the  scores  is  given  in  table  9.     1. The  highest  scores  for  all  the  statements  are  found  under  score  4  and/or  5.  Taking  score  4  and  5   together  51%  of  the  respondents  say  that  decisions  within  the  ’coalition’  are  taken  without  prior   consulting  of  the  ICCO  Alliance  and  even  more  that  a  feeling  of  responsibility  exists  for  the   cooperation  (68%).  In  Box  5  some  additions  by  respondents  can  be  read.    65,8%  of  the   respondents  state  that  actors  of  the  ’coalition’  feel  ownership  of  their  cooperation  and  its  results   (combined  scoring  on  4  and  5).       Box  5  –  Additions  by  respondents  on  decision  making     -­‐ -­‐

-­‐

Cependant, il  faut  préciser  que  certaines  décisions  concernant  les  éléments  contractuels  nécessitent  la   consultation  préalable  de  l'Alliance  ICCO  car  c'est  contractuel.     Il  est  vrai  que  l'alliance  X  est  créée  mais  elle  dépend  entièrement  de  ICCO  encore  pour  développer  ses   actions  sur  le  terrains.  ICCO  est  partenaire  de  l'alliance  mais  elle  a  encore  besoin  du  soutien  de  ICCO  aussi   bien  sur  le  plan  technique  que  financier.       Our  decision  on  the  annual  plans  is  based  on  the  bi-­‐annual  reflection  meetings  where  partner  present   their  wish-­‐list/challenges  and  area  of  common  interventions  are  identified.  We  have  the  decision  making   team  in  place  to  take  day  to  day  decisions  on  behalf  of  the  coalition.  

2. To  many  respondents  (73%  -­‐  scoring  4  +  5)  the  ’coalition’  is  complementary  to  what  individual   actors  are  doingThe  scoring  on  identification  of  clear  win-­‐win  effects  of  the  cooperation  shows  a   mixed  picture:  from  score  3  to  score  5  (26,8%,  26,8%,  and  31,7%  respectively).    One  

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

39


representative further  explained:  The  win-­‐win  effects  refer  to  results  of  actions  whereby   stakeholders  feel  satisfied  at  some  extent  and  feel  fully  part  of  them.     Table  9  -­‐  Ownership  and  complementarity  of  the  ’coalition’   3   5     1     4   not  relevant;   reactions   2   (to  some   (Fully     (fully  disagree)   (agree)   do  not  know   (#)   extent)   agree)   a.  Decisions  within   the  ’coalition’  are   taken  without  prior   5   4   10   13   8   1   41   consulting  of  the   ICCO  Alliance   b.  Actors  of  the   ’coalition’  feel   ownership  of  their   1   5   7   14   13   1   41   cooperation  and  its   results   c.  Actors  of  the   ’coalition’  feel   1   2   10   12   16   0   41   responsible  for  the   cooperation   d.  The  ’coalition’  is   complementary  to   1   1   8   12   18   1   41   what  individual   actors  are  doing   e.  There  are  clear   win-­‐win  effects  of   0   6   11   11   13   0   41   the  cooperation   identified  by  all       3. As  mentioned  earlier,  it  is  difficult  to  assess  differences  between  and  within  themes  due  to  the   small  numbers.  Interesting  though  is  that  for  statement  a)  Decisions  within  the  ’coalition’  are   taken  without  prior  consulting  of  the  ICCO  Alliance  both  the  themes  Food  and  Nutrition  security   and  the  Basic  education  score  lower  than  the  other  themes.       Section  VII:  Financial  sustainability   Respondents  were  asked  to  provide  information  about  financial  issues  related  to  the  ‘coalition’  in   order  to  gain  insight  into  their  financial  sustainability.  The  responses  can  be  found  in  Table  10  below.     Our  main  observations  are  the  following.    

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

40


Table 10  -­‐  To  what  extent  are  the  ’coalition’  and  its  participants  financially  sustainable   not   1     5     4   relevant;   reactions   (fully   2   3   (Fully     (agree)   do  not   (#)   disagree)   agree)   know   a.  The  actors  in  the  ’coalition’  rely   1   7   13   9   11   0   41   on  funds  of  the  ICCO  Alliance   b.  Financing  of  activities  of  the   ’coalition’  is  largely  carried  out   16   12   9   1   2   1   41   with  own  sources   c.  A  financial  strategy  of  the   ‘coalition’  has  been  developed   6   15   13   3   3   1   41   based  on  diversification  of   resources   d.  Financing  of  activities  of  the   ’coalition’  is  largely  carried  out   9   15   9   4   3   1   41   with  diverse  sources     1. Activities  of  the  ’coalition’  are  hardly  carried  out  with  own  sources  (statement  b).  This  holds  for   all  of  the  themes.  It  matches  with  the  responses  to  statement  d:  only  17  %  (fully)  state  that   activities  of  the  ‘coalition’  are  largely  carried  out  with  diverse  sources.  Moreover,  48,8%  of  the   respondents  state  that  the  actors  rely  on  funds  of  the  ICCO  Alliance  (statement  a,  score  4  +  5   combined).  As  far  as  the  small  numbers  of  ‘coalitions’  per  theme  can  tell,  it  seems  that  especially   Basic  health  and  Basic  education  rely  on  ICCO  Alliance  funds.         2. Almost  none  ‘coalition’  has  developed  a  financial  strategy  based  on  diversification  of  resources   (Score  2  and  3:  36,5%  and  31,7%  respectively).  Only  a  few  ‘coalitions’  focusing  on  FNS  (2  of  6   ‘coalitions’),  Conflict  transformation  and  democratization  (3  of  4),  and  Basic  education  (1  of  6)  do   so.  Financial  management  is  an  area  that  is  not  developed  fully  and  needs  more  attention   according  to  remarks  added  by  a  few  respondents  –  see  Box  6  below.     3. Comparing  the  responses  of  ‘coalitions  building  on  existing  coalitions,  networks  and  the  like’,   with  those  of  ‘coalitions  formed  on  instigation  of  the  ICCO  Alliance’,  it  appears  that  the  first  type   scores  a  bit  higher  than  the  latter.  It  seems  that  to  some  extent  (score  3)  they  carry  out  activities   with  their  own  resources  and  with  diverse  resources  and  have  developed  a  sort  of  financial   strategy  towards  diversification  of  resources.     Box  6  –  Additional  remarks  about  financial  management   -­‐ -­‐ -­‐ -­‐

-­‐ -­‐ -­‐

-­‐

This is  still  a  developing  but  growing  area  that  is  still  very  slow     We  need  to  develop  this  areas  in  future   La  coalition  est    dans  un  processus  de  restructuration.  la  coalition  mettra  en  place  dans  les  prochains  jours   un  plan  de  mobilisation  de  ressources.   The  partners  in  coalition  were  trained  on  financial  Management  including  financial  strategies,  fundraising,   procurement  procedures,  etc.  One  of  the  requirements  in  each  Contract  is  fundraising  and  annual   reduction  of  funds  from  ICCO/DCA   Presque  toutes  les  activités  de  la  coalition  sont  financées  par  l'Alliance  ICCO.  Cependant,  la  coalition  a   élaboré  un  plan  pour  assurer  le  funraising  et  la  mobilisation  des  fonds.   They  rely  for  the  Education  Program  mainly  on  funding  from  ICCO.  Almost  all  partner  have  other  donors   for  their  programs  /  activities.   L'alliance  sécurité  alimentaire  dépend  entièrement  de  ICCO  encore  pour  le  financement  de  ses  activité.   Un  défi  important  sur  lequel  nous  travaillons  est  la  diversification  des  sources  de  financement.   We  have  started  discussions  about  diversifying  our  funding.  We  have  approached  two  donors  and  plan  to   continue  developing  a  diverse  funding  base  for  the  coalition.  

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

41


Section VIII:  The  internal  functioning  of  the  'coalition'   About  nine  statements  were  included  in  the  survey  to  find  out  to  what  extent  internal  conditions  for   joint  working  and  action  are  in  place.    The  results  are  reflected  in  table  11  below.  Some  main   observations:   1. A  number  of  internal  conditions  for  joint  working  and  action  are  in  place:     -­‐ Actors’  competences  (statement  a):  respondents  mostly  agree  that  the  actors  of  the   ’coalition’  have  most  competencies  (networking,  conflict  resolution,  interpersonal  skills)  to   function  smoothly  (48,7%  -­‐  score  4);     -­‐ Trustful  relationships  (statement  b):    according  to  most  respondents  relationships  between   the  actors  of  the  ’coalition’  are  built  on  trust  (score  4  +  5  together  78%  of  the  responses);   -­‐ Definition  of  roles  (statement  d):    the  roles  between  the  actors  of  the  ’coalition’  are  well-­‐ defined,  building  on  each  other’s  strengths  (score  4  +  5  together  68%  of  the  responses);  A   respondent  added:  Avec  le  manuel  des  procédures,  le  fonctionnement  de  la  coalition  est   organisé  par  des  instruments  qui  définissent  les  rôles  et  responsabilités  de  chaque  membre.   -­‐ Communication  strategy  (statement  c):  63,4%  of  the  respondents  state  (score  4+5  together)   that  the  ’coalition’  has  put  an  effective  internal  communication  strategy  between   participating  actors  in  place,  but  still    29,3%  are  indecisive  (score  3);   -­‐ Participation  (statement  e):  48,7%  of  the  respondents  agree  (score  4)    that  the  actors  of  the   ’coalition’  participate  actively  to  ensure  smooth  functioning  of  the  cooperation;   -­‐ Governing  body  (statement  g):  in  46,3%  (score  5)  of  the  ‘coalitions’  a  representative   governing  body  is  said  to  be  in  place.  A  respondent  added:  Internal  structure  is  in  place  and   functions  well:  Coordination  Council,  thematic  sub-­‐groups:  internal  migration,  external   migration,  sending  communities,  focal  points  by  regions,  Information  Committee,  General   Meeting.  Comparing  the  responses  of  ‘coalitions  building  on  existing  coalitions,  networks  and   the  like’  with  those  of  ‘coalitions  formed  on  instigation  of  the  ICCO  Alliance’,  a  considerable   number  of  respondents  of  the  latter  ticked  score  3  (to  some  extent)  as  well  (10  out  of  31   responses,  score  4:  13  out  of  31)   -­‐ Internal  learning  (statement  h):  respondents  (fully)  agree    that  the  actors  of  the  ’coalition’   make  active  use  of  learning  opportunities  within  the  ’coalition’  (score  4  +5  together  68,3%).     Two  respondents  clarified:  The  coalitions  conduct  peer  to  peer  visits  and  also  share  technical   expertise  in  trainings;  and,  Within  the  coalition,  the  partners  had  exchange  programmes   within  the  partnership  to  learn  from  each  other  the  best  practices  and  implement  them  in   their  respective  programme  areas.     2. Still  a  number  of  respondents  ticked  score  3  (to  some  extent)  as  shown  in  the  table.  In  particular,   the  scoring  on  formalisation/  implementation  of  a  joint  governance  or  co-­‐responsibility  of  the   ’coalition’  are  almost  equal  for  score  3,  4,  and  5  (around  29%)  (statement  f).  The  same  equal   scoring  applies  to  actors’  active  use  of  learning  and  exchange  opportunities  outside  the   ’coalition’.      

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

42


Table 11  -­‐  The  internal  functioning  of  the  'coalition':  To  what  extent  are  internal  conditions  for  joint   working  and  action  in  place   1     5     not   4   reactions   (fully   2   3   (Fully   relevant;  do     (agree)   (#)   disagree)   agree)   not  know   a.  The  actors  of  the  ’coalition’   have  most  competencies   (networking,  conflict  resolution,   1   3   9   20   8   0   41   interpersonal  skills)  to  function   smoothly   b.  Relationships  between  the   actors  of  the  ’coalition’  are  built   1   2   6   16   16   0   41   on  trust   c.  The  ’coalition’  has  put  an   effective  internal  communication   0   3   12   18   8   0   41   strategy  between  participating   actors  in  place   d.  The  roles  between  the  actors   of  the  ’coalition’  are  well-­‐ 0   5   7   13   15   1   41   defined,  building  on  each  other’s   strengths   e.  The  actors  of  the  ’coalition’   participate  actively  to  ensure   2   3   8   20   8   0   41   smooth  functioning  of  the   cooperation   f.  Joint  governance  or  co-­‐ responsibility  of  the  ’coalition’       2   2   11   12   13   1   41   is  formalised  and  implemented   g.  A  representative  governing   3   3   6   9   19   1   41   body  is  in  place   h.  The  actors  of  the  ’coalition’   make  active  use  of  learning   0   4   9   14   14   0   41   opportunities  within  the   ’coalition’   i.  The  actors  of  the  ’coalition’   make  active  use  of  learning  and   0   6   11   11   13   0   41   exchange  opportunities  outside   the  ’coalition’      

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

43


Section IX:  Effects  of  ’coalition’  or  cooperation       A  last  series  of  questions  were  posed  to  gain  insight  into  the  perceived  effects  of  the  ‘coalition’  or  the   cooperation  between  its  actors.  For  the  outcomes  see  table  12  below.  We  did  not  observe   differences  between  coalitions  that  already  existing  before  they  received  from  support  from  the   ICCO  Alliance  and  those  there  were  formed  on  instigation  of  the  Alliance.  Respondents  were  asked  to   add  an  example  of  an  effect–  see  some  responses  in  Box  7.    Some  main  observations  are  given   below.   1. A  number  of  pre-­‐defined  effects  were  given  which  the  respondents  could  score:   -­‐ Organizational  development  of  the  individual  actor  (statement  a):  almost  the  same  number  

-­‐

-­‐

-­‐

-­‐

-­‐

of respondents  indicated  that  such  development  took  place  as  those  who  said  it  happened   ‘to  some  extent’–  about  31%;  the  other  scores  were  lower.     Increased  realization  among  actors  that  jointly  undertaking  activities  has  added  value   (statement  b):  the  majority  of  the  respondents  ticked  agree  or  fully  agree  (score  4+5   together  75%).   Increased  realization  among  actors  that  joint  fundraising  has  added  value:  again,  almost  the   same  number  of  respondents  agreed  to  this  statement  c  as  those  who  felt  such  happened  ‘to   some  extent’  –  about  26%;  We  have  seen  in  section  VII  that  most  ‘coalitions’  do  not  yet  have   a  financial  strategy.     Increased  realization  among  actors  that  joint  learning  has  added  value  (statement  d):  the   highest  scoring  concerns  score  4  and  5  (  both  scores  together  75%)  This  is  in  line  with  the   views  on  internal  learning  indicated  in  section  VIII:  the  actors  of  the  ’coalition’  make  active   use  of  learning  opportunities  within  the  ’coalition’.   Purposely  member  diversification  to  effectively  address  the  root  causes  of  poverty  and   injustice  (statement  e):  here,  most  respondents  agree  (32,5%)  or  score  that  they  do  so  to   some  extent  (40%).   The  overall  scoring  concerning  ‘Sustainable  forms  of  cooperation  between  different  actors   are  increasingly  developing  due  to  the  programmatic  approach’  (statement  f)  shows  that   most  respondents  agree  with  this  statement  (42,5%)  with  almost  equal  number  scorings  on  3   and  5  (25%  and  22,5  %  respectively).        

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

44


Table 12  -­‐  Effects  of  ’coalition’  or  cooperation   1     5     not   4   reactions   (fully   2   3   (Fully   relevant;  do     (agree)   (#)   disagree)   agree)   not  know   a.  Organisational  development   takes  place  by  individual  actors   1   5   12   13   9   0   40   due  to  the  participation  in  the   ’coalition’   b.  The  actors  of  the  ’coalition’   increasingly  realise  added  value   1   2   7   16   14   0   40   in  jointly  undertaking  activities   c.  The  actors  of  the  ’coalition’   increasingly  realise  added  value   4   8   10   11   7   0   40   in  joint  fundraising     d.  The  actors  of  the  ’coalition’   increasingly  realise  added  value   1   0   9   16   14   0   40   in  terms  of  joint  learning   e.  The  ‘coalition’  varies  its   member  composition  to   5   8   10   13   3   1   40   effectively  address  the  root   causes  of  poverty  and  injustice   f.  Sustainable  forms  of   cooperation  between  different   actors  are  increasingly   1   3   10   17   9   0   40   developing  due  to  the   programmatic  approach     Box  7  –  Examples  of  effects  of  cooperating  in  a  ‘coalition’   Capacity  building/Exchange   -­‐ Cooperation  on  training,  advocacy  and  lobby.   -­‐ Capacity  building  that  include  accompaniment  by  skilled  staff  and  experience  exchange   -­‐ Within  the  coalition  the  'district  approach'  of  one  of  the  partners,  with  involvement  of  district  organisations   in  training  and  capacity  building,  is  taken  over  by  other  partners.   -­‐ La  realización  conjunta  de  un  proceso  de  sistematización  de  las  experiencias  de  formación  de  docentes  en   servicio,  con  enfoque  de  Educación  Intercultural  Bilingüe.   Joint  fundraising   -­‐ Some  of  the  partners  within  the  network  have  jointly  written  successful  proposals,  eg  ….  Other  partners  have   shared  office  space  and  staff  with  fellow  network  partners,  eg  ..  Other  partners  have  strengthened  their  cost-­‐ recovery  mechanisms,  eg  ….   -­‐ Soumission  conjointe  d'une  proposition  de  projet  auprès  de  l'Union  Européenne  (X):  les  membres  de  la   coalition  ont  formulé  d'une  manière  participative  et  conjointe  le  projet.    L'intervention  aur  lieu  dans  un  site   commun,  les  apports-­‐  contribution  propre  exigé  par  l'appel  à  proposition  seront  supportés  par  les  membres.   -­‐ The  coalition  has  opened  up  to  multiple  funders.   Cooperation  in  activities   -­‐ Cooperation  between  ..    Regional  producers  organisations  &  …  a  national  Represent  Process  in  factory  and  ..   National  NGOs  and  …  international  development  partners.   -­‐ Actualmente,  X  se  encuentra  en  una  etapa  de  Fortalecimiento,  después  de  3  años  de  su  conformación.  Se   están  construyendo  poco  a  poco  los  mecanismos  de  cooperación,  interacción,  compromiso,  negociación  y   monitoreo  dentro  del  emprendimiento,  considerando  el  compromiso  que  todos  van  asumiendo  dentro  de   esta  nueva  instancia  de  articulación.  No  ha  sido  fácil,  pero  se  está  creando  un  reconocimiento  de  parte  de   todos  los  socios  al  ir  buscando  nuevos  proyectos  y  oportunidades  de  manera  a  ir  cubriendo  las  necesidades   identificadas  dentro  de  la  organización,  en  el  sector  orgánico  y  de  cada  uno  de  los  integrantes.   Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

45


-­‐ Partnership building  between    coalition  members,    influenced  government  authority  to  develop  policies  to   integrated    address  HIV  and  health  related  issues  through  integrating  it  into  government  health  and   development  system.   -­‐ Actors  working  in  the  same  geographical  area  increasingly  coordinate  their  interventions  with  each  other.   -­‐ Dans  le  traitement  des  problématiques  liées  à  la  SAN,  Y  se  mobilise  (i)  en  menant  des  actions  de  plaidoyer   des  différents  niveaux  (régional,  national  et  international)  en  (ii)  en  développant  des  projets  visant  à   renforcer  et  améliorer  la  SAN  des  couches  vunlérables.    chaque  entité  membre  remonte  au  niveau  du   Secrétariat  Exécutif  les  questions  nécessitant  une  intervention  conjointe  pour  avoir  plus  des  voix  et  pour  être   écoutés.   Diversification  of  the  ‘coalition’  /  networking     -­‐ ICCO  est  en  partenariat  durable  avec  ….  Toutefois,  la  Coalition  ..  commence  en  2011  et  est  un  debut  de   cooperation  durable  parce  que  le  partenariat  est  diversifie  et  complementaire  avec  ces  8  membres  de  la   Coalition.   -­‐ L'exemple  que  nous  pouvons  citer  pour  assurer  une  durabilité  de  la  coopération  constitue  la  mise  en  réseau   des  partenaires  techniques  et  financiers.   -­‐ Establishing  thematic  platforms/networking  (established  many  addiitonal  specific  thematic  platforms  with   government,  institualization  of  various  cooperation).  For  example,  in  X  the  platform  has  established  a  ..Public   Council  on  migration  including  parliamentarians,  ministries,  NGOs  -­‐  within  the  thematic  of  sending   communities.  or  Established  "Regional  Forum"  (multi  stakeholder,  multilevel)    within  the  group  of  external   migration.   -­‐ Besides  working  with  the  ICCO  Cooperation,  the  program  now  seeks  linkages  with  other  partners  in  the  ACT   Alliance  (e.g.  Christian  Aid).   Outcomes  of  activities   -­‐ The  Community  Groups  at  the  village  level  are  getting  more  aware  about  their  roles  and  responsibilities  and   collective  action  to  address  their  needs  such  as  relevant  and  quality  education.  They  are  also  able  to  demand   services  from  the  government.   Other   -­‐ For  example,    X  provides  recorded  programmes  on  health  for  community  listener  groups  at  Y.   -­‐ Par  le  biais  du  programme  des  ONG  pour  la  sécurité  alimentaire  et  nutritionnelle  au  A,  les  ONG  membres  de   l'alliance  Sécurité  alimentaire  ont  obtenu  plus  de  visibilité  dans  leurs  actions.   -­‐ Les  membres  de  la  coalition  ont  décidé  de  restructurer  le  reseau  pour  devenir  une  organisation  stable  et   durable.   -­‐ Where  roles  of  all  the  actors  and  coalition  vision  are  clearly  defined  and  decisions  are  participatory  made  to   promote  ownership  of  the  coalition  among  the  actors.  

Lastly,  a  few  respondents  added  some  remarks  concerning  cooperation  and  coalition  building  and   the  survey  exercise  itself    –  these  are  included  in  box  8.      

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

46


Box 8  -­‐  Additional  remarks  on  the  survey/issues  addressed  in  the  survey   • Promoting  coalition  is  the  way  to  address  root  causes  of  poverty  and  injustice.  However,  ensuring  its   effectiveness  takes  time  and  requires  resources.  There  is  high  level  of  commitment  from  ICCO  Cooperation   towards  coalition  building.  The  partners  (local  actors)  also  started  getting  better  understanding  on   coalition  building.  However,  still  there  is  long  ways  to  go.  We  started  with  our  (ICCO  Cooperation's   partners)  and  the  diversity  among  the  members  of  coalitions  that  we  initiated  is  minimal.  If  I  take  the  case   of  the  coalition  I  am  working  with  it  consists  eight  partners  all  funded  by  ICCO  Cooperation  and  so  far  they   couldn't  bring  on  board  non-­‐funded  actors.  On  the  other  hand  for  the  coalition  to  bring  intended  changes,   there  is  a  need  to  have  different  types  of  actors  at  different  levels.  Hence,  this  should  be  an  area  where  we   need  to  focus  on  for  the  future.  I  work  with  two  coalitions,  but  since  generally  the  two  are  similar,  filling   the  survey  twice  will  not  add  any  value.   • It  is  a  relevant  exercise  and  should  be  done  periodically.   • Ce  genre  de  suivi  permettra  de  procéder  au  renforcement  organisationnel  de  la  Coalition   • Thank  you  for  the  opportunity  of  assessing/reflecting  ourselves.   • Nous  avons  prevu  un  atelier  regional  entre  les  partenaires  de  ..  du  …  pour  partager  les  experiences  entre   les  3  pays.  Aussi,  nous  prevoyons  la  Theorie  du  Changement  pour  la  Coalition  …..   • This  survey  is  very  important  to  jauge  the  progress  of  coalitions  towards  the  programmatic  approach.  It's   also  an  eye  opener  regarding  things  that  have  been  achieved  compared  to  what  should  be  done.   • C'est  un  questionnaire  assez  compréhensible  et  explicite.   • In  addition  to  common  activities  within  the  coalition,  there  are  emerging  joint  working  among  2-­‐3   organisations  within  the  coalition  to  address  a    specific  issues  probably  internal  within  those  organisation   or  within  a  specific  region/area  even  outside  the  common  programme  operation  region.  So  cooperation   within  the  coalition  is  also  possible  and  should  not  be  seen  as  a  problem.  Resource  for  the  coalition  to   operate  for  the  near  future  is  very  important,  however,  this  could  be  followed  with  a  Fundraising  strategy   that  can  also  consider  financial  resources  within  the  country  and  also  from  donors  outside  ICCO  alliance.   otherwise  the  immediate  results  show  that    there  is  win-­‐win  situation  for  most  partner  organisation  in  the   area  of  learning  and  sharing.   • X  is  the  lead  agency  in  Y  Beyond  2015  to  lead,  initiate  and  facilitate  the  national  consultation   process/deliberations  among  civil  society  organizations  to  come  out  with  post  2015  MDG  development   framework.  

Section  X:  Analysis  against  the  relevant  statements  of  the  TOR     The  TOR  requested  more  detailed  information  on  various  items  about  which  the  survey,  to  a  certain   extent,  has  provided  some  insight.  We  address  them  below.       • Evolution  of  the  programme  coalitions  and  variations  thereof  in  various  thematic  and   geographical  settings   The  survey  does  not  give  a  picture  on  how  the  ‘coalitions’  have  developed  because  it  did  not  include   a  time  dimension.  From  a  geographic  point  of  view,  most  responses  have  been  provided  for  African   ‘coalitions’.  Thematically,  most  responses  were  provided  for  the  theme  Basic  Health  and  HIV  AIDS.       • Roles  and  responsibilities  of  the  various  members  of  the  coalitions  and  the  ways  in  which  these   have  evolved  (including  ICCO  staff  and  external  advisors)   The  survey  gives  some  information  on  roles  and  responsibilities  in  terms  of  clarity  about  such  roles.   Respondents  were  asked  about  the  extent  to  which  the  roles  between  the  actors  of  the  ’coalition’   were  well-­‐defined,  building  on  each  other’s  strengths.  In  section  VIII  we  have  seen  that  the  majority   agrees  that  this  is  the  situation  in  their  ‘coalition’  (score  4+5  together  68,37%).  The  survey  did  not  ask   about  the  specificities  of  the  roles  and  responsibilities  and  those  of  ICCO  staff  and  external  advisors.       • The  sustainability  of  the  cooperation  within  the  programme  coalitions/  The  effectiveness  of  the   programmatic  approach  as  a  methodology  to  strengthen  sustainable  forms  of  cooperation   between  different  actors  in  Southern  countries   The  survey  included  two  statements  about  sustainability.  First,  about  financial  sustainability  (section   VII).  The  survey  revealed  that  activities  of  the  ’coalition’  are  hardly  carried  out  with  own  sources.  This   holds  for  all  of  the  themes.  Almost  half  of  the  respondents  (48,7%  scored  4+5)  indicated  that  the   Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

47


actors of  the  ‘coalition’  relied  on  funds  of  the  ICCO  Alliance,  while  a  much  smaller  group  said  that   their  ‘coalition’  uses  diverse  resources  among  which  IA  funds  (17%  score  4+5).  Almost  none   ‘coalition’  has  developed  a  financial  strategy  based  on  diversification  of  resources.  Financial   management  is  an  area  that  is  not  developed  fully  and  needs  more  attention  according  to  remarks   added  by  a  few  respondents.  It  is  in  line  with  the  responses  to  the  statement  that  there  is  Increased   realization  among  actors  that  joint  fundraising  has  added  value  (about  26%  of  the  respondents   agreed  to  this  statement  and  the  same  percentage  indicated  that  they  did  so  to  some  extent  (score   3).  Comparing  the  responses  of  ‘coalitions  building  on  existing  coalitions,  networks  and  the  like’  with   those  of  ‘coalitions  formed  on  instigation  of  the  ICCO  Alliance’,  it  appears  that  the  first  type  scored  a   bit  higher  than  the  latter.  The  survey  did  not  dig  deeper  in  the  reasons  behind  this  observation.     Second,  the  question  was  asked  about  sustainable  forms  of  cooperation  due  to  the  programmatic   approach:  Sustainable  forms  of  cooperation  between  different  actors  are  increasingly  developing  due   to  the  programmatic  approach  (Section  IX).  The  answer  indirectly  tells  something  about  sustainability   of  the  ‘coalitions’:  most  respondents  can  agree  with  this  statement  (65%)  with  a  much  lower   percentage  in  score  3  (25%).  This  holds  for  all  themes.  Other  survey  statements  can  be  considered   proxy  indicators  of  sustainability  of  ‘coalitions’:   o On  Effects  of  ’coalition’  or  cooperation  (Section  IX):  Most  respondents  indicated  that  there  is   an  increased  realization  among  actors  that  jointly  undertaking  activities  and  joint  learning   both  have  added  value.   o On  Implementation  and  implementation  capacity  of  the  ‘coalition’  (Section  III):  The  majority   of  the  respondents  agreed  that  the  ’coalition’  reaches  its  targets  and  objectives.  Since  not  all   ‘coalitions’  have  a  monitoring  system  in  place  that  is  used  it  is  not  clear  on  which  this   perception  is  based.  Most  respondents  were  also  positive  about  the  flexibility  in  which  the   ‘coalition’  adapts  its  activities  based  on  learning.  Furthermore,  most  respondents  perceived   the  staff  of  all  actors  in  the  ’coalition’  sufficiently  capable  to  address  the  issues  on  which  the   cooperation  focuses.   o On  Ownership  and  complementarity  of  the  ’coalition’  (Section  VI):  Over  half  of  the   respondents  (51,2%)  indicated  that  decisions  within  the  ’coalition’  are  taken  without  prior   consulting  of  the  ICCO  Alliance.  According  to  the  majority  of  the  respondents  (about  68%)  a   ‘feeling  of  responsibility’  exists  for  the  cooperation  as  well  as  a  ‘feeling  of  ownership  of  the   cooperation  and  its  results’.  This  finding  links  well  with  one  of  the  ‘pearls’  discussed  in  the   report  of  the  first  phase  of  the  evaluation.       • The  ways  in  which  and  the  extent  to  which  gender  equality  and  human  rights  are  being   integrated  /  taken  up  in  programmes     Two  sections  in  the  survey  addressed  the  issues:  Section  IV  and  V.  The  overall  conclusions  are  that   over  60%  of  the  respondents  feel  that  the  ‘coalition’  has  a  clear  vision  on  gender  equality  and  make   efforts  to  decrease  gender  inequality,  while  about  25%  have  a  mixed  feeling  (score  3).    Concerning  a   clear  vision  on  a  rights-­‐based  perspective,  the  respondents  score  a  bit  lower  than  on  gender  equality   but  still  over  50%.  However,  according  to  the  respondents  most  activities  of  the  ‘coalition’  include   efforts  to  strengthen  the  capacities  of  rights  holders,  especially  vulnerable  groups  (78%  scored  4+5   together).  Also,  the  majority  (56%)  states  that  activities  of  the  ‘coalition’  include  efforts  to  hold  duty   bearers  accountable.  Furthermore,  not  all  ‘coalitions’  monitor  the  impact  of  its  activities  on  gender   equality  (36,6%  score  4+5)  but  based  on  the  responses  many  ’coalitions’  monitor  the  impact  of  its   activities  on  vulnerable  groups  (65,8%  score  4+5).  This  result  is  interesting  because  48,8%  of  the   respondents  agree  that  the  coalition  has  a  monitoring  system  in  place  and  29,3%  even  denies  that   there  is  such  a  system  (SectionIII).           • The  most  significant  results  of  working  with  the  programmatic  approach  to  date   For  all  themes  almost  all  respondents  gave  at  least  score  3  and  mostly  higher  about  the   programmatic  approach:    Sustainable  forms  of  cooperation  between  different  actors  are  increasingly   Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

48


developing due  to  this  approach.    The  survey  also  includes  some  perceptions  on  the  results  of  the   ‘coalitions’  which  could  be  considered  proxy  indicators  on  significant  results  –  the  effects  of   ’coalition’  or  cooperation  which  we  already  addressed  above.  Also,  respondents  reacted  to  the   statement  There  are  clear  win-­‐win  effects  of  the  cooperation  identified  by  all.  The  scoring  on  this   statement  showed  a  mixed  picture  ranging  from  indecisive  and  fully  agree  (score  3  –  5).  The  survey   did  not  ask  to  clarify  the  effects  or  to  give  examples.       Verona  Groverman,  Kees  Zevenbergen,  9  October  2013     Annex  -­‐  List  of  coalitions  about  which  information  was  collected  through  the  survey  (34)   1. ALOCES  E-­‐DR  Congo   2. COSPASAK  Coordination  of  the  PASAK  Program  (Program  d’Appui  a  la  Securite    Alimentaire   du  Sud  Kivu)   3. ICCO  Alliance  Health  program  Ethiopia   4. SSHARE  South  Sudan   5. ICCO  Alliance  Health  program  Uganda   6. Lango  FNS  Cluster  Uganda     7. Distaster  Risk  Reduction  Platform  for  Teso  (DRRP4T)  Uganda     8. Central  Asia  on  the  Move  (CT&D)  Kyrgyzstan,  Tadjikistan,  Russia  and  Kazakhstan   9. Program    Gran  Chaco  (CT/D)  Bolivia,  Argentina,  Paraguay     10. Association  Paraguay  organic     11. Ruta  del  Sol  Peru   12. Afghanistan  Health  Alliance   13. South  Odisha  Development  Initiative  India   14. Development  Focus,  Bangalore  India   15. Western  Orissa  Education  Watch  (WOEW)   16. Health  Bridge  Alliance  India   17. HIV/AIDS  &  SRHR  Coalition  Nepal   18. Seaweeds  Net  (Samar  Island  Seaweeds  Value  Chain  Network)  Philippines   19. PhilSEN  (Philippine  Social  Enterprise  Network)   20. Vietnam  River  Network   21. Land  and  Forest  Coalition  Vietnam   22. Coalition  LIFE  Madagascar   23. Uchembere  Network  Malawi     24. National  Rice  Development  Platform  Malawi   25. SALT  Alliance  South  Africa   26. Basic  Health  &  HIV/Aids  Coalition    Zimbabwe   27. Coalition  Sécurité  Alimentaire  et  de  Nutrition  du  Burkina  Faso  (C/SAN)   28. Alliance  Programme  Spécial  de  Sécurité  Alimentaire  et  Nutritionnelle  du  Bénin  (ProSSAN)   29. Alliance  Soja  du  Bénin   30. Alliance  Anacarde  du  Bénin     31. Ghana  WASH  Alliance     32. Alliance  for  Strengthening  Education  in  Ghana  (ASEG)   33. Programme  Multi  Acteurs  ENF-­‐SRDS/J    Senegal   34. Palestinian  Human  Rights  Organisations  Council  (PHROC)  Defense  for  Children  International   Palestine       No  information  was  received  about  the  following  coalitions:   1. Amhara  Cluster  Ethiopia   2. Oromiya  Cluster  Ethiopia   3. Afar  Cluster  Ethiopia   Annex  4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

49


4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 16

WASH Uganda,  Kenya,  Ethiopia   TUCO  Kenya   STAR  School  Programme  Coalition  Kenya   Food  security  and  thematic  program  consortium  of  the  European  Union  Republic  of  South   Sudan  Western  Bahr  el  Ghazal  state   Basic  Health  and  HOV/Aids  GZB,  Light  for  the  World,  Dorcas  South  Sudan   Centroamerica  Democratica  (CAD)  Guatemala,  El  Salvador  and  Honduras     Programa  de  Seguridad  Juvenil  (PSJ:  Juvenile  Security  Program)   Mesoamerican  Alliance  of  Peoples  and  Forests   FOROLACFR    Latin  American  Forum  for  Rural  Finance     REDIMIF  Microfinance  Network  of  Guatemala     RED  KATALYSIS  Central  America    (  Guatemala,  Honduras,  Salvador,  Nicaragua,  Costa  Rica)   AGEXPORT  Guatemalan  Export  Asociation   Association  of  Microfinance  Institutions  in  Kyrgyzstan   Coordination  Council  on  Development  of  microfinance  in  Kyrgyzstan   RIMISP  Latinamerican  Center  for  Rural  Development     SUSO  –  VSC  Brazil,  Bolivia,  Peru   Gran  Chaco  Program   Regional  Food  Security  Programme  focusing  on  the  Human  Right  to  Adequate  Food    Perú,   Bolivia  y  Paraguay   Consortium  Agro  ecological  Peru   Bangladesh  WASH  Alliance   Health  Alliance  Bangladesh   Vikas  Bazar  Network  India   Siksha  se  Parivartan  (SSP)  –  Jharkhand  India   Siksha  Chetana  (SC)  -­‐  Orissa   DAHAR  –  Jharkhand  India   Bastar  Ujar  –  Chhattisgarh  India   National  Coalition  on  Food  and  water  security  (NCFAW)  Nepal   Towards  Ecological  Recovery  Regional  Alliance  (TERRA)  Mekong  Region  (SEA)   Asia  Indigenous  Peoples  Pact  (AIPP)   National  Rural  Women’s  Congress  (NRWC)  Philippines   Mindanao  Peace  Partners  Philippines   RightsNet  Philippines   NOMIA  (Negros  Island  Muscovado  Industry  Association)  Philippines   PhilNet-­‐RDI  (Philippine  Network  of  Rural  Development  Institutes)   PACCI  (Provincial  Access  Centers  Consortium,  Inc)  Philippines   PFEC  (Philippine  Federation  for  Environmental  Concerns)   FuND  Philippines  (Fundraisers’  Network  for  Development)   OCCP  (Organic  Certification  Center  of  the  Philippines  Philippines)   Organic  rice  Northern  Mindanao   Organic  rice  Southern  Mindanao   Microfinance  Council  of  the  Philippines   Bicol  Microfinance  Council  Philippines   Mindanao  Microfinance  Council   National  Confederation  of  Cooperatives  Philippines   Name?  Agriculture  ANGOLA   ICPM16  Madagascar   AINA17  Madagascar  

Initiative Commune pour le Plaidoyer à Madagascar : Consortium of four international NGO: CARE, Medair, FAO, MdM, ICCO. Currently in its 3rd phase. ICCO participates in collaboration with its local partner SAF/ FJKM.

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

50


51. PAMOLEA18 52. Health  coalition  Malawi   53. Programme  multi  acteurs  d’éducation,  de  formation  professionnelle  et  de  la  promotion  de  la   SRDS/AJ    Burkina   54. Alliance  Ananas  du  Bénin   55. Alliance  WASH-­‐Bénin  (AWB)   56. Program  Multi  Acteurs  d’Alphabetisation  Mali     57. FSN  coalition  Mali   58. Who  Profits?  Economic  research  together  with  partners  Israel   59. United  Civilians  for  Peace  (UCP)     60. OPGAI   61. Kairos  oPt  and  ww  

17

Actions  Intégrées  en  Nutrition  et  Alimentation.  Consortium  of  :  AIM,  CARE,  FAO,  FIDA,  GRET,  ICCO,  PAM,   Welthungerhilfe.  The  program  is  in  its  initial  phase.  Period  :  2013  -­‐  2015  with  a  budget  ±  €  13  million  (EU  funding).  ICCO   participates  in  the  consortium  in  collaboration  with  the  local  partner  SAF  /  FJKM.     18 Projet d’Appui à la Maitrise d’Ouvrage Local pour l’Eau et l’Assainissement: Consortium of 3 Malagasy NGO (SAF/FJKM, Fikrifama et FSG) and ICCO. Period : 2007 – 2013.

Annex 4  –  Report  of  the  second  phase  of  the  Evaluation  Programmatic  Approach  –     Survey  about  the  development  and  performance  of  coalitions  

51


Annex 5  –  Terms  of  Reference  Advancement  of  ICCO  Alliance’s  working  with  the   Programmatic  Approach  2009  –  2012,  phase  3,  field  study  

Introduction   This  document  provides  the  ToR  for  the  field  study  phase  of  the  Evaluation  Programmatic  Approach.   In  the  ToR  of  this  evaluation  it  was  envisaged  that  an  eventual  field  study  had  to  be  justified  by  the   evaluators  and  to  be  based  on  the  findings  of  the  first  phase  of  the  evaluation;  the  desk  study  and   the  online  survey.     The  desk  study  provided  a  comprehensive  research  into  all  documents  written  on  the  development   of  and  the  experiences  of  ICCO  Cooperation  on  working  with  the  Programmatic  Approach.  The   findings  of  the  documents  were  verified  and  deepened  in  interviews  with  the  MT  of  ICCO   Cooperation,  programme  officers  and  the  coordinator  Capacity  Development.   The  results19  of  the  research  were  presented  and  shared  with  the  MT  of  ICCO  Cooperation   (September  2013).  There  was  a  consensus  on  the  main  findings  with  regards  to  the  core  elements  of   the  Programmatic  Approach  in  practice  (the  so-­‐called  pearls  of  IA  practice).  Phase  two  of  the   evaluation  was  done  through  a  survey  that  was  sent  to  Programme  Officers  involved  with  coalitions   and  representatives  of  the  coalitions.  The  survey  aimed  at  creating  more  clarity  about  the   developmental  aspects  of  coalitions  and  the  programmes  they  implement  (and  answers  some  of  the   evaluation  questions  on  these  topics).     Findings  phase  one   The  initial  synthesis  report  defines  besides  a  notion  of  the  need  to  come  up  with  clarity  on  the   concept  ‘programmatic  approach’,  and  the  need  for  more  leadership  on  the  programmatic  approach   in  the  future,  6  elements  of  the  programmatic  approach  which  are  to  be  considered  its  essence  and   need  to  find  their  way  into  future  formulation  of  the  ICCO  Cooperation’s  way  of  working  on  the   programmatic  approach.  The  elements  are  shortly  (see  annex  1  for  a  more  elaborate  overview  of  the   pearls):  ownership,  local  dynamics,  partnership,  trust  &  accountability,  joint  action,  learning  and   flexibility.20     Findings  phase  two   The  findings  from  the  survey21  have  largely  confirmed  the  findings  from  phase  one.  More  than  fifty   per  cent  of  the  survey  respondents  were  positive  about  ownership  within  coalitions/  programs,   feeling  of  responsibility  for  the  cooperation,  the  coalition’s’  implementation  capacity,  integration  of   gender  equality  and  rights-­‐based  approach  and  effects  of  the  cooperation.  Less  confidence  is   reported  in  relation  to  financial  sustainability  and  fund  diversification.     The  findings  of  the  survey  have  also  been  used  to  gain  an  insight  in  which  coalitions  score  high  on  the   survey  elements  that  are  comparable  to/have  relevance  for  the  ‘pearls’  and  can  therefore  be  seen  as   proxy  for  achievement  on  the  pearls.  The  thinking  behind  this  is  that  if  coalitions  score  high  on  these   survey  elements  there  is  something  to  learn  from  that  experience  about  the  conditions  that  made   this  possible.  The  choice  of  coalitions  to  be  included  in  phase  3  should  be  as  much  as  possible  within   this  range  of  high  scoring  coalitions.       Objective  phase  3   Based  on  the  initial  findings,  it  seems  to  be  relevant  to  design  the  field  visits  of  the  evaluation  PA  in   such  a  way  as  to  get  a  deeper  insight  into  the  practice  of  the  formulated  pearls.     19

Evaluation Programmatic  Approach  IA.  Initial  synthesis  of  findings,  September  2013.    PREZI  Kees  Zevenbergen  for  MT  October  2013   21  Evaluation  Programmatic  Approach,  Survey  about  the  development  and  performance  of  coalition.  Verona   Groverman  and  Kees  Zevenbergen,  9  October  2013   20

Annex 5  –    ToR  field  study  

52


The purpose  of  the  three  field  studies  is  to  unearth  the  lessons  learned  from  three  coalitions  that  are   considered  to  be  successful  and  to  formulate  guidance  for  the  further  development  of  the   Programmatic  Approach  (seen  through  the  lenses  of  the  6  already  identified  ‘  pearls’).     Main  objective  for  the  field  studies  would  therefore  be  to  shed  more  light  on  the  questions  of  why,   how  and  what  exactly  a  number  of  strong  performing  coalitions  have  done  to  become  strong   performers,  including  an  analysis  of  the  circumstances  and/or  preconditions  that  might  have   contributed  to  their  successful  emergence  (including  the  roles  that  ICCO  Cooperation  staff  played   therein  as  ‘backbone  support  organization’22).     The  strong  performers  will  be  selected  on  the  basis  of  findings  of  the  survey  about  the  development   and  performance  of  coalitions23  (phase  2  of  the  PE  Programmatic  Approach)  and  on  what  Regional   Managers  consider  to  be  their  best  performing  programmatic  coalition  in  the  region.       Main  research  questions     The  main  research  questions  relate  to  the  above  mentioned  six  key  elements  of  the  programmatic   approach,  the  ‘pearls’,  which  together  are  assumed  to  contribute  to  systemic  change.  Coalitions,  i.e.   groups  of  organisations  and  institutions  that  purposely  join  forces  to  collectively  address  a  certain   key  problematic  (‘a  big  issue’),  could  be  considered  as  ‘instruments’  to  achieve  such  change.  In  the   field  study,  the  coalitions  and  its  members  are  the  subject  of  study  through  which  deepened  insight   into  the  PA  will  be  gained.  We  make  a  distinction  between  members  of  a  coalition  and  stakeholders,   considering  the  latter  as  a  broader  group  of  actors  that  are  not  necessarily  involved  in  the  full   process  to  address  a  key  problematic  (starting  from  a  joint  analysis  to  monitoring  and  reflection).   During  the  field  study  primary  attention  will  be  on  the  coalition  and  its  members  and  there  where   relevant  other  stakeholders  will  be  included.     During  the  field  study,  the  researchers  will  address  the  following  general  questions  for  all  pearls   distinguished:   • To  what  degree  has  the  pearl  concerned  indeed  materialised  by  or  in  the  specific  context  of   the  coalition?   • What  concretely  have  the  members  of  a  coalition  done  in  this  respect,  how  have  they  done  it   and  when  did  they  do  it?   • Who  of  the  coalition’s  members  have  been  involved  in  the  process,  what  roles  has  each  of   them  played  and  how  did  they  work  together?   • What  factors  in  particular  helped  to  make  the  process  a  success?     • If  the  pearl  concerned  appears  not  to  have  materialised  to  a  significant  degree,  how  has  the   coalition  addressed  the  issues  at  stake  (‘how  has  it  worked  around  the  issues,  or,  what   alternative  path  has  been  followed)?  Are  there  any  specific  conditions  to  be  indicated  for   this?     Apart  from  the  general  questions  above  that  will  be  addressed  for  all  pearls24,  the  following  specific   questions  will  be  addressed  as  well  for  a  number  of  pearls.  

22

See John  Kania  &  Mark  Kramer  (2011)  Collective  impact.  Standford  Social  Innovation  Review.  Winter  2011,     where  reference  is  made  to  five  conditions  of  successful  collective  impact  in  initiatives,  i.e.  a  common  agenda,   shared  measurement  systems,  mutually  reinforcing  activities,  continuous  communication,  and  backbone   support  organizations.   23  Strong  performers  are  programmatic  coalitions  who  -­‐  according  to  answers  on  given  on  the  survey  -­‐  perform   well  to  excellent  on  3-­‐5  of  the  pearls.   24  Concerning  pearl  1  (for  change  processes  to  be  relevant,  they  are  to  be  grounded  in  a  thorough  and   continued  understanding  of  the  ever-­‐changing  context,  issues  at  stake,  power  relations  amongst  the  major   actors  and  the  dynamics  on  how  these  actors  relate  to  one  another  and  to  the  systems  of  which  they  are  part)   Annex  5  –    ToR  field  study  

53


Pearl 2:  for  change  to  happen,  working  at  multiple  levels  with  multiple  stakeholders  is  necessary.       • Is  the  full  ARE  IN25  configuration  included  in  the  coalition?   • Are  other  stakeholders  included  in  the  coalition’s  activities  and  if  so,  on  which  issues  do  they   cooperate  and  how  are  the  working  relations  maintained?   • Is  the  coalition  and,  where  applicable,  the  multi-­‐stakeholder  configuration  composed  of  the   right  players  to  address  the  problematic  in  its  systemic  character?     Pearl  3:  for  stakeholders  to  indeed  join  forces  effectively  and  enter  into  some  form  of  joint  and   emergent  action,  a  shared  vision  on  the  change  and  transformation  to  be  promoted  and  about  how   to  go  about  that  (in  other  words  a  mutual  consensus  on  a  ‘theory  of  change’)  seems  crucial.   • To  what  degree  did  the  strategic  planning  process  result  in  a  clear  agenda  of  action?  Does   the  agenda  concern  the  coalition  as  a  whole  or  mostly/only  the  activities/  actions  of  specific   members?     • To  what  degree  did  the  strategic  planning  process  result  in  clear  roles  of  members  of  the   coalition  building  on  and  reinforcing  each  other’s  strengths?       Pearl  4:  for  change  to  be  sustainable,  it  seems  crucial  to  address  ‘the  big  issues’,  the  systems  that   generate  poverty  and  injustice.  This  pearl  closely  relates  to  previous  pearls.  The  following  questions   are  relevant  to  be  addressed  in  the  field  study.     • To  what  degree  does  the  agenda  for  action  (pearl  3)  effectively  address  the  big  issues  (based   on  the  contextual  and  institutional  understanding  -­‐  pearl  1)  and  what  assumptions  are   underlying  the  relation  between  the  two?   • To  what  degree  does  the  ‘big  issue’  guide  the  daily  practice/  implementation  of  the   activities/  action  of  the  coalition?       Pearl  5:  for  change  to  be  effective,  learning  on  what  happens,  on  what  works  and  what  does  not  and   consequently  flexibly  adapting  one’s  set  of  activities  is  key.       • To  what  degree  does  the  coalition  have  a  shared  measurement  system?26   • How  are  single  loop  learning  (are  we  doing  things  right),  double  loop  learning  (are  we  doing   the  right  things)  and/or  triple  loop  learning  (are  we  testing  our  assumptions  about  how   change  happens)  taking  place  and  how  is  this  embedded  in  the  daily  practice?     Methods  of  data  collection  and  analysis   This  part  of  the  evaluation  will  be  carried  out  in  the  form  of  a  field  study.  The  fieldwork  will  be   carried  out  with  3  coalitions,  and  is  designed  to  find  answers  on  the  main  questions  above.         The  field  research  will  be  undertaken  through  analysis  of  existing  documents  (context  analyses,   theories  of  change,  MoU’s,  agenda’s  and  plans  for  action,  PMEL  systems)  and  through  interviews   with  various  stakeholders,  focus  group  discussions,  workshops  bringing  together  coalition  members,   where  possible  drawing  on  network  analyses  and  power  analyses  to  complement  the  research.       Before  the  start  of  the  field  study,  the  evaluators  will  propose,  in  consultation  with  the  coordinator   Capacity  Development  and  the  PMEL  unit  a  short  list  of  program  coalitions  to  be  visited.  Main  criteria   for  selection  are:    -­‐   a  high  score  on  the  pearls  in  the  survey;   -­‐   as  much  as  possible  matching  the  pearls  mentioned  by  the  RMs   and  pearl  6  (relations  amongst  the  cooperating  parties  are  based  upon  trust  and  mutual  accountability)  the   general  questions  will  be  sufficient  to  gain  a  deepened  insight.   25  ARE  IN:  those  with  Authority,  Resources,  Expertise,  Information,  Need/  Stake     26  See  footnote  4.   Annex  5  –    ToR  field  study  

54


-­‐ not  included  in  the  action  researches  or   -­‐     part  of  the  first  phase  of  the  PE  PA   The  coalition  distribution  in  a  country  must  be  such  that  the  evaluation  can  be  done  within  its   timeframe.     The  division  of  the  field  studies  among  the  consultants  is  the  following:   -­‐ Verona  Groverman:  2  field  visits   -­‐ Kees  Zevenbergen:  1  field  visit   The  field  visits  will  be  carried  out  in  the  period  November  –  beginning  of  December  2013     In  the  final  analysis  stage,  after  the  field  study,  the  various  data  and  findings  will  be  compared  and   analysed  to  answer  the  research  questions.     Deliverables   A  concise  report  on  the  main  findings  of  the  field  study,  and  an  adapted  overview  of  quality  criteria   for  the  6  pearls,  their  assessment  in  practice  and  points  for  attention  for  working  with  the  pearls.   Based  on  the  3  sub-­‐reports  (phase  1,  2  and  3)  a  short  summarising  report  is  delivered,  in  which  the   questions  of  the  ToR  are  answered,  recommendations  to  ICCO  Cooperation  are  given  as  well  overall   lessons  learned  .  The  emphasis  of  the  overall  report  will  be  on  the  recommendations  to  ICCO   Cooperation  regarding  how  to  define  the  Programmatic  Approach  in  a  conceptually  clear  manner   (including  proposing  a  new  ‘name’),  and  detailing  what  the  preconditions  that  need  to  be  fulfilled  to   be  able  to  promote  as  ICCO  Cooperation  the  flourishing  of  the  ‘pearls’.  If  in  the  context  of  the   evaluation  (a)  new  pearl(s)  is  (are)  identified  these  need  to  be  included  in  the  report.  In  this  sense   the  summarizing  report  needs  to  have  a  ‘formative’  character.   This  summarizing  report  will  be  delivered  before  December  31,  2013.        

Annex 5  –    ToR  field  study  

55


Annex -­‐  Pearls  of  the  IA  Practice  –  a  first  attempt  at  quality  criteria  

A number  of  elements  of  the  programmatic  approach  are  however  much  cherished  by  most  people   interviewed,  and  indeed  often  mentioned  in  explaining  the  IA  way  of  working  to  partners  and   outsiders  alike.  These  elements  are  considered  to  constitute  the  ‘pearls  of  the  IA  practice’  that  are   most  definitely  to  find  their  way  into  the  future  formulations  of  the  IA  way  of  working:     Pearls  of  Practice   What  is  needed?   How  to  assess?   for  change  processes  to  be   • Context  analysis   • Regular  appreciation  of   relevant,  they  are  to  be  grounded   (geography  and  theme   availability,  quality  and   in  a  thorough  and  continued   specific)   relevance  of  analyses   understanding  of  the  ever-­‐ • Problem  tree   changing  context,  issues  at  stake,   • Actor  constellation  map   power  relations  amongst  the   • Drivers  of  Change   major  actors  and  the  dynamics   analysis   on  how  these  actors  relate  to  one   • Political  Economy   another  and  to  the  systems  of   analysis   which  they  are  part;   • Conflict  Transformation   analysis   • Grassroots   Democratisation  analysis   • Power  analysis   • Value  Network  Analysis   • All  regularly  updated  and   shared  by  all  concerned   for  change  to  happen,  working  at   • Actor  constellation  map   • Regular  appreciation   multiple  levels  with  multiple   whether  all  actors   • Value  Network  Analysis   27 stakeholders  is  necessary;   relevant  for  change  are   • Use  ARE  IN  principle   engaged  in  the  agenda  for   action   for  stakeholders  to  indeed  join   forces  effectively  and  enter  into   some  form  of  joint  and  emergent   action,  a  shared  vision  on  the   change  and  transformation  to  be   promoted  and  about  how  to  go   about  that  (in  other  words  a   mutual  consensus  on  a  ‘theory  of   change’)  seems  crucial;   for  change  to  be  sustainable,  is   seems  crucial  to  address  ‘the  big   issues’,  the  systems  that  generate   poverty  and  injustice;  

Theory of  Change   Agenda  for  action   Consensus  thereon  by   parties  concerned  

Regular appreciation  of   quality  of  ToC  and  of   effective  engagement   thereof  by  parties   concerned  

Convincing story  on  how   the  agenda  for  action   effectively  addresses  and   changes  the  systems…     • Using  methodology  that   allows  for/  promotes   emergence  of  systemic   changes  often  marked   by  complexity    

Pitches accepted  at   decision-­‐making  bodies   Strategies  developed   regularly  updated  and   ‘validity’  tested   Assumptions  explicit  and   used  in  PMEL  

• • •

27

Those  with  Authority,  Resources,  Expertise,  Information,  Need  (stake)  are  welcomed  and  participate  actively.  

Annex 5  –    ToR  field  study  

56


Cynefin framework   used  to  determine   strategy   Four  Quadrants   framework  used   Emergence  sought.  

for change  to  be  effective,   learning  on  what  happens,  on   what  works  and  what  does  not   and  consequently  flexibly   adapting  one’s  set  of  activities  is   key;   relations  amongst  the   cooperating  parties  are  based   upon  trust  and  mutual   accountability.    

• • • Learning  agenda  and   investments   • Monitoring  systems   operational   • Flexible  and  rolling   planning  and  budgeting   logics   •

• • • •

Horizontal and   downward  accountability   mechanisms   Peer  to  peer  exchanges   Transparent  governance   set-­‐up   Use  ARE  IN  principle28   ICCO  is  steward  in  trust   building,  relations   building  and  mutual   accountability  

Regular appreciation  of   quality  of  PMEL  systems   (including  the  effective   use  made  thereof)   Theory  of  Change  and   assumptions     incorporated  in  PMEL   Regular  appreciation  of   quality  of  cooperation   through  self  –assessment   using  Pscan  

Key  issue  is  on  the  ownership  of  the  change  processes.  It  can  be  argued  that  the  owner(s)  of  these   processes  are  the  ones  that  determine  the  quality  criteria  and  that  are  to  uphold  these.  In  other   words:  the  owners  of  the  change  processes  are  to  regularly  appreciate  to  what  degree  the  various   elements  of  ‘good  practice’  are  present  and  up  to  standards.

28

Those  with  Authority,  Resources,  Expertise,  Information,  Need  (stake)  are  welcomed  and  participate  actively.  

Annex 5  –    ToR  field  study  

57


Annex 6  -­‐  Report  of  the  field  (third)  phase  of  the  Evaluation  Programmatic  Approach      

Evaluation Programmatic  Approach  IA   FINDINGS FIELD PHASE INTRODUCTION

This paper  presents  the  findings  of  the  field  phase  of  the  evaluation  of  the  programmatic  approach   as  shaped  and  implemented  by  the  ICCO  Alliance  (IA).     The  outcomes  of  the  first  phase  of  the  evaluation  (a  phase  during  which  the  evaluators  read  and   analysed  all  material  available  on  the  launch,  emergence,  and  implementation  of  the  programmatic   approach,  including  material  on  the  subsequent  learning  thereon,  and  during  which  additional  data   was  gathered  in  dialogue  with  key  players  in  the  ICCO  Alliance)  were  laid  down  in  a  document   entitled  “Evaluation  Programmatic  Approach  IA  –  Initial  Synthesis  of  Findings”  (September  2013).   These  findings  were  presented  to  and  discussed  with  IA  leadership  during  their  September  MT   meeting.     The  research  and  dialogues  revealed  that  a  number  of  elements  of  the  programmatic  approach  are   much  cherished  by  most,  and  indeed  often  mentioned  in  explaining  the  IA  way  of  working  to   partners  and  outsiders  alike.  These  elements  are  considered  to  constitute  the  ‘pearls  of  the  IA   practice’.  These  ‘pearls’  are  shortly  on:   • anchoring  interventions  in  context  and  local  dynamics;   • working  with  multiple  actors  at  multiple  levels;   • sharing  a  vision  on  change  and  how  to  achieve  that;   • addressing  the  big  issues;   • learning  and  adapting;  and  on   • creating  trust  and  mutual  accountability.   INTRODUCING  THE  FIELD  PHASE  OF  THE  RESEARCH   The  purpose  of  the  field  studies  was  to  unearth  the  lessons  learned  from  three  coalitions  that  are   considered  to  be  interesting  and/or  successful  and  to  formulate  guidance  for  the  further   development  of  the  Programmatic  Approach.  The  main  objective  for  the  field  studies  was  to  shed   more  light  on  the  questions  of  why,  how  and  what  exactly  a  number  of  coalitions  have  done  to   become  strong  performers,  including  an  analysis  of  the  circumstances  and/or  preconditions  that   might  have  contributed  to  their  successful  emergence  (including  the  roles  that  IA  staff  played  therein   as  ‘backbone  support  organisation’).   During  the  field  study,  the  researchers  addressed  the  following  general  questions  for  all  pearls   distinguished:   • To  what  degree  have  the  six  pearls  indeed  materialised  in  the  specific  context  of  the  coalition?   • What  concretely  have  the  members  of  a  coalition  done  in  this  respect,  how  have  they  done  it   and  when  did  they  do  it?   • Who  of  the  coalition’s  members  have  been  involved  in  the  process,  what  roles  has  each  of  them   played  and  how  did  they  work  together?   • What  factors  in  particular  helped  to  make  the  process  a  success?    

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

58


If a  pearl  appears  not  to  have  materialised  to  a  significant  degree,  how  has  the  coalition   addressed  the  issues  at  stake  (‘how  has  it  worked  around  the  issues,  or,  what  alternative  path   has  been  followed)?  Are  there  any  specific  conditions  to  be  indicated  for  this?  

In brief:  the  purpose  of  the  field  research  was  to  better  understand  and  document  the  conditions   and  factors  that  contributed  to  the  emergence  of  successful  coalitions.     INTRODUCTION  OF  THE  THREE  COALITIONS   The  selection  of  the  three  coalitions  to  be  visited  was  based  upon  an  analysis  of  the  results  of  the   online  survey  and  furthermore  on  what  Regional  Managers  considered  to  be  their  best  performing  or   most  interesting  coalitions  in  the  region.29     During  November  and  December  2013,  the  evaluators  visited  the  following  three  coalitions:   • in  Vietnam:  the  Vietnam  Rivers  Network;   • in  Uganda:  the  Ugandan  Health  Programme;  and   • in  Senegal:  le  Programme  Multi  Acteurs.   During  their  field  work,  the  evaluators  had  meetings  with  all  relevant  staff  of  participating   organisations,  exchanged  with  the  coalitions  leadership,  interviewed  individual  coalition’s  members   and  some  other  stakeholders,  and  held  workshops  with  all  concerned  to  unearth  the  lessons  that   could  be  learned  from  the  collaborating  efforts  of  the  organisations  concerned.   Let  us  first  introduce  the  three  coalitions.  This  will  help  in  understanding  the  various  observations  of   the  evaluators.     The  Vietnam  Rivers  Network  (VRN)  –  VRN  works  to  protect  the  health  of  riverine  communities  and   ecosystems.  In  November  2005  a  few  concerned  individuals  established  VRN.  VRN  has  by  now   evolved  in  an  open  forum  whose  membership  includes  NGOs,  researchers,  academics,  government   officials,  local  communities,  and  individuals,  all  concerned  with  protection  of  rivers  and  river-­‐ dependent  communities  and  sustainable  development  in  Vietnam.     In  2007/2008  VRN  went  through  a  process  of  strategy  development  resulting  in  VRN’s  Strategy  2008-­‐ 2020,  based  on  which  annual  action  plans  are  made  and  results  reviewed  in  annual  meetings  open  to   all  members.  Its  activities  include  sharing  information,  conducting  research  on  social  and   environmental  impacts  caused  by  hydropower  projects  and  other  water-­‐related  development   projects  in  Vietnam,  and  doing  advocacy  on  these  issues.  VRN  has  been  successful  in  advocacy  in   which  VRN  members,  communities  and  other  local  stakeholders  are  actively  involved.  Through  its   careful,  transparent  and  evidence  based  approach,  VRN  has  built  up  reputation  and  government   recognition.   VRN  has  set  up  a  structure  with  a  management  team,  composed  of  three  key  organisations  (each   operating  in  one  of  the  regions  of  Vietnam)  and  a  secretariat,  taken  up  by  one  of  the  three  key   organisations  on  a  rotational  basis.  A  third  important  entity  in  the  structure  is  the  advisory  board   composed  of  (active)  individual  members.     The  three  key  organisations  provide  an  important  link  with  the  communities.  Their   programmes/projects  focus  on  awareness  raising,  building  capacities  on  issues  related  to  water   governance,  and  empowerment  of  communities  which  helps  male  and  female  community  members   to  voice  their  concerns  and  make  their  voices  heard  through  the  linkages  that  the  organisations   establish  with  authorities  at  different  levels.  At  the  same  time  they  make  authorities  more  conscious   of  the  fact  that  community  members  have  important  issues  to  bring  forward.     ICCO  has  been  one  of  the  main  funders  of  the  VRN  evidence-­‐based  advocacy  programme  since  2009.     VRN  has  another  core  funder  and  a  few  other  donors  for  specific  activities.  Funds  provided  by  IA  are   29

Some very likely candidates were not included in the sample as they had participated in learning exercises or action researches, or as they were evaluated recently, or because visiting them was logistically not possible. Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

59


used for  capacity  building  of  VRN  members;  empowerment  of  affected  communities  through  rights   promotion,  funding  and  setting  up  the  community-­‐based  river  monitoring;  research  to  create   evidence  for  advocacy  work;  advocacy  related  to  laws  on  water  resources  and  downstream  impacts   of  hydropower  dams.  The  role  of  the  IA  programme  officer  is  mainly  facilitating  with  some  brokering.   In  terms  of  decision-­‐making,  VRN  is  totally  independent  from  any  donor.     The  Ugandan  Health  Programme  (UHP)  –  UHP  is  a  coalition  of  partners  of  four  IA  members  (Tear,   TWR,  Woord  en  Daad,  ICCO).  In  2011,  stimulated  by  the  IA,  the  partners  joined  forces  to  implement   and  monitor  a  MFSII–funded  IA  thematic  country  programme  related  to  health/HIV&AIDS.  The   programme’s  two  objectives  are:  1)  Well-­‐established  accountability  mechanisms  in  which  civil  society   effectively  calls  the  health  system  to  account  for  the  delivery  of  equally  accessible  basic  health  care;   and  2)  Capacitated  change  agents  through  which  civil  society  promotes  effective  prevention  of  SRH   problems,  HIV  transmission  and  disabilities.  The  UHP  has  a  strong  funding  relation  with  IA,  the  IA   being  its  main  funder.  The  IA  regional  office  also  plays  an  important  role  in  brokering  and  supporting   capacity  development.  The  members  of  UHP  feel  a  strong  ownership  of  the  coalition  and  take   decisions  independently.     The  membership  of  UHP  is  ‘closed’  and  consists  of  six  ‘like-­‐minded’  NGOs  mainly  working  on  grass   roots  level  in  the  North  and  North  East  of  Uganda.  Five  of  them  have  been  involved  in  the   establishment  and  development  of  the  UHP  and  the  sixth  member,  focusing  on  advocacy,  joined   rather  recently.     UHP  members  implement  programme  activities  that  are  in  line  with  their  own  mandate,  approach   and  geographical  area.  These  fit  nicely  with  the  IA  ambitions  in  health.  Apart  from  these  projects,  the   members  engage  in  joint  UHP  activities:  capacity  building  of  the  members,  sharing  of  experiences   during  meetings,  exchange  visits  to  each  other’s  projects,  and  collaborating  in  joint  events.  UHP  does   not  jointly  engage  in  advocacy;  presently  one  member  organises  an  advocacy  campaign  and  others   join  in.     UHP  has  a  light  structure:  a  (paid)  coordinator,  steering  group,  and  financial  controller.  The  members   hold  (bi-­‐annual)  meetings  for  review,  sharing,  reflection  and  planning  of  common  activities.     The  ‘Programme  Multi  Acteurs’  (PMA  Senegal)  –  PMA  is  an  IA-­‐financed  programme  implemented  by   five  Senegalese  NGOs.     ICCO  has  financed  activities  of  individual  Senegalese  NGOs  since  the  80’s  of  the  last  century.  These   activities  were  mainly  in  the  field  of  education  and  literacy.  The  introduction  of  the  programmatic   approach  has  led  the  IA  to  actively  stimulate  the  collaboration  between  the  Senegalese  NGOs,  which   has  resulted  in  the  creation  of  a  ‘Consortium  Multi  Acteurs’  in  2009.  This  coalition,  at  the  time   composed  of  13  Senegalese  NGOs,  launched  a  ‘Programme  Multi  Acteurs  Education’  during  2009   (PMA-­‐1).  The  coalition  entered  a  rather  painful  process  of  reformulating  its  activities  and  of  reducing   its  membership  as  a  consequence  of  budget  reductions,  some  reported  irregularities  within  a  partner   organisation  and  of  ICCO’s  choice  to  terminate  its  support  to  education  programmes  and  to  refocus   on  vocational  training  and,  later  on  in  the  process,  also  on  sustainable  economic  development   through  value  chain  development.  This  process  resulted  eventually  in  the  current  ‘Programme  Multi   Acteurs  2  Second  Phase’  covering  activities  in  non-­‐formal  education,  technical  and  vocational   training,  value  chain  development  and  sexual  and  reproductive  health.     PMA-­‐2  is  implemented  by  five  Senegalese  NGOs  (of  which  two  are  national  offices  of  international   NGOs).  The  activities  of  the  Consortium  are  presented  in  a  single  (multi-­‐)  annual  planning  document.   Based  on  this  document,  the  five  individual  organisations  each  present  their  activities  in  separate   annual  plans  and  budgets  that  are  approved  in  bilateral  contacts  between  the  IA  regional  office  in   Bamako  and  the  individual  NGOs.   The  coalition  is  led  by  a  ‘comité  de  pilotage’  and  coordinating  work  is  done  by  a  focal  point  (one  of   the  five  NGOs)  and  a  programme  coordinator.  The  programme  coordinator  organises  regular   planning  and  monitoring  meetings.     Annex  6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

60


The IA  staff  of  the  regional  office  plays  a  very  ‘hands  on  role’  both  in  determining  the  thematic   direction  of  the  activities  undertaken,  as  well  as  in  the  individual  planning  and  budget  approval   phase.     INITIAL  OBSERVATIONS   The  three  coalitions  visited  are  very  different  in  many  a  way:   • genesis  –  the  VRN  is  established  by  individuals  driven  by  an  endogenous  will  to  unite  and  join   forces,  whereas  the  UHP  and  the  PMA  are  formed  on  the  instigation  of  the  IA;   • composition  –  the  VRN  is  composed  of  a  rich  pallet  of  organisations  and  individuals  (NGOs,   research  institutes,  local  communities,  advocacy  organisations,  government  officials),  whereas   the  UHP  and  the  PMA  are  composed  of  more  or  less  comparable  NGOs  with  which  the  individual   IA  members  have  had  prior  (often  long-­‐standing)  funding  relations.  The  UHP  has  however   recently  been  expanded  with  a  dedicated  advocacy  outfit;   • raison  d’être  –  whilst  the  core  of  the  VRN  activities  is  on  joint-­‐action  (evidence-­‐based  advocacy),   the  core  of  the  UHP  and  PMA  programme  is  on  implementing  mainly  field-­‐based  activities  of  the   individual  NGOs.  (Indicative  for  this  is  already  in  the  names  of  the  coalitions:  VRN  being  a   network  and  UHP  and  PMA  being  programmes);   • programming  –  linked  to  the  above,  the  VRN  programme  presents  only  the  activities  to  be   undertaken  by  the  network  or  for  the  network  purposes,  whilst  the  UHP  and  PMA  programmes   mainly  consist  of  activities  to  be  undertaken  by  the  individual  NGOs  with  only  very  limited   attention  for  joint-­‐activities;   • external  orientation  –  VRN  is  very  much  oriented  towards  the  outside  world  in  realising  its  core-­‐ business  (evidence-­‐based  advocacy)  and  readily  and  easily  engages  with  non-­‐member  parties  if   it  is  in  its  interests.  UHP  and  PMA  are  more  internally  oriented;   • funding  –  the  IA  is  the  sole  funder  of  the  UHP  and  PMA  programmes  (although  the  individual   partners  do  sometimes  have  other  sources  of  funding  for  the  other  activities  they  undertake),   creating  a  great  dependency  on  the  IA  for  both  the  implementation  of  the  individual  activities  as   well  as  of  the  continuation  of  the  collaboration.  VRN  has  a  broad  funding  base  for  both  its  joint   activities  as  well  as  for  the  activities  of  the  individual  members,  increasing  its  autonomy  and   independency  in  decision-­‐making;   • areas  of  expertise  –  the  VRN  and  the  UHP  undertake  activities  in  thematic  areas  in  which  they   have  experience  and  a  solid  track  record;  the  programmes  they  implement  concern  their  core   business.  The  PMA,  especially  after  its  reorientation,  is  taking  its  partners  in  thematic  areas  in   which  they  do  not  have  any  (or  very  limited)  experience  or  track  record  (especially  with  regards   to  business  development  services,  value  chain  development  and  support  to  economic  activities);   • IA  support  –  IA  support  to  the  VRN  mainly  consists  of  moral  and  financial  support,  whereas  IA  in   Uganda  also  provides  for  active  in  situ  tailored  capacity  building  activities  and  facilitating   support  to  UHP,  and,  to  a  minor  extent,  brokering.  IA  support  to  PMA  in  Senegal  mainly  consists   of  financial  support,  more  general  capacity  building  support  through  trainings  in  Bamako  and   through  specific  guidance  on  the  strategic  direction  of  the  programme;   • visibility  of  the  coalition  –    VRN  has  a  very  clear  online  profile  and  mentions  ICCO  as  one  of  its   donors.  The  UHP  is  invisible  online.  The  PMA  is  equally  invisible  online.  The  five  participating   NGOs  do  not  mention  the  PMA  on  their  websites  and  only  few  mention  ICCO  as  a  donor.  The  IA  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

61


website does  not  mention  PMA  and  it  states  that  ‘The  ICCO  Cooperation  runs  an  Education   program  in  Senegal’.   business  opportunities  for  IA  –  from  a  more  business-­‐oriented  perspective,  the  picture  is   equally  diverse.  The  issue  is  to  what  extent  the  IA  can  raise  its  profile  or  increase  its  fundraising   potential  through  support  to  either  one  of  the  three  coalitions.    For  VRN,  the  IA  role  as  a  funder   is  important,  but,  as  the  VRN  activities  focus  on  water  security  and  advocacy  on  environmental   issues,  profiling  on  the  major  IA  themes  does  not  seem  directly  obvious.  Rebranding  might   improve  the  situation.  For  PMA,  the  current  thematic  fit  seems  better,  although  one  might  raise   the  question  whether  the  reorientation  of  the  programme  does  not  call  for  a  reorientation  to  a   partner-­‐set  better  equipped  for  intervening  in  value  chains.  For  UHP,  IA  business  opportunities   seem  best:  coalition  of  trusted  partners,  thematic  fit,  well  appreciated  role  of  IA  staff.  Challenge   here  is  to  profile  the  IA  as  a  provider  of  crucial  and  valued  capacitating,  brokering,  and   facilitating  services  that  merits  further  support  in  the  future.  

The evaluators  have  not  analysed  the  degree  to  which  the  three  coalitions  produce  their  outputs  and   achieve  their  outcomes.  They  are  therefor  not  able  to  relate  the  abovementioned  diversity  to  any   significant  differences  in  the  effectiveness  of  their  operations.  The  evaluators  have  however  analysed   the  various  coalitions  on  the  degree  to  which  they  relate  to  the  various  key-­‐elements  of  the  IA   programmatic  approach,  the  ‘pearls  of  the  IA  practice’.  Such  under  the  assumption  that  the  better   that  one  scores  on  these  pearls,  the  more  likely  it  is  that  the  coalitions  are  successful  in  their   endeavours  in  pursuing  and  eventually  realising  sustainable  system’s  change.     MORE  DETAILED  ANALYSIS  ON  THE  PEARLS  OF  THE  IA  PRACTICE   Pearl  1:  For  change  processes  to  be  relevant,  they  are  to  be  grounded  in  a  thorough  and  continued   understanding  of  the  ever-­‐changing  context,  issues  at  stake,  power  relations  amongst  the  major   actors  and  the  dynamics  on  how  these  actors  relate  to  one  another  and  to  the  systems  of  which  they   are  part.   All  coalitions  have  in  one  way  or  another  tried  to  analyse  the  context  in  which  they  operate  and   define  and  implement  their  activities.  Quality,  completeness  and  timeliness  however  vary.     Positive  developments:   • all  realise  that  thorough  understanding  of  context  is  very  important  to  shape  and  implement   relevant  and  effective  interventions;   • all  invest  time  and  resources  in  acquiring  that  understanding;   • more  thorough  analyses  are  made  when  results  of  researches  (often  executed  by  professional   researchers)  are  discussed  and  analysed  during  (multi-­‐stakeholder)  workshops  with  at  least  the   participation  of  the  key-­‐players  of  the  organisations  engaged  in  the  coalition;   • the  better  analyses  are  made  when  they  zoom  in  from  a  more  generic  level  to  a  well  defined   and  specific  geographic  or  thematic  area.   Issues  for  further  reflexion:   • most  context  analyses  are  produced  in  function  of  the  formulation  of  programmes  to  be   submitted  to  donors.  They  are  often  a  chapter  in  a  document  that  is  only  updated  when  a  new   proposal  is  to  be  submitted;   • most  context  analyses  are  situation  or  problem  oriented.  They  sketch  the  nature  and  the   magnitude  of  the  problems  in  a  given  area  and  situation.  Rare  are  the  analyses  that  ask  why   questions  and  that  unearth  the  underlying  root  causes  of  the  problems  indicated  and/or  the   Annex  6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

62


underlying power  dynamics  that  either  contribute  in  perpetuating  the  problems  indicated  or   that  could  be  mustered  and  harnessed  to  stimulate  change  processes  to  sustainably  deal  with   these  root  causes;   tools  and  methods  for  solid  and  repeated  contextual  research  and  analysis  indicated  in  the  IA   guidance  note  on  the  programmatic  approach  are  seldom  used.    

Pearl 2:  For  change  to  happen,  working  at  multiple  levels  with  multiple  stakeholders  is  necessary.   All  coalitions  work  at  multiple  levels  with  multiple  stakeholders,  although  at  varying  degree.  The   extent  to  which  coalitions  include  those  with  authority,  resources,  expertise,  information  and  need   or  stake  (ARE  IN)  in  their  actions  however  varies.     Positive  developments:   • all  partners  currently  active  in  the  three  coalitions  come  from  a  background  where  they  went   about  their  business  more  or  less  alone.  In  most  coalitions,  the  IA  has  played  a  valuable  and   cherished  role  in  bringing  them  together  and  in  supporting  them  in  shaping  their  collaboration;   • most  valued  in  multi-­‐stakeholder  settings  are  the  opportunities  to  generate  more  clout  in   achieving  ones  objectives  and  to  learn  from  one  another;   • most  of  the  coalitions  have  emerged  from  a  stakeholder  analysis  during  the  start-­‐up  phase,   which  however  hardly  included  an  analysis  of  the  power  dynamics  between  actors;   • the  better  coalitions  are  those  that  indeed  include  those  with  ARE  IN.  Even  better  coalitions  are   those  that  not  only  include  ARE  IN,  but  ARE  IN  at  the  various  levels  needed  to  generate  effective   and  lasting  change;   • some  coalitions  have  achieved  results  the  individual  members  could  have  never  achieved  alone,   especially  with  regards  to  effective  evidence-­‐based  advocacy  in  which  real  life  stories  and  hard   data  are  coupled  to  community  mobilisation,  top-­‐end  communication  and  policy  influencing  up   to  the  highest  levels;   • successful  multi-­‐level  multi-­‐actor  collaboration  depends  more  on  highly  engaged  and  motivated   individuals  than  on  the  formal  engagements  between  organisations.   Issues  for  further  reflexion:   • most  coalitions  are  composed  of  comparable,  like-­‐minded,  NGOs  that  do  more  or  less  the  same   thing  in  different  geographical  areas.  Few  are  the  coalitions  that  join  actors  that  truly  add  value   to  one  another  on  their  typical  area  of  expertise  in  a  concerted  way  on  a  specific  change   process;   • most  actors  in  coalitions  do  things  or  the  type  of  things  that  they  used  to  do  before  and  invest   only  very  limited  resources  in  joint-­‐action  or  actions  for  which  they  truly  need  each  other;   • joint-­‐action  is  in  general  more  focussed  on  strengthening  the  capacities  of  the  coalition’s   members  than  on  system-­‐change  oriented  coalition  activities;   • most  coalitions,  especially  the  ones  that  are  created  on  instigation  of  the  IA,  are  not  composed   of  those  partners  that  really  need  each  other  to  launch  successful  change  processes,  but  are   mainly  composed  of  partners  with  which  the  IA  had  a  prior  funding  relation;   • most  coalitions  could  generate  more  success  when  investing  more  on  evidence-­‐based  advocacy   activities  either  themselves  or  through  including  experienced  researchers,  communicators  and   lobbyists  in  their  coalitions.  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

63


Pearl 3:  For  stakeholders  to  indeed  join  forces  effectively  and  enter  into  some  form  of  joint  and   emergent  action,  a  shared  vision  on  the  change  and  transformation  to  be  promoted  and  about  how   to  go  about  that  (in  other  words  a  mutual  consensus  on  a  ‘theory  of  change’)  seems  crucial.   All  coalitions  have  developed  a  multi-­‐annual  (strategy)  plan,  often  including  a  vision  and  mission   statement.  The  degree  to  which  the  rationale  for  joint-­‐action  is  developed  and  the  degree  to  which   pathways  towards  change  and  transformation  are  identified  (either  written  or  in  spirit)  however   vary.     Positive  developments:   • all  coalitions  have  engaged  in  consultative  and  collaborative  efforts  to  develop  (multi-­‐annual)   planning  documents;   • most  activities  are  geared  to  uplifting  the  situation  of  poor  and  vulnerable  people;   • the  better  coalitions  do  not  only  have  a  shared  vision  on  the  change  and  transformation  to  be   promoted,  but  have  also  clearly  indicated  why  and  how  the  participating  organisations  need   each  other  to  effectively  support  those  and  realise  sustainable  change.  Or,  to  rephrase;   • the  better  coalitions  focus  more  on  concerted  and  joint-­‐action  to  generate  change  than  on  the   various  activities  of  the  individual  organisations;   • the  ‘theory  of  change  approach’  (ToC)  is  generally  considered  to  be  a  very  useful  approach  in   focusing  the  participating  organisations  on  the  real  issues  at  stake,  on  the  best  ways  and  means   to  jointly  go  about  them  and  on  who  could  do  what  best  to  generate  sustainable  results.   Issues  for  further  reflexion:   • the  ToC  is  still  in  its  infancy  stages.  Although  the  initial  trainings  on  the  ToC  have  generated   interest  and  enthusiasm,  in  situ  support  to  coalitions  in  actually  developing  a  tailored  ToC  was   solicited  in  most  cases  and  will  most  definitely  strengthen  them  in  developing  a  shared  vision  on   the  change  and  transformation  to  be  promoted  and  on  how  to  go  about  that;   • it  appears  to  be  of  key-­‐importance  that  this  support  is  well  facilitated  (and  that  is  probably  best   provided  by  external  facilitators)  as  the  process  of  developing  a  ToC  might  reveal  internal   weaknesses  in  (the  composition  of)  various  coalitions,  and/or  reveal  ownership  issues;   • although  gender  is  considered  a  cross-­‐cutting  issue  by  all,  only  members  working  at  the   community  level  address  gender  or  women’s  issues.  The  higher-­‐level  agenda’s  for  joint-­‐action   do  not  reflect  commitment  to  gender  equality.   Pearl  4:  For  change  to  be  sustainable,  it  seems  crucial  to  address  ‘the  big  issues’,  the  systems  that   generate  poverty  and  injustice.   This  is  a  tough  one!  All  coalitions  do  indeed  address  the  big  issues  that  complicate  the  lives  of  poor   and  vulnerable  people  (access  to  health,  water,  resources,  economic  opportunities,  etc.).  Rare  are   however  the  coalitions  that  address  their  underlying  causes,  the  systems  that  generate  poverty  and   injustice.       Positive  developments:   • the  initial  work  done  with  the  ToC  helps  to  get  better  insights  in  what  the  big  issues  are  and   which  their  systematic  dimensions  are;   • evidence-­‐based  advocacy  seems  to  provide  for  very  promising  (and  probably  lasting)   opportunities  to  generate  systems  change  (for  example:  empowered  communities  able  to  stop   dam  construction,  proof  that  joint-­‐action  can  indeed  make  a  difference,  laws  that  are  modified   to  better  protect  the  environment  and  interests  of  communities).  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

64


Issues for  further  reflexion:   • understanding  the  difference  between  ‘addressing  a  problem’  and  ‘addressing  its  root-­‐causes’   remains  a  challenge.  For  many  concerned  working  in  for  example  HIV/AIDS  is  addressing  a  big   and  systemic  issue.  Whilst  for  them  working  on  one  of  its  root-­‐causes,  for  example  working  on   why  especially  young  men  are  increasingly  promiscuous  (and  working  on  related  masculinity-­‐ issues)  is  far  less  likely;   • addressing  systems  that  generate  poverty  and  injustice  is  a  highly  politicised  and  often  risky   endeavour  that  does  not  only  require  solid  power  analyses  (that  are  often  lacking  as  indicated   above),  but  also  the  time  to  make  a  difference.  The  current  short-­‐term  and  results  focused  IA   contracts  might  not  favour  the  aspired  systems  change;   • more  effort  might  be  needed  to  align  with  and  strengthen  existing  government  structures  that   are  mandated  by  policy  frameworks.  NGOs  might  be  more  effective  by  being  more  transparent   and  open  to  promote  willingness  at  government  agencies.   Pearl  5:  For  change  to  be  effective,  learning  on  what  happens,  on  what  works  and  what  does  not  and   consequently  flexibly  adapting  one’s  set  of  activities  is  key.   All  concerned  realise  that  learning  is  of  key-­‐importance,  not  only  because  it  contributes  to  becoming   effective  and  efficient  agents  of  change,  but  also  because  the  process  of  learning  itself  enhances  the   quality  of  relations  and  interactions  within  a  coalition.  The  quality  of  learning  however  varies.     Positive  developments:   • all  coalitions  organise  (bi-­‐)annual  meetings  to  review  outputs  and  outcomes  of  the  annual  plans   with  a  focus  on  ‘are  we  doing  things  right’;   • factors  that  help  to  make  the  process  of  learning  (through  reviewing,  (documented)  peer   learning  and  exchange  visits)  and  adapting  a  success  are  the  active  participation  of  the   cooperating  parties,  and  the  inclusion  not  only  of  leadership,  but  especially  of  the  thematic   specialists  of  the  organisations  concerned.   Issues  for  further  reflexion:   • monitoring  systems  and  tools  are  often  of  poor  quality  and  generate  more  output  data  that   outcome  information;   • the  monitoring  efforts  are  focused  more  on  results  generated  by  the  individual  organisations   than  on  results  of  the  coalition;   • for  learning  to  happen,  one  needs  more  information  than  on  outputs  and  outcomes  only.   Learning  also  requires  process-­‐information.  ‘How  did  we  go  about  in  realising  this  result?’  This  is   especially  true  for  delicate  processes  of  successful  advocacy  which  are  currently  hardly   documented  and/or  analysed;   • very  little  double  and  triple  loop  learning  takes  place.  Questions  like  ‘are  we  doing  the  right   things’  and  ‘how  do  we  know  right  from  wrong’  are  seldom  asked.  Apart  from  the  fact  that   these  are  tough  questions,  people  and  their  organisations  are  often  so  occupied  by  the  daily   matters  at  hand  (and  the  often  very  heavy  planning  and  reporting  formats)  that  they  just  don’t   have  or  take  the  time  for  them.   Pearl  6:  Relations  amongst  the  cooperating  parties  are  based  upon  trust  and  mutual  accountability.   Yes,  the  better  these  relations  are,  the  more  likely  it  is  that  the  above-­‐mentioned  pearls  do  indeed   start  to  shine.       Positive  developments:   Annex  6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

65


• •

all coalitions  have  invested  in  relation  building  and  mutual  accountability;   important    ‘instruments’  to  promote  trust  and  mutual  accountability  are  regular  communication   of  updates  by  the  coordinator,  the  regular  bi-­‐annual  review  meetings  in  which  discussion  take   place  –  people  agree,  disagree,  and  respect  each  other,  peer-­‐to-­‐peer  exchanges,  a  clear  division   of  roles  and  responsibilities  among  partners,  involvement  of  leadership  in  the  coalition’s   activities,  shared  leadership  in  the  coalition,  and  a  felt  and  realised  ownership  over  the   coalition’s  direction.  

Issues for  further  reflexion:   • In  cases  in  which  coalitions  are  still  dealing  with  the  fall-­‐out  of  the  recent  reorientation  and   reshuffling  of  membership  as  promoted  by  IA,  they  are  more  of  a  marriage  of  convenience  than   based  upon  genuine  trust  and  respect;   • Transparent,  participatory  and  decentralised  processes  of  budget  allocation  and  approval   increase  the  quality  of  the  relations  amongst  the  cooperating  parties.   REVIEWING  THE  PEARLS  OF  THE  IA  PRACTICE…   Reviewing  the  three  coalitions  through  the  lens  of  the  six  pearls  proved  not  only  to  be  a  feasible   process,  but  also  to  be  a  highly  interesting  process  as  it  clarified  major  strengths  and  weaknesses  of   the  coalitions  and  the  areas  on  which  they  could  work  to  improve  and  strengthen  their  operations   and  increase  their  shot  at  effectiveness.     There  are  however  two  areas  that  seem  not  to  be  sufficiently  covered  by  the  six  pearls.  And  that  are   the  areas  that  have  to  do  with:   • the  ownership  over  and  quality  of  the  internal  organisation  of  the  coalition,  dealing  with  issues   like  decision-­‐making  over  strategic  direction,  planning  and  budgeting,  communications,   coalition-­‐wide  PMEL  and  financial  control,  capacity  building  and  institutional,  social,  economic   and  financial  sustainability;   • the  nature  and  quality  of  the  support  provided  by  the  backbone  support  organisation,  in  casu  by   the  IA.   Both  areas  are,  as  was  confirmed  during  the  three  field  visits,  like  the  earlier  mentioned  pearls,  of   critical  importance  in  generating  and  supporting  sustainable  change  processes.   The  field  visits  have  also  confirmed  the  validity  of  the  findings  and  initial  recommendations  of  the   first  phases  of  the  current  evaluation.  In  the  concluding  paper,  these  will  be  enriched  with  a  number   of  lessons  learned  during  the  field  visits  and  a  number  of  final  recommendations  towards  the  IA  on   how  to  further  strengthen  the  programmatic  approach  of  the  alliance.     Verona  Groverman,  Kees  Zevenbergen  –  February  2014      

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

66


Annex A    -­‐  documents  reviewed  field  visits     Literature  -­‐  Uganda   - Thematic  Country  Plan  2011-­‐2015,  Uganda  –  Health  and  HIV  &  AIDS  -­‐  Uganda  Country  Plan   2013,  undated   - Country  year  plan  2013  (May  2013,  updated  version)   - Ochen,  Richard  &  Janharmen  Drost.  Program  Progress  Report  2012  Uganda  Country  Health  and   HIV/AIDS  programme.  January  2013   - Proposed  action  IA-­‐  Uganda  Health  Program.  Global  Face-­‐to-­‐face  meeting  related  to  the  Basic   health,  SRHR  &  HIV  program  of  the  ICCO  Alliance.  24-­‐26  June  2013,  Soest,  the  Netherlands   - ICCO-­‐ALLIANCE  UGANDA  HEALTH  PROGRAM  LEARNING  AGENDA,  undated   - Invitation  for  UMOJA  training  workshop  for  the  Uganda  Health  Partner  Programme  -­‐  ICCO   Alliance  and  ICCO  Alliance  Uganda  Health  &  HIV  Cluster  AIDE  MEMOIR).     Literature  –  Vietnam     About  the  network:     - VIETNAM  RIVERS  NETWORK  (VRN).  VRN  STRATEGY  2008  -­‐2020.  Hanoi,  January  2009  (including   annexes)     - VRN’s  2011  Annual  Report.  Undated  (reporting  concerning  the  VRN  STRATEGY  2008  -­‐2020)     Concerning  ICCO  support:     - The  Center  for  Water  Resources  Conservation  and  Development  (WARECOD).  Final  proposal  to   ICCO  2011  –  2013.  Capacity  strengthening  of  the  Vietnam  Rivers  Network:  Advocacy  and   Research  on  rivers  and  sustainable  use  of  water  resources  in  Vietnam,  including  Annex  1  –  9.   Hanoi,  Vietnam.  January  2011       - Annex  1  Annex  1  VRN  member  list  update  in  January  2011  (in  Vietnamese)   - Annex  2  Annex  2  VRN  strategy  and  Action  Plan   - Annex  3    Funding  sources  for  VRN  action  plan  2011-­‐2013   - Annex  4  Vietnam's  Water  Resource  Challenges  from  Nancy   - Annex  5-­‐Community  based  rivers  monitoring  and  protection       - Annex  6  VRN  operation  regulation  –  EN   - VNR  Annex  7  ICCO  project  timeline  for  2011-­‐2013     - VNR  Annex  8  and  9  Estimated  budget  for  ICCO  project  2011-­‐2013   - Project  Plan.  VRN  Program  2011-­‐2014  Rivers  and  Watershed  Management  (76-­‐03-­‐02-­‐037).  27   May  2011.   -

Specific c ontract  terms  a nd  c onditions.  V RN  P rogram  2 011-­‐2014  R ivers  a nd  W atershed   Management  (76-­‐03-­‐02-­‐037)  

-

WARECOD. Narrative  Report  2012.  Capacity  strengthening  of  the  Vietnam  Rivers  Network:   Advocacy  and  Research  on  rivers  and  sustainable  use  of  water  resources  in  Vietnam.  28  Feb.   2013  (almost  the  same  report  as  WARECOD.  Capacity  strengthening  of  the  Vietnam  Rivers   Network:  Advocacy  and  Research  on  rivers  and  sustainable  use  of  water  resources  in  Vietnam   Annual  project  update  2012.  January  12th2013)         Le  Hien.  Feedback  on  progress  reporting  2012-­‐  Financial  report,  Narrative  report,  Auditor’s   report.  09-­‐07-­‐2013  

-

-

The Center  for  Water  Resources  Conservation  and  Development  (WARECOD).  Final  proposal  to   ICCO.  Promoting  River  Protection  through  Local  Action,  Linking  and  Advocacy  (LALA  for  Rivers)  -­‐   January  2014  to  December  2015,    including  annex.  Hanoi.  31  July  2013.     Project  p lan  –  C onsiderations  V RN  2 014-­‐2015  K eeping  P eople  S afe  from  W ater  R eservoirs.   23  A ugust  2 013   Project  Plan.  VRN  2014-­‐2015  Keeping  People  Safe  from  Water  Reservoirs  (76-­‐03-­‐02-­‐051).  24   October  2013  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

67


Other reports   - WARECOD  FINANCIAL  MANAGEMENT  DEVELOPMENTS-­‐  2012   - Strength  Civil  Society  Sector.  QUESTIONNAIRE  ON  THE  STRENGTH  OF  CIVIL  SOCIETY  –  WARECOD   RESPONSE.  January  2012   - Paul  McCarthy  &  Jude  Rand.    BASELINE  CAPACITY  ASSESSMENT  OF  CONSORTIUM  PARTNERS.  OXFAM   NEDERLAND  /  WWF  GREATER  MEKONG  PROGRAMME,  COMMUNITY  ENGAGEMENT  IN  GOVERNANCE  OF   MEKONG  AND  SALWEEN  RIVER  BASINS  .  August  23,  2013   - VRN  Overview  agenda  PA  -­‐    TOC  September    2013    and  a  few  powerpoints  from  Saskia  van   Drunen  (trainer)   Literature  –  Senegal   - Programme  Multi  Acteurs  2  second  phase,  juillet  2013   - AEA  –  document  projet  PMA  2,  août  2012   - AEA  –  rapport  technique  2012   - CERFLA  –  programme  d’appui,  août  2012   - CERLFA  –  note  filière  lait   - CERFLA  –  budget  opérationnel  an  1  –  PMA2   - CERFLA  –  rapport  annuel,  janvier  2013   - ONG  3D  –  Cadrage  de  la  préparation  de  la  seconde  phase  du  PMA,  2012   - ONG  3D  –  Projet  DRSP  An  2,  octobre  2013   - THP  –  rapport  annuel  2012   - THP  –  project  plan  2013-­‐2014   - USE  –  PMA  USE  2012-­‐2015   - USE  –  rapport  d’activités  2012  PMA  II  USE   - USE  –  cadre  logique  de  l’intervention  du  PMA2  USE     Annex  –  Time  schedule  and  people  met  during  field  visits     Uganda   Date    and  place     Activities  in  Uganda     20/11     workshop  -­‐  participants:   ACET  Office   Paul  Kabunga,  Director  ACET  (partner  van  Tear)   Makindye,   Sarah  C.  Nampindo  ,  Program  Manager  TAIP  (partner  van  Tear)   Kampala   Martin  Kizito,  Program  Officer  TAIP   James  Peter  Olupot,    Executive  Director  PAG  Kidep  (partner  van  Woord  en   Daad)   Richard  Ochen,  Program  Manager    HNU  (partner  van  ICCO)   Carol  ..,  Trans  World  Radio  Kenya  (partner  Trans  World  Radio  Nederland)   Samuel  M.  Ogutu,  General  Manager  UCAN  (partner  van  Woord  en  Daad)     Edward  Atenu,  Program  Officer  UCAN   Interview  with:   Richard  Ochen  -­‐  Programme  Manager,  HealthNeed  Uganda   21/11   Interviews  with:   IC  –  RO  office   Paul  Kabunga,  Director  ACET     Muyenga,   Samuel  M.  Ogutu,  General  Manager  UCAN     Kampala   (Interview  with  Director  of  UNERELA+  was  cancelled)   22/11   Interviews  with     hotel  in  Kampala   Taaka,  Janepher    -­‐  Programme  officer  programme  DCA   IC  –  RO  office   Janharmen  Drost  -­‐  Programme  Officer  Health  ICCO  Cooperation  Regional  Office   Muyenga   Central  and  Eastern  Africa   Annex  6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

68


23/11   Vietnam   Date    and  place     28/11,     WARECOD  office,   Hanoi          

29/11   Windy  Hotel,  23   Thong  Phong   lane,  Ton  Duc   Thang,  Ha  Noi  

30/11 morning     30/11     1/12  morning   1/12  afternoon    

2/12 morning   CSRD  office,  Hue  

2/12 afternoon,  

Data processing  and  report  writing   Departure  in  the  evening  

Activities in  Vietnam   Interviews  with:     Ms  Dao  Thi  Viet  Nga,  Director  of  WARECOD,  Coordinator  VRN  in  the  North     Ms  Duong  Thu  Hang,  project  officer  WARECOD,  fundraiser  for  VRN   Ms  Hoang  Thi  Tu  Oanh,  project  officer  WARECOD,    communication  officer  VRN   in  the  North   Mr.  Tien  si  Vu  Ngoc  Long,  Director  of  Southern  Institutive  of  Ecology,  Director  of   CBD,  Coordinator  VRN  in  the  South     Ms.  Nguy  Thi  Khanh,  Director  of  GreenID,    former    VRN  coordinator,    currently   advocacy  expert  of  VRN,     Mr.  Dang  Ngoc  Quang,  Director  of  RDSC,    member  of  advisory  board   Ms.  Truong  Anh  Thi,  researcher  at  CBD,  communication  officer  VRN  in  the   South     Dinner  with:   Ms.  Ms  Lâm  Thị  Thu  Sửu,  Director  CSRD  in  Central  Vietnam,    Coordinator  VRN   nationally     Workshop  -­‐  participants:     Ms  Lâm  Thị  Thu  Sửu,  VRN’s  Coordinator  nationally   Ms  Dao  Viet  Nga,  VRN  Coordinator  of  the  North     Mr  Tien  si  Vu  Ngoc  Long,  Coordinator  VRN  in  the  South   Ms  Ngụy  Thị  Khanh,  Advocacy  expert  VRN   Ms  Trương  Anh  Thơ,  communication  officer  VRN  in  the  South   Ms  Hoàng  Thị  Tú  Oanh,  communication  officer  VRN  in  the  North   Ms  Dương  Thu  Hằng,  fund  raiser  for  VRN   Dao  Trong  Tu,  Hydropower  expert,  director  of  CEWARED  member  of  advisory   board   Dao  Trong  Hung,  human  ecologist,  member  of  advisory  board   Mr  Dan  Tiep  Phuc,  law  expert,  member  of  advisory  board   Mr  Nguyen  Tien  Long,  energy  expert  CEWARED,  member  of  advisory  board   Mr  Dang  Ngoc  Quang,  member  of  advisory  board   Ms  Lương  Thị  Trường,  Human  rights  expert  CSDMA,  member  of  advisory  board   Interview  with  Mr.  Le  Anh  Tuan,  Vice  director  Dragon  Institute  –  Can  Tho   University  in  the  Mekong  region,    member  of  advisory  board   Data  processing  and  report  writing       Travel  to  Hue     Report  writing   Interview  with  Mr  Le  Hien  –  Programme  Officer  Vietnam    ICCO  -­‐  South  East  Asia   and  Pacific  Regional  Office   Report  writing   Interviews  with:   Ms.  Ms  Lâm  Thị  Thu  Sửu,  Director  CSRD  in  Central  Vietnam,    Coordinator  VRN   nationally     Mr  Tran  Ba  Quoc  ,  project  officer  CSRD,  supporting  VRN   Ms  Pham  Thi  Dieu  My,  project  officer  CSRD  ,  supporting  VRN   Ms  Tran  Chi  Thoi,  project  officer  CSRD.  supporting  VRN   Financial  arrangements  with  CSRD  related  to  assignment   Interviews  with:  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

69


coffee shop  Hue    

2/12 evening   3/12   Senegal   Date  and  place     17/11/2013   18/11,    office   ONG3D,  Dakar          

19/11 morning   19/11  afternoon   20/11  morning   20/11  afternoon   21/11  morning   21/11  afternoon   22/11  morning   21/11  afternoon   21/11  night      

Mr Nguyen  Van  Que,  Union  of  Science  and  Technology,  Social  Impact   Assessment,  member  of  VRN       Ms  Le  Thi  Nhu  Nguyen,  Hue  University,  Social  Impact  Assessment,  member  of   VRN       Mr  Pham  Mau  Tai,  Director  local  NGO  Quang  Bing  province,  member  of  VRN       Travel  back  to  Hanoi   Departure  from  Vietnam  

Activities in  Senegal   Travel  Amsterdam  -­‐  Dakar   Meeting  with  Comité  de  Pilotage  of  the  PMA  coalition.  Present:   Mme.  Cissé,  director  of  THP,  president  of  the  CdP   Mr.  Diallo,  director  of  AeA   Mme.  Loune,  secrétaire  exécutive  CERFLA   Mr.  Cissé,  coordinateur  ONG3D   Mr.  Diop,  director  of  USE   Mme.  Diallo,  consultante  ICCO   Mr.  Moctar,  coordinateur  PMA   Meeting  with  Aide  et  Action  director  and  staff   Meeting  with  The  Hunger  Project  director  and  staff   Meeting  with  CERFLA  director  and  staff   Meeting  with  USE  Project  director  and  staff   Meeting  with  ONG3D  director  and  staff   Meeting  with  PMA2  Coordinator,  preparation  debriefing   Debriefing  and  dialogue  with  PMA  Comité  de  Pilotage   Meeting  with  PMA2  Coordinator,  finalisation  of  the  mission   Travel  Dakar  -­‐  Amsterdam  

Annex 6  –  Report  of  the  third  (field)  phase  of  the  Evaluation  Programmatic  Approach    

70

Profile for ICCO

Evaluation ICCO Alliance Programmatical Approach  

Final report annexes

Evaluation ICCO Alliance Programmatical Approach  

Final report annexes

Profile for icco
Advertisement