Final Management Response – Women’s Voice and Leadership – Pakistan

Management Response Template This template is included as a recommended template in the CAMSA. August 2014 Please consult the Oxfam Evaluation Guidance1 when planning an evaluation. This template should be filled out when closing an evaluation.

What’s this template?

This template summarizes the reflections of your team and program/project stakeholders in relation to the evaluation’s findings and recommendations and offers an opportunity to comment on the utility of the evaluation process and final report. The template is divided into two parts: The first part should be published together with the evaluation report (or its executive summary); the second part is for internal use only as it helps you define and track a detailed action plan in response to the evaluation findings.

Why should this template be filled out?

Oxfam Policy on Program Evaluation All evaluation reports must be accompanied by a management response that communicates careful consideration of the evaluation’s findings and recommendations, detailed actions that will be taken to respond to these findings, and offers an opportunity to comment on the utility of the evaluation process and final report.

(Page 4: Paragraph 11)

The overall purpose of a management response (MR) is to ensure that findings, conclusions and recommendations from Oxfam evaluations are given careful consideration and are acted on. Developing a management response in consultation with the relevant stakeholders helps us document our main learnings from evaluations as well as track our actions in response to the recommendations.

Who should fill it out?

The evaluation manager is responsible for ensuring that a management response is prepared within a reasonably brief period of time after the finalization of the evaluation report or other products (see Oxfam Policy on Program Evaluation). The team that is responsible for the program implementation should participate in the development of the response. If the evaluated project or program has been implemented together with partners and/or communities, these stakeholders should, in a way appropriate to the evaluation context, also participate in the discussion of findings and the development of a response (specifically regarding recommendations that are addressed to them).

When should this tool be filled out?

The management response should be filled out when you discuss the findings and recommendation of an evaluation with relevant stakeholders, i.e. at the end of an evaluation. Before responding to the evaluation findings and recommendations, you should assess the quality of the evaluation and the validity of its findings and recommendations (see Evaluation Quality Assessment Tool in CAMSA). Where you choose to not act upon findings and recommendations, because they are not clearly justified by the evidence presented in the evaluation report/products, this should be clearly communicated. However, together with the relevant stakeholders you should discuss also those recommendations on which Oxfam may not be able to act directly (e.g. when partners are addressed in recommendations) and agree on options to address these.

1

Also available in the relevant sections in CAMSA.

1

Final Evaluation /2024/WVL-P The preparation of the management response, review by senior management and posting on SUMUS (as well as submission for posting on the Oxfam public website) should happen within a reasonably short timeframe from the moment the evaluation is submitted to the evaluation manager.

What to do with the tool once completed?

Internally, the MR should be used to track actions as agreed. After an evaluation is completed the MR should be posted on SUMUS and the Oxfam website together with the evaluation product itself (i.e. the evaluation report) and the evaluation quality assessment tool. Please notify the OI secretariat by email (meal@oxfaminternational.org) where the documents are posted.

CAMSA minimum requirements regarding the management response 1.

The commissioning manager must issue a management response to the review or evaluation. The team that is responsible for the program implementation can participate in the development of the response. It should include how Oxfam plans to use the findings and recommendations to review program planning and strategy.

2.

To ensure transparency to Oxfam’s constituents, Oxfam will routinely place the executive summary and management response for all evaluations of Oxfam programs or projects on www.oxfam.org and/or the affiliate’s website. Barring unacceptable risk or repercussions to staff, partners or program efforts, the complete reports from all final program or project evaluations will be posted on the website.

Oxfam Management response to the evaluation of enter title of the evaluation This information is for internal use only and should not be published.

Prepared by: enter name of person preparing the management response Noreen Khalid, Uzma Zarrin, Roxanne Murrell Contributors: include the names of colleagues and partners that contributed actions to the response See point B4, below.

Signed off by: enter person who signed off the draft response on behalf of management Syed Shahnawaz Ali, Country Director, Oxfam in Pakistan René Guerra Salazar, Manager, Women’s Economic Empowerment and Transformative Leadership Date: May 22, 2024 Country/Region/Campaign: Pakistan /Asia

Please remember that this part of the management response should be written in an accessible way for external audiences!

A: Context, background and findings

2

Final Evaluation /2024/WVL-P 1.

The context and background of the evaluation, i.e. the purpose and scope of the evaluation.

Women’s Voice and Leadership – Pakistan (WVL-Pakistan) is a five-year, $8 million project funded by Global Affairs Canada (GAC) and implemented by Oxfam Canada (OCA), in collaboration with Oxfam in Pakistan (OiP). The project is part of GAC’s Women’s Voice and Leadership (WVL) Program, a global initiative to support the capacity and activities of local and regional women’s organizations and movements seeking to empower women and girls, advance the protection of women’s and girls’ rights, and achieve gender equality.

The project works with 12 Women’s Rights Organizations (WROs), 4 provincial alliances, and 1 national alliance, and fast responsive short grantees which are very diverse in geographical and thematic areas, as well as capacity in terms of programming and resources. The project partners are located across all provinces in Pakistan – Khyber Pakhtunkhwa (KPK), Balochistan, Punjab, Sindh, and Islamabad.

The project was designed and implemented under a shrinking space for civil society and rights-based work within Pakistan, with groups and organisations working towards women’s rights facing restrictive regulatory frameworks, limited space for organizing and limited access to funding for gender transformative changes and programming.

The overall objective of the Final Evaluation was to assess the progress towards ultimate, intermediate, and immediate outcome indicators from inception (2019) to the present. Building on the baseline and Mid-Term Review (MTR) findings, the evaluation focused on the relevance, coherence, effectiveness, efficiency, sustainability, and impact of the project. It compared project indicators between baseline and End Line stages, capturing unexpected results, changes, partnerships, successes, challenges, and lessons learned.

The sub-objectives for the End-line evaluation were as follows: • Assess the project’s progress towards outcomes (ultimate, intermediate, and immediate, including the indicators), particularly the capacity strengthening of WROs and enhancing women’s leadership capacities, the changes in WROs’ technical and programmatic capacities, and effectiveness of the collaboration between WROs and alliance to effect gender sensitive policy change and implementation in Pakistan, based on these indicators identify key lessons learned and best practices for our partners, beneficiaries, donors, and other stakeholders.

• Identify the key enabling and impeding factors/ conditions for success. This should include factors of a national scope as well, including the COVID-19 pandemic, climate-change induced events such as floods, political-economic instability, and other relevant legislative/policy changes.

• Test the project assumptions in its relation to the Theory of Change. In addition to that identify the unintended outcomes of the project.

• Review and assess the process of change in each project pillar and attached intended results in relation to the prescribed Theory of Change for WVL project. To assess project and partner approaches to sustainability and document partners level of effort and input to sustain minimum actions generated through the project at organizational level. (provide recommendation for partners’ exit strategy. Also, provide report summery, indicators conclusions and recommendations to guide future programming in Pakistan on women’s right and gender equality, including supporting WROs in becoming gender-just organizations)

3

Final Evaluation /2024/WVL-P

2.

Summary main findings and recommendations

MAIN FINDINGS Overall, the WVL-P project was found to be relevant, coherent, effective, efficient, sustainable, and creating an impact in the landscape of women and girls’ rights in the target communities and areas in Pakistan. Out of all the indicators reported at the End Line, 19 of 24 have met or exceeded targets (this includes 22 indicators listed in the End Line TORs, as well as the two ultimate outcome indicators). This holds true for the ultimate outcome and under all three Pillars.

Impact Findings: WVL-P’s impact can be seen across all its Pillars, affecting women’s transformative leadership, enhancing the WROs’ programmatic capacity for gender-just programs, and in building networks and linkages between WROs and alliances. The following is a summary of the main impacts of the WVL-P project: i) Assessment done using Oxfam’s Capacity Assessment Tool for Gender Just Organizations (CAT4GJO) indicates that all WROs have improved across all domains in terms of gender-just organizational strengthening (there are six domains in total). All WROs have made new or improved their policies on inclusion, gender, and addressing workplace harassment. ii) All WROs’ management shared that their ability to manage and lead their organizations has improved to a good or excellent degree. iii) There has been a significant increase in reach in the number of community members/ target beneficiaries through the WROs’ programming as compared to before WVL-P project implementation. iv) Critical issues were addressed in the target areas through the long-term and short-term grantees, prioritizing small WROs and newly emerging dimensions such as COVID-19. v) Improved capacity of WROs by incorporating a feminist lens to how they carry out monitoring, evaluation, accountability, and learning (MEAL) for programming and prioritizing ‘nothing about us, without us’, and understanding power dynamics between the community and themselves. vi) Knowledge dissemination and sharing improved within academia and civil society through knowledge sharing events/ WVL-P project website. All research grantees shared that the project led to greater dissemination opportunities. vii) The project has made significant strides in influencing legislative and policy developments related to women's and girls' rights in Pakistan.

Sustainability Findings Overall, the effects of the project will continue beyond the end of the project period. Although no particular lens for assessing sustainability has been shared by GAC, during the End Line, Oxfam, WROs and alliances were asked to share what sustainability means to them, and to comment on

4

Final Evaluation /2024/WVL-P how they perceive this under the WVL-P project. The various stakeholders consulted including the Oxfam team and WROs shared the opinion that sustainability of the WVL-P project must be seen from various angles. This included looking at sustainability both from an individual and organizational perspective. Using this lens, the following are the key findings related to sustainability under the project: i)

At the individual level, the effects and impact of the project are likely to be sustained especially with reference to the technical capacities of individuals, such as skills developed in organizational capacity strengthening, applying a feminist lens to MEL and programming, gender leadership, participatory research methodologies, and gender responsive budgeting. The individual capacities will help individuals to share knowledge within their organizations, wherever they work, and also become resource persons and share these skills and knowledge forward. One example that is already visible in this regard is of an ED sharing her policy strengthening skills with other organisations as a consultant. Another example is a number of women taking on leadership roles/positions within their organizations after having completed the gender leadership program.

ii)

At the organizational level, WROs have shown a commitment to strategically align their programming and organizational plans to the feminist approaches. Under WVL-P, where discussion with staff at all levels of the organization during field visits reflects a deep commitment to a gender transformative approach and ownership of the project. The introduction of new policies and improvement of existing policies such as on safeguarding and HR and institutionalization of skills learn by individual staff in particular will sustain beyond the end of the project. The staff at WROs have become more open to the idea of having women lead the organization. The use of FMEAL approaches is become the part of the organization as well.

iii)

The networks and connections established between WROs and alliances will be sustained where the collaboration and willingness of the partners to engage under the FEM Consortia2 can already be seen as a positive step in this regard, aiming to connect WROs with alliances in the long-term. Strengthening the network through effective (friendly and accessible forums either online or in-person) and regular communication with alliances and WROs are considered crucial for long-term success.

WROs and alliances both shared that financial support is identified as crucial for sustaining some of the changes in management, boards, and organizational functioning and practices. Financial sustainability is identified as a major challenge for both WROs and alliances. With the end of the project, WROs may struggle to sustain their organizations with the same number of staff, depending on the funding portfolio of the WROs. Alliances operate on a volunteer basis and similarly face a shrinking funding landscape for women’s rights causes.

RECOMMENDATIONS 2

The FEM Consortia is a consortium of 12 women's rights organizations and five (5) alliances formed through the WVL-P Project. It embraces feminist principles to transform entities and sustain the impact of WVL-P. The overarching goal is to increase the enjoyment of the human rights of women and girls and advance gender equality in Pakistan.

5

Final Evaluation /2024/WVL-P

For Partners: • Sustain institutional memory and continue institutionalization through refreshers and trainings within the organization: At the moment, WROs have invested time and efforts to institutionalize the various practices by the WROs towards passing on knowledge and skills from the project.

This practice must be continued, with refreshers at the organizational level and the training of new staff on the relevant tools and techniques including the Gender Leadership Program, FMEAL, and Participatory Rural Appraisal.

• Prioritize practice of annual reflection using the CAT4GJO to formulate yearly working plans: Under WVL-P, WROs have made it a practice to conduct yearly CAT4GJO exercises which build into their capacity strengthening and annual working plans. This practice should be continued even after the end of the project, with staff already well-versed in administering the tool, and having the requisite templates and formats available with them for use. Additionally, the staff trained on CAT4GJO may leverage their skills as consultants for other organizations looking to assess organizational capacity from a gender-just lens.

• Generate discussion between staff and board on ways to further strengthen safeguarding policies and practices: all WROs reflected in their CAT4GJO that whilst they had improved their safeguarding policies, they wished to further improve these policies and practices of these policies within their organization and at the field level. A discussion on how to improve practices must include dialogue between the BoD, management, and staff at all levels of the hierarchy to capture the varying perspectives and experiences regarding safeguarding of all individuals in the WROs. • Conduct bi-annual, participatory review on the use of feminist principles and language within the organization: all organizations were intentionally including feminist principles in their workspace and trying to improve the working environment for women. Further improvements can be made on a bi-annual basis where female staff are surveyed and suggestions taken regarding needs and requirements to make the workspace friendlier and more conducive.

• Strengthen gender-just linkages and relationships with external WROs and government stakeholders: WROs and alliances should pass on their strengths by training and collaborating with grassroots level WROs who were not part of the project, through collaboration and support through trainings on the feminist approach, CAT4GJO, FMEAL tools, PRA, QuickBooks, and GLP. These external WROs may be supported to improve their gender-just programming ability. A need has been highlighted by WROs and alliances to engage more closely with government stakeholders as well.

For Oxfam: • Further support for financial sustainability of partner alliances: in a second phase or follow-up project in Pakistan or in similar contexts, greater focus is needed to ensure the financial sustainability of the alliances involved, perhaps through developing linkages with regional/ global advocacy funds to enable more regular funding opportunities for advocacy.

• Leverage innovative FMEAL strategies: further innovative FMEL strategies may be adopted for contexts such as that of Pakistan, where some tools such as photovoice and Cellphilm are challenging to implement in the field due to cultural norms.

• Greater engagement with government stakeholders and male allies: both alliances and WROs have reflected a need to work more closely with government stakeholders regarding the space available for advocacy and rights-based work and with male allies to ensure that the approach towards women’s rights is more holistic in a second phase of the project.

6

Final Evaluation /2024/WVL-P •

Scaling up CAT4GJO: the value of the CAT4GJO tool for small to mid-level WROs has been reflected under the project, where its use can be scaled up as a general way of working with partners elsewhere in Oxfam’s work. In the data validation workshop, an alliance member shared how the scaling up of the tool may also include customization for alliances to utilize the CAT4GJO as well.

B: Oxfam’s response to the validity and relevance of the evaluation findings, conclusions and recommendations. Include 3.

Summary of evaluation quality assessment, i.e. quality of the evaluation is strong/mixed/poor and short assessment of the process (e.g. good, wordy report)

The evaluation report was divided into two products: one for internal use only, which includes the full report and a PowerPoint presentation, and the other abridged version for external users such as donors and website dissemination. The public report was reviewed and approved by the security focal person in OiP and relevant team members. The report uses a feminist lens and reflects real-time success and areas of improvement.

Overall, the assessment of the report can be categorized as good considering the quality of learnings captured as well as a very inclusive and empowering process.

4.

Main Oxfam follow-up actions (detailed follow-up actions should be included in the table below)

Oxfam agrees with the recommendations. However, the project has ended, and the Oxfam office in Pakistan is closing. There will be no opportunities to implement the recommendations. Partners also agreed with the recommendations and will implement them to the best of their abilities.

5.

Any conclusions/recommendations Oxfam does not agree with or will not act upon - and why (this reflection should consider the results of the evaluation quality assessment)

Oxfam agreed with all the recommendations provided. However, as noted earlier, the Oxfam office in Pakistan is closing, which limits Oxfam’s ability to actively act on the recommendations.

6.

Additional reflections that have emerged from the evaluation process but were not the subject of the evaluation.

7

Final Evaluation /2024/WVL-P None Detailed Action Plan on Key recommendations – for internal use only A: Summary actionable evaluation recommendations – See response to Question 4.

Please list all recommendations that require specific actions as per the management response above.

B: Detailed action plan One action may address several recommendations. In this case list all recommendations that are addressed.

Signed off by: Syed Shahnawaz Ali | Country Director Oxfam in Pakistan

__________________________________________

René Guerra Salazar | Manager, Women’s Economic Empowerment and Transformative Leadership, Oxfam Canada

__________________________________________ Date: May 22, 2024

Country/Region/Campaign: Pakistan /Asia

8