14 minute read
KPI 11: Results are delivered efficiently
KPI 11: Results are delivered efficiently
Unsatisfactory KPI score
Advertisement
2.50
A total of 36 evaluation documents were reviewed to assess the ILO cost-efficiency of delivery. These included 12 independent evaluations and a mix of clustered evaluations, synthesis evaluations and meta-analysis. The findings were validated with the Annual Evaluation Reports and the ILO Programme Implementation reports for the 2017-20 assessment period, also reviewed among the total number of documents. In relation to the ILO’s efficiency, a sample of 31 evaluation documents from the 2017-20 assessment period was reviewed. These findings were validated against the Annual Evaluation Reports and the ILO Programme Implementation reports for the 2017-20 assessment period, among other documentary evidence.
The ILO delivers results cost-effectively, but efficiency in terms of delivering on time is an area for improvement, according to the ILO’s performance information. Resources are used efficiently and cost-effectively, yet there are resource constraints at the project level, according to the evaluation reports. The Annual Evaluation Report 2019-20 summarises, “Whereas over one third of projects faced resource constraints, resources were used strategically and effectively”. This is confirmed by financial reporting information, which the assessment team has used to supplement the analysis based on the understanding that evaluation reports (as well as interviews) may focus on resource constraints and therefore not be sufficient to assess the ILO’s cost-efficiency of delivery.
To assess cost-efficiency, a selection of four reports on ILO finances and funding modalities were included in the assessment. Financial reporting information shows that the ILO is cost-effective overall. It also shows that there are linkages between financing and delivery. For instance, the “Financial report and audited consolidated financial statements for the year ended 31 December 2019” demonstrated that funding increases for 2018-19 were accompanied by a higher delivery rate due to improved planning and delivery.
In this context of resource constraints, the ILO leverages synergies to maximise the achievement of common goals, according to the evaluation reports. The ILO collaborated with other UN agencies, making efforts to find efficiency gains, as evidenced in the evaluation reports, strategy documents and interviews. In the Development Cooperation Strategy 202025, for example, the ILO aims to make more use of large integrated programmes that can pool funding. Efficiency gains are also found in collaboration with the private sector, as they are readily accepting of the ILO’s conditions, as seen in the “High-level evaluations of strategies and Decent Work Country Programmes 2019”.
Regarding the ILO’s efficiency, the evaluation reports contain instances of delayed implementation, mostly at the start of a project. The internal reform processes have led to more efficient implementation of operations since 2013. However, performance on “efficiency of operations” is rated as “average” in the latest two Annual Evaluation Reports. There are examples in independent evaluations that support this finding, with instances of delays in project start in many cases due to local adaptation and consensus building among constituents. Interviews confirmed the existence of delays, indicating that there have also been issues with timely recruitment of staff for interventions – a limitation that is currently being addressed so that technical specialists are in place for the start of a project.
In countries with limited ILO presence, access to resources is a constraint, efficiency can be a challenge, and there are missed opportunities for social dialogue, according to the evaluation reports. Furthermore, the reports of high-level and independent evaluations suggested that the ILO is sometimes “over ambitious” about what it can achieve through its projects. Interviews confirmed these findings and noted that the ILO needs to be realistic about what is achievable within timeframes, particularly in the context of the tripartite structure.
MI 11.1: Interventions/activities assessed as resource-/cost-efficient Score
MI rating
MI score Satisfactory
3
4. Highly satisfactory: Interventions are designed to include activities and inputs that produce outputs in the most cost/resource efficient manner available at the time.
3. Satisfactory: Results delivered when compared to the cost of activities and inputs are appropriate even when the programme design process did not directly consider alternative delivery methods and associated costs.
2. Unsatisfactory: Interventions have no credible, reliable information on the costs of activities and inputs and therefore no data are available on cost/resource efficiency.
1. Highly unsatisfactory: Credible information is provided which indicates that interventions are not cost/resource efficient.
MI 11.1 Analysis
A total of 36 evaluation documents were reviewed to assess the ILO cost-efficiency of delivery. The majority of these pointed to resource constraints but the efficient use of limited resources. While evaluation reports highlighted project “resource constraints”, the ILO uses resources strategically and effectively, according to the Annual Evaluation Report 2019-20. The four financial reports reviewed suggested that there were resource constraints in projects but that the ILO used available resources efficiently. The ILO’s cost-efficiency of delivery was evident across all the evaluation reports and performance information on interventions found in the Decent Work Results and Development Cooperation Dashboards, as well as the Annual Evaluation Reports.
The cost-effectiveness of the ILO was confirmed by financial reporting information used to supplement the analysis. Financial performance information demonstrated that ILO operations are resource efficient. For instance, the “Financial report and audited consolidated financial statements for the year ended 31 December 2019” showed that funding increases for the period 2018-19 were accompanied by a higher delivery rate due to improved planning and delivery. Individual funding modalities, such as the RBTC, are cost-effective in financial reporting. Resources are seen to be used effectively and efficiently at the field level, according to interviews. Interviews also indicated the cost savings seen as a result of the business process review, such as increasing automation, energy efficiency and moving staff to front-office roles (see also MI 3.2 and MI 4.3).
The ILO’s annual performance reporting shows that it is leveraging synergies in the context of resource constraints to maximise the achievement of common goals. Interviews confirmed this finding from the evaluation reports. The organisation also aims to find efficiency gains through partnerships with other UN agencies. For example, one of the lessons from the “Independent evaluation of the ILO’s strategy and actions for creating and extending social protection floors 20122017” is that the ILO thought leadership is an “effective way to leverage the ILO limited resources”. The Better Work Programme, in which the ILO works with governments, global brands, factories and social partners to make improvements in working conditions in the garment sector, is also an example of the ILO leveraging resources. Interviews indicated that the Better Work Programme is made self-sustainable by charging fees to companies for the ILO’s work, and most costs are met in this way. The staff interviewed also noted that the ILO has leveraged funding in research activities, for instance, with private foundations around the SDGs.
There were examples in the evaluation reports of the donor context, in terms of donor requirements and the stability of donor funding, having an impact on efficiency. For instance, in the elimination of child labour in Latin America, steady donor funding helps ensure the longevity of the intervention. The risk of shortfalls or closing of funding from donors has a negative impact on efficiency, as seen in the evaluations on ILO capacity development, and in the “Independent evaluation of the ILO’s strategy and actions for creating and extending social protection floors 2012-2017”, the subject of a high-level evaluation. 12, 18, 26, 39, 68, 75, 135, 156, 189, 194-195
Source documents
The ILO carefully applies limited resources and innovates where necessary, according to the reports. This was seen across a number of evaluation reports, for instance in the “Independent evaluation of the ILO’s strategy and actions for creating and extending social protection floors 2012-2017” and in the “High-level evaluations of strategies and Decent Work Country Programmes 2019”. The evaluation reports highlighted that ILO resources reach further through capacity building, technical assistance, policy advice and knowledge sharing. The ILO is expanding innovative publicprivate partnerships and financing (links to time-efficiency in MI 11.2). According to the interviews, innovative financing is sought, such as through social impact investing.
The SIDA-ILO Partnership Programme (SIPP) is an example in the evaluation reports of the ILO leveraging synergies in the context of resource constraints. According to the “Final Independent Clustered Evaluation of Policy Outcome 8: Protecting workers from unacceptable forms of work and Cross-cutting policy driver: Gender equality and non-discrimination (SIDA-ILO Partnership Programme – Phase I)”, the resources allocated for each Country Programme Outcome “was no more than USD230,000. To address this financial restraint, the SIPP 2018-19 formed various synergies with existing ILO funding and interventions in order to leverage multiple resources and elevate the collective impact to a new level, particularly in the implementation of Global Products.” Similarly, under the Taqeem Programme to improve gender outcomes in the rural economy in the Middle East and North Africa, the ILO leveraged funding to over double the original amount through collaboration with the International Fund for Agricultural Development, according to the reports.
The evaluation reports presented numerous examples of the ILO leveraging resources in a variety of different ways, and in all cases, synergies are created to make resources stretch further. For instance, in Ethiopia, the ILO worked closely with government institutions and other implementing partners that brought human and technical resources in the formulation and roll-out of the country’s industrialisation strategy. This made optimal use of the available resources of the ILO. The ILO recognises the need for “longer-term, large-scale and integrated programmes” that will help generate sustainable impact. The organisation encourages un-earmarked funding, which allows for greater agility and flexibility. In the Development Cooperation Strategy 2020-25, the ILO stated that it aims to make more use of large integrated programmes that can pool funding for increasing impact and generate economies of scale. This has been a successful strategy in Bangladesh, Ethiopia and Viet Nam, as noted in the Development Cooperation Strategy 2020-25. 12, 18, 26, 39, 68, 75, 135, 156, 189, 194-195
MI 11.1 Evidence confidence
MI 11.2: Implementation and results assessed as having been achieved on time (given the context, in the case of humanitarian programming)
MI rating
MI score
4. Highly satisfactory: All or nearly all the objectives of interventions are achieved on time or, in the case of humanitarian programming, a legitimate explanation exists for delays in achieving some outputs/outcomes.
3. Satisfactory: More than half of the intended objectives of interventions are achieved on time, and this level is appropriate to the context that existed during implementation, particularly for humanitarian interventions.
2. Unsatisfactory: Less than half of the intended objectives are achieved on time but interventions have been adjusted to take account of the difficulties encountered and can be expected to improve the pace of achievement in the future. In the case of humanitarian programming, a legitimate explanation exists for delays.
High confidence
Score
Unsatisfactory
2
1. Highly unsatisfactory: Less than half of stated objectives of interventions are achieved on time, and no credible plan or legitimate explanation is identified that would suggest significant improvement in achieving objectives on time.
MI 11.2 Analysis
A sample of 31 evaluation documents from the 2017-20 period was reviewed to assess the ILO’s efficiency. These included 12 independent evaluations and meta-analyses. The sample also comprised the “High-level evaluations of strategies and Decent Work Country Programmes” from 2017, 2018, 2019 and 2020, the Annual Evaluation Reports, and ILO Programme Implementation reports from the assessment period.
The Annual Evaluation Reports showed “average performance” on the implementation and efficiency of operations, with steady improvement since 2013, although “recurrent weaknesses” were noted in terms of “efficiency of management”. The Annual Evaluation Report 2019-20 listed the strongest results in the ILO’s capacity to manage and support project implementation, establish internal synergies, and disseminate knowledge for organisational learning. This report also demonstrated improvement against the previous annual evaluation report where this area was rated “low to average”, with project management being a particular area for improvement. The reports of high-level and independent evaluations suggest that the ILO is sometimes “over-ambitious” about what it can achieve through its projects, as stated, for example, in the “Independent High-level Evaluation: ILO’s Strategy and Actions towards the Formalization of the Informal Economy, 2014-18”. Interviews confirmed that in the context of the tripartite structure, the ILO needs to be realistic about what is achievable within the given time frames.
Moreover, the “High-level evaluations of strategies and Decent Work Country Programmes 2020” stated that “[m]easuring the long-term impact of the ILO’s enterprise work remains a weakness, resulting in restrictive assessments of efficiency” in the ILO’s work in promoting sustainable enterprises; “[s]ome obstacles to better efficiency are the limited collaboration and teamwork within the Office and the reported limited staff available to inform and influence/advise target audiences”; and “inconsistent methodologies, know-how and quality of RS&KM [research and knowledge management] across the Office contribute to reduced relevance as well as inefficiencies” in the ILO’s RS&KM strategies and approaches. In the case of the “ILO’s Decent Work Programme in the Andean countries of the Plurinational State of Bolivia, Colombia, Ecuador, Peru and the Bolivarian Republic of Venezuela”, it is noted that “[t]he project-based structure of ILO project offices created inefficiencies” and “the rigidity of the programming and the administrative delays created challenges that affected the implementation of projects. This ultimately impinged on the effectiveness, efficiency, impact and sustainability of the results.”
Evaluation reports also evidenced instances of delays in project start. Interviews confirmed these delays, which are caused by lengthy recruitment processes. According to interviews, in order to address this issue, the ILO now advertises for staff prior to commencement, with the caveat that the position is dependent on donor funding. However, the recruitment process is still lengthy (around five months), according to staff interviewed. As also noted in KPI 5, interviews indicated that considering the time needed to train the staff recruited, it still may take six to eight months to fill a vacancy in programmes. The ILO staff interviewed also indicated that approving a company’s involvement in a flagship programme can take from three months to one year, which is a long period of time that can negatively affect the overall programme efficiency. Staff also noted that in some cases delays in the starting of project implementation can be caused by changing policy contexts at the country level (such as a change in government), as well as a commitment to standardisation instead of contextual adaptation. 10, 24, 26, 66, 135, 189, 192, 194-195
Source documents
The ILO’s response and adaptation to the operating environment of the country were noted in the evaluation reports as a cause of slower implementation. For instance, while evaluations on the ILO Syria response noted that the organisation was “initially slow to respond”, this was because the ILO prepared the groundwork in the context to build political buy-in. It is also stated that “the reason for slow take off was due to a variety of factors relating to the absence of clear contextualized guidance”. According to the “High-level evaluations of strategies and Decent Work Country Programmes 2018”, the organisation’s response in Syria led to a “UN-wide response within the jobs and livelihoods sector”. The ILO has implemented lessons learned from the intervention for its response to other labour crises. A mid-term evaluation of forced labour interventions in 2018 showed that the ILO made use of meaningful synergies, for instance in Mauritania, Nepal and Peru, by forging strategic and long-term partnerships in the context of instability and higher levels of change of government staff in all three countries.
In countries with limited or no ILO presence, and where access to resources is a constraint, time efficiency can be a challenge, according to some of the evaluation reports reviewed. The evaluation on ILO interventions in social protection floors summarised this common issue by stating that “too much ILO staff time is being devoted on the ground to fundraising” and that the ILO “miss[es] windows of opportunity during high-level exchanges with policy-makers and in donor forums in cases where there are no senior ILO staff based in the country concerned”.
The evaluation reports showed that the ILO collaborates with the private sector to find efficiency gains. This was evident in the “High-level evaluations of strategies and Decent Work Country Programmes 2019” and the “Independent evaluation of ILO’s public-private partnerships, 20082018”. Also, staff interviewed for this assessment noted that the private sector’s general acceptance of the ILO’s rules and values facilitates the efficient implementation of projects with companies. In some cases, implementation with companies is seen as more efficient than with the public sector, as noted in the “Independent evaluation of ILO’s public-private partnerships, 2008-2018”. However, the evaluations also stated that the due diligence process has sometimes created “avoidable delays”. In the “High-level evaluations of strategies and Decent Work Country Programmes 2019”, efficiency as one of the evaluation criteria in private-public partnerships was rated as “somewhat unsatisfactory”.
The ILO’s own reports depict the Office as quick and efficient in its response to COVID-19 as a global thought leader and convenor. The ILO swiftly mobilised constituents globally around the Centenary Declaration, as evidenced in the “Report of the Director-General Sixth Supplementary Report: The response of the International Labour Office to the COVID-19 pandemic”. Its role as convenor enabled continuity through virtual means. The ILO’s activities in capacity development and training also adapted well virtually. Many programmes that had knowledge platforms, such as in South-South Technical Cooperation, were able to continue with minimal disruption, as seen in the latest evaluation reports. There were, for instance, regional social dialogue constraints in the Andean region, which occurred due to other labour market issues of more immediate priority. Interviews indicated that in other areas there were some project delays due to COVID-19, but those started to be overcome in the second half of 2020. 10, 24, 26, 66, 135, 189, 192, 194-195
