SkYE Programme - Awarding a training grant to a qualifications body: the case of the SSDA/NQD

SkYE Talk: Issues Unpacked Awarding a training grant to a qualifications body: the case of the SSDA/NQD Issue 04

Introduction The Skills for Youth Employment (SkYE) Programme in the Caribbean aimed to develop market-relevant skills among young people and a more productive, better trained and inclusive workforce across four countries, Saint Lucia, Grenada, Dominica, and St. Vincent and the Grenadines (SVG). Through the SkYE Training Fund, the programme awarded grants to organisations delivering training leading to nationally, regionally or internationally recognized certificates. This case study examines a grant awarded to the Sector Skills Development Agency (SSDA) / National Qualifications Directorate (NQD)1 in SVG. It draws on interviews with staff at training institutions and the SSDA/NQD to consider benefits and challenges of awarding a training grant to a National Training Agency. It finds that alignment with the agency’s role in the eco-system improved the efficiency of the grant, but that results-based financing needs to be carefully designed to avoid undermining the quality of training.

The SSDA is the National Training Agency (NTA) for TVET in SVG. It consists of a Board and a secretariat, the National Qualification Department (NQD).

1

1

The SkYE Training Fund

The SkYE Training Fund was established to fund market relevant, certified training for young people (aged 16 to 30 years) from disadvantaged backgrounds or living with disabilities. Training institutions with appropriate capacity to deliver training for disadvantaged youth and those with disabilities were identified during an initial institutional mapping phase. In SVG, five public providers were identified to deliver inclusive training and student support services with funding from SkYE. The five training providers were Kingstown Technical Institute, Georgetown Technical Institute, Barrouallie Technical Institute, Campden Park Technical Institute, and the Division of Adult and Continuing Education. Expecting to award grants directly to each training provider, SkYE hit procedural snags and as a result, the SSDA was approached to act as the ‘lead agency’ for the grant. In effect, SkYE contracted the qualifications authority (SSDA/NQD) to supply certificated training services.

2

The SSDA Grant In 2020, SkYE awarded a three-year grant to the SSDA to deliver certificated training to 1120 young people from across SVG, from disadvantaged backgrounds and including young people with disabilities. Training courses could be either short (8 week) courses in specific skills leading to a unit certificate, or year-long programmes, that lead to the award of a full National Vocational Qualification (NVQ) or Caribbean Vocational Qualification (CVQ). All SkYE grant recipients were required to carry out graduate tracer studies 3 to 6 months after the end of training. The SSDA, Training Institutes and SkYE collaborated to co-create the project that would be delivered with the grant funding.

By the end of the SkYE project the SSDA had not only achieved an impressive certification rate of 84.7%2 it had also over-achieved on the grant certification targets by 8%. Of those graduates that responded to the tracer survey, 41% of SSDA graduates were employed or self employed within 3-6 months of completing their training. All certificated graduates received their certificates within three months of their final assessments3. From the fund manager’s perspective, the grant to the SSDA was both efficient and effective.

To learn from this experience, and inform future programming, the SkYE Lead Adviser: Capacity Development interviewed SSDA/NQD staff, and managers and instructors at the Training Institutes, to identify factors contributing to the grant’s success, but also highlight challenges that this arrangement presented to the SSDA/NQD.

1120

Young people from across SVG

84.7% Certification rate

41%

Employed/self employed in 3-6 months

2 84.7% of trainees enrolled on training, completed and achieved a recognised certificate.

3 Tracer surveys responses from SSDA graduates suggest an employment rate within 3 months of 41%.

We note the size of sample responding to this question/tracer was only 10% of all graduates.

3

Factor 1:

Alignment with roles in the skills ecosystem in SVG By awarding the grant to SSDA, the SkYE Training Fund aligned with institutional logics within the SVG skills ecosystem. This alignment built on existing responsibilities and organisational practices, enabling greater efficiency.

Legitimacy, respect and goodwill The SSDA’s position as quality assurance and certifying authority for courses offered at the Training Institutes was seen by all training providers as contributing to the success of the project. The institutional authority, and the position of SSDA staff who are senior education officers by appointment, and therefore accorded a measure of propriety within the education system, enabled the SSDA’s leadership. However, training providers also attributed success to the professionalism of the technical personnel of the agency, the high regard and goodwill between all entities as a major success factor.

Prior experience of managing donor-funded programmes in partnership The SSDA and Training Institutes have prior experience of jointly running training programmes with external funding, for example they collaborated on the implementation of the UK aid-funded ‘Pre-technology’ programme which was a pre-cursor to SkYE. In addition, SSDA and Training Institutes have partnered in the Caribbean Development Bank (CDB) funded ‘Access to Another Nexus in Education and Workforce Development’ (ANEW) programme, and the World Bank funded Youth and Adults Training for Employment (YATE) . These experiences have strengthened working relationships and fostered synergies between the organisations.

Clarity on roles and shared priorities The roles of each partner in the SkYE-funded Grant Project aligned with existing responsibilities and practices in the system, and modifications for the purposes of the grant were agreed during the co-creation process.

Training Institutes were responsible for delivering training. They had flexibility to schedule training during regular hours, after hours, or in combination, and recruited trainees using outreach materials, media platforms, and running targeted information

4

sessions for ‘hard-to-reach’ groups. They also had freedom to offer places to past (eligible) graduates to take more advanced skills training (Level 2) or to complete full CVQs where only units were completed before. Training Institutes were responsible for selecting trainees and entering learning contracts with them. Already very familiar with the assessment and validation requirements of the SSDA/NQD, Training Institutes managed teaching, learning and assessment, including portfolio preparation. They arranged and supported the short internships that formed part of the training. Training Institutes planned the procurement of teaching resources and consumables, and submitted procurement requests to the SSDA. They could also form delivery partnerships with training providers and/or businesses with expertise in particular skills, for example Cosmetology. In these partnerships, the Training Institutes retained responsibility for internal quality assurance.

Within the SkYE Grant Project, the SSDA played two key roles. On the one hand, as the national qualifications agency, SSDA/NQD carried out the external quality assurance of all training delivered, external verification of assessments, and certification. On the other hand, as the grant recipient, the SSDA managed the relationship with the SkYE Training Fund, including all reporting. Responsible for grant finances, the SSDA paid invoices for (freelance) instructors and verifiers, and also paid invoices for student support services (such as stipends, transport). The SSDA also procured teaching resources and consumables, as requested by the Training Institutes. Finally, the SSDA also took on the responsibility of implementing the graduate tracer surveys.

5

Factor 2

Leveraging existing efficiencies within the skills eco-system Respondents to the case study interviews also attributed the grant project’s success to the efficiency of the certification process used by the SSDA. This had two very important benefits: it reduced the rate of student drop out seen between training and final assessment, and it ensured grant payments (which were linked to milestone achievements) were paid more promptly. Prompt payments were vital to ensure cash flow to deliver more SkYE training.

Quality assurance framework The assessment and quality assurance framework adopted by apex bodies determines the efficiency of certification. Two established QA frameworks for assessment are used in the Eastern Caribbean. One uses ‘instructor-assessors’ to assess and determine learners' competencies. The other relies on independent assessors who are not involved in training delivery, but are contracted to plan, conduct, and review assessment activities4. The SSDA uses the instructor-assessor model.

Assessment is carried out by instructors, with internal verification carried out by Training Institutions to ensure that standards are maintained and improved. External

4 These models are examined in detail by Orlando Hewitt and Paul Creese in their 2022 ‘Review of Quality Assurance of TVET’ which is available at www.skyecaribbean.com

6

verification of assessment is still carried out by the Awarding Body (SSDA) but by keeping assessment tied to provision, the approach is demonstrably more cost effective, efficient, and sustainable. For the SSDA grant, the average time taken between trainees’ final assessment and certification was 2.5 months which is generally within the ‘certification release’ ballpark advised by the Caribbean Association of National Training Agencies (CANTA).

Whilst the efficiencies within the quality assurance model greatly benefited the SkYE grant, the scale of the programme made significant demands on the system. More than 1000 additional students were injected into the certification system (including through SkYE grants awarded to CVQ providers in SVG that were not part of the SSDA grant), most of whom completed their training with a two-year post-pandemic window, creating substantial workload for the small team at the SSDA/NQD. The SSDA/NQD technical staff were commended by respondents to the case study interviews for their tenacity in conducting and coordinating the process leading to final certification.

Co-financing and efficiencies All SkYE grant projects were co-funded by grant partners. In the case of the grant to SSDA, some of the co-financing was provided by the Government of SVG through covering the operational and overhead costs of the public training facilities. As we will discuss below, the government co-financing proved essential for continuity of training activities in the context of results-based grant payments.

7

Challenges experienced with the SSDA Grant Whilst the award of the grant to SSDA worked well, especially from a Training Fund management perspective, lessons for future programming have been learned.

Demands on organisational capacity The SSDA experienced considerable demands on the organisation’s human capacity from the high volume of quality assurance of courses and certification of graduates generated by the SkYE grant, as well as the demands made by the Training Fund Manager in terms of the management of the grant itself. SkYE was also operating in a sector with multiple ongoing donor projects, as well as ‘business as usual’ TVET provision, and Prior Learning Assessment and Recognition (PLAR) processes. The core staff of the SSDA/NQD worked with a high level of commitment to the programme objectives, and good will. However, the well-being of staff and risks to organisational sustainability of huge increases in operational demands must be of high priority.

The demands placed on the SSDA by SkYE far outweighed the SSDA’s absorptive capacity. SkYE funded a post to assist with the increased administration, but this was only for one year of the programme. Future programming would need to consider whether significant, short-term increases in provision are immediately sustainable across all parts of the skills system. Short term embedded human resource capacity was needed to facilitate short-term investments of funds. The sustainability of development partners’ support to organisations needs to be viewed within much longer-term view of sustainable financing – including financing that can act strategically and ‘inject’ extra resources into the system as required.

Tracer Surveys At the outset, the SSDA took responsibility for implementing graduate tracer studies.

Although awarding bodies can – and do - commission tracer studies to monitor the performance of the skills system, such studies do have specific, contextually related challenges, especially in contexts where graduates change their contact details often.

The SSDA/NQD faced considerable challenges tracing graduates, with graduate contact details quickly outdated, and limited access to the internet making email unreliable. In the last year of the grant, SSDA delegated tracer studies to the Training Institutes who distributed questionnaires to graduates in their final training session and asked them to complete them and send them back after three months. The most successful tracer methods were telephone interviews (where contact numbers were

8

still valid) and certificate presentation ceremonies (where these were organized more than 3 months after the end of training). Even so, together the SSDA/NQD and their Training Institutes surveyed about 41% of their graduates, which remains an impressive sample.

The SKYE grant requirements highlighted that tracer studies are still not a routine part of monitoring the training system in SVG. Although the SSDA/NQD may not be best placed to implement tracer studies, the agency could lead the development of more effective tracer survey methodologies for their context and cascade these technologies to Training Institutes to ensure that survey data is available for evaluating individual courses as well as monitoring the system as a whole.

Results Based Financing SkYE grant disbursements were paid based on the achievement of agreed milestones.

Whilst milestones achieved early in grant project implementation ensured a degree of ‘advance payment’ to cover costs incurred in delivering subsequent milestones, all respondents to the interviews agreed that the approach was very onerous for the SSDA/NQD. SSDA and Training Institutes experienced under-financing during implementation, which caused delays in procurement of, and shortages in, consumables and other training resources, impacting the quality of training.

The SSDA/NQD could address these emergencies through co-financing, but in reality this meant the short-term cross-pollination of resources that then impacted on other projects as well. Partners in the SSDA grant felt that under-resourcing of the initial milestones arose because the SSDA grant was treated as ‘a single training partner’, rather than five training providers, and did not sufficiently consider the funding needed to deliver multiple courses across multiple sites simultaneously. However, it also reflects difficulties with results-based financing in high-cost delivery contexts.

Funds within the agreed budget to deliver the grant project were held back, for payment after milestones were delivered. Challenges with this model were exacerbated for the SSDA, given the volume of expensive technical training being delivered. Instructors expressed dismay at having to ration resources during training when funding was available but locked into later milestones with lower delivery costs.

This lesson speaks powerfully to the need for careful valuation of milestones in results-based grant funding.

9

Reporting to the Fund/er All grant partners were required to provide monthly progress reports to SkYE, which fed into the Fund Manager’s monthly reports to the Funder (UK aid). In interviews for this case study, staff from the SSDA noted that it was difficult to elicit reports from Training Institutes in a timely manner. The SSDA also coordinated (across five providers, multiple courses, and hundreds of students) the submission of education datasets to SkYE (i.e. enrolment data including key biographical data, plus routine updates on participation, completion, assessment and certification). Whilst SkYE grant partners report that the programme strengthened their monitoring systems, in future programming, the range of capacities required to deliver additional monitoring requirements need to be fully appreciated by ‘downstream partners’, as well as by the ‘lead’ grant recipient, and the Fund.

10

Conclusion

In awarding a grant to deliver certificated training to a qualification agency, SkYE leveraged considerable efficiencies for the Fund. These were achieved through the alignment with established governance, management, and quality assurance practices within the publicly-funded skills ecosystem. In the case of the SSDA/NQD and the Training Institutes, these practices were strengthened by professionalism within these partnerships that were established long before SkYE. However, whilst the structure presented efficiencies for the Fund Manager, it created significant capacity demands on the lead partner, particularly given the dual roles of grant management and certification body. Furthermore, when combined with results-based grant disbursements, the model impacted negatively on the quality of training delivery, compromising these relationships. Whilst partners did not reject results-based financing, lessons from the SSDA/NQD case emphasise that the design of grant disbursements patterns requires greater appreciation for the profiling of budget expenditure in complex grant projects, and that this is likely to require deeper processes of grant project co-creation.

11

www.skyecaribbean.com

@SkYECaribbean

@SkYEintheCaribbean

@skyecaribbean

The Skills for Youth Employment (SkYE) Programme in the Caribbean is a 5-year programme funded by UK aid which aims to develop a more productive, better trained and inclusive workforce across four countries- St Vincent & the Grenadines, St Lucia, Dominica and Grenada. The objective of the programme is to develop relevant market skills among young people (aged 15-30) in the Eastern Caribbean to promote greater youth employment, greater productivity and greater inclusiveness in the labour market, including better training and employment access for disadvantaged youth and people with disabilities. The programme aims to improve the quality, reach and sustainability of technical and vocational education and training in the four focus countries.