On-going evaluation of the COASTAL Project Report 2 – October 2011 (revised)
On-going evaluation of the COASTAL project Report 2: October 2011
Acknowledgements We would like to thank everybody who contributed to this report and the research upon which it is based. In particular we would like to thank the COASTAL project staff and the many individuals who were being supported by the project who took the time to speak to the Wavehill team. This evaluation would not have been possible without each of those contributions.
Report prepared by:
Richard Brooks, Director - richard.brooks@wavehill.com Louise Petrie, Researcher
Quality assurance:
Endaf Griffiths, Director
Client contact:
Clive Prior, Regional Project Director - clive.prior@swansea.gov.uk
Version:
Final (revised)
ŠCopyright Wavehill Ltd 2011
Page | 1
On-going evaluation of the COASTAL project Report 2: October 2011
Contents Summary of the report ................................................................................................. 3 1.
Introduction and Context ....................................................................................... 9 1.1. 1.2. 1.3.
2.
Outputs, results, impacts and outcomes.................................................................12 2.1. 2.2.
3.
The COASTAL project ......................................................................................................... 9 The evaluation ................................................................................................................. 11 Structure of the report ..................................................................................................... 11
Participation Outputs and Results .................................................................................... 12 Impacts and outcomes ..................................................................................................... 22
Views of Strategic Stakeholders; 2011 ....................................................................31 3.1. Work Programme............................................................................................................. 32 3.2. Progress ‘Work Star’ data is not being captured or used by all projects ............................ 32 3.3. An increased emphasis on the externalisation of social care services and independence within participants ....................................................................................................................... 33 3.4. New Service Models – externalised Social Firms ............................................................... 34 3.5. The balance between social care services and employment services ................................ 34 3.6. Weak employment demand ............................................................................................. 38 3.7. Competition for ESF participants ...................................................................................... 38 3.8. Cross (Local Authority) border services ............................................................................ 39 3.9. Procurement of services and suppliers is complex ............................................................ 39 3.10. Collaboration and sharing of good practise................................................................... 39 3.11. A critical response to this report................................................................................... 40
Appendix 1: Updated baseline data ..............................................................................41 Appendix 2: Analysis of COASTAL project management data ..........................................43 Output and result definitions ....................................................................................................... 43 Analysis of Coastal participants.................................................................................................... 45
©Copyright Wavehill Ltd 2011
Page | 2
On-going evaluation of the COASTAL project Report 2: October 2011
Summary of the report This is a summary of the October 2011 report of the on-going evaluation of the COASTAL project which is being undertaken over a five year period and run parallel to the delivery of the project. The evaluation is being carried out by the social research and evaluation specialists Wavehill. Evaluation is an important component of publically funded projects, services and activities. It is used to measure the efficiency and effectiveness of projects and programmes, to understand how policy is being delivered to citizens, and to test if project teams are achieving what they are being funded for. Evaluation identifies what is working well and provides a critique where strategy and delivery is not effective. The headline targets of the COASTAL project are currently under revision and awaiting WEFO approval, but the proposed targets at the time of writing are (October 2011): • • • •
8,500 economically inactive participants to be engaged; 1,000 participants to move into sustainable employment (12%); 6,500 participants gaining a qualification (76%); and 8,100 participants gaining other positive outcomes (95%).
A possible funding risk to COASTAL and delivery agents The evaluation has found that, in some cases, delivery agents have been slow to adapt into the COASTAL focus on employability and employment outcomes, rather than the provision of a more social care based model. This has quite significantly reduced the propensity of COASTAL to reach its outcome and results targets. As a result funding may be at risk to the COASTAL project as a whole or to individual delivery agents. Outputs, results, impacts and outcomes to date As of the end of June 2011, COASTAL has supported just over 2,194 (1,200; 2010) participants. This is lower than the numbers we would have expected to have been engaged so far if the project is to meet its participation targets of 8,500. The lowest ‘results’ across COASTAL are in the number of participants progressing into employment, only 37 from 2,194 participants. This is considered low as the original targets anticipated 32% (2,870/9,020) finding employment and the proposed revised targets 11.8% (1,000/8,500). Clearly, at the current rate of progress, far fewer participants will achieve employment outcomes than anticipated when the COASTAL project was planned. Essentially COASTAL must be able to demonstrate that participants increase their employability as a result of inclusion and participation in the portfolio of COASTAL projects. This means that other outcomes and results must be identified and recorded to demonstrate the benefits that COASTAL brings to its participants. As things stand, it is hard to identify any progress in terms of participants improving their employability or moving into employment outcomes based on the monitoring data that is currently available. This is not to say there has been no progression, but there is little data (evidence) to demonstrate the progress that has been made.
©Copyright Wavehill Ltd 2011
Page | 3
On-going evaluation of the COASTAL project Report 2: October 2011
The 2010 evaluation report highlighted the lack of WEFO results and the need to provide evidence of distance travelled, or soft outcomes. This is still the case in 2011 and, with so few WEFO results being recorded, it is even more important to measure the softer benefits of COASTAL participation. Work Star to record progression in COASTAL participants The primary tool being used to measure progress towards ‘employability’ and employment is Work Star. In 2010 this was selected by the COASTAL board as a regular way of capturing the change in individual participants, and then grouping or aggregating the scores to show project and then COASTAL progression. The evaluation has found that the bedding in of this tool has not been particularly successful, in part because there was not enough training and support in its use. A new focus should be placed on using the Work Star in 2012, with suitable training to ensure correct and frequent use and Wavehill providing the central COASTAL team with more support in analysing the data as it is collected. The balance between social care and employment and employability in COASTAL projects The balance between COASTAL as a social care project and COASTAL as an employment and employability project needs to be reviewed. ESF is primarily an economic programme that aims to raise the GDP of the convergence area in West Wales through a range of economic interventions. ‘To create a high skill, knowledge driven economy, with full employment, a skilled, adaptable workforce and responsive businesses, at the cutting edge of sustainable development. The aim of Priority 2 being … Increasing employment and tackling economic inactivity to raise levels of employment and economic activity, and secure higher participation in the labour market.’ WEFO - West Wales and the Valleys Convergence Programme Operational Programme for the European Social Fund 2007-2013 The COASTAL participants are in many cases a very long way from the labour market, and many have a very low probability of finding mainstream employment opportunities in the short to medium term. This places them in the category of not being able to work or not thinking about work. This was illustrated using the graphic below in the 2010 report. Figure 1: An illustration of the continuum of participants COASTAL will work with
Not thinking about work Participants at this point need in-depth support
©Copyright Wavehill Ltd 2011
Self-reliance In-work Work ready Participants at this point require less support
Page | 4
On-going evaluation of the COASTAL project Report 2: October 2011
Feedback from stakeholders in 2010 and 2011 suggests that the majority of those participants that the COASTAL project is currently working with are towards the left side of the line in the graphic Figure 1. This places a question about the propensity of the COASTAL participants to increase economic activity and increase employment under the Priority 2 theme from which COASTAL is part funded. Further evidence for this comes from the proportion of COASTAL participants that are reaching the ESF output and result targets of entering learning, gaining qualifications, finding employment and gaining skills that increase employability, measured as ‘other outcomes’. The proportions of participants reaching these results are very low, less than 3% progressing into qualifications, further learning or employment as a proportion of all participants. This suggests that the large majority of the COASTAL cohort still requires in-depth support and are probably attending some COASTAL projects primarily for social care, rather than for employability. Conclusion COASTAL is an ambitious project that brings together partners and delivery agents from different Local Authorities and silos of delivery. The ultimate aim is to create a paradigm shift in social care services, where people with various disabilities can move towards more mainstream and independent lives by becoming more employable and ultimately employed. There is much evidence to suggest gaining employment includes personal therapeutic benefits as well as social and economic status, but employment outcomes need to be made to realise these benefits. The COASTAL project has many of the elements in place to be successful, but must now focus on employability and employment. If it does not funding may be at risk as WEFO targets are unlikely to be met.
©Copyright Wavehill Ltd 2011
Page | 5
On-going evaluation of the COASTAL project Report 2: October 2011
Key findings and recommendations This final section of the report will briefly draw together the key findings of the evaluation and present the recommendations that are, accordingly, being made. Recommendation 1 Issue Employment outcomes are not being realised at a rate that will satisfy the WEFO contract. A greater number of job opportunities need to be identified for COASTAL participants. Evidence • The employment rate within Coastal is below 2%. Recommendation It may be that COASTAL, as a collective or the individual delivery agents, need to extend and emphasise the role of employment liaison officers, responsible for identifying businesses willing to work with COASTAL participants. The role could be to work between the COASTAL teams and the local employers to negotiate employment places and employment experience. This dialogue may be increased initially through SETs (Specialist European Teams – WEFO).
Recommendation 2 Issue Whilst there are differences between delivery agents, taken as a whole, COASTAL is not focussed enough on employability and employment outcomes. This creates a funding risk to COASTAL as a whole and also to individual delivery agents. Evidence • Analysis of outputs and results. • Discussions with stakeholders. Recommendation The correct balance between COASTAL as an employment project and COASTAL as a social care project needs to be agreed on between the partners as there are some significant differences in opinion and potentially a risk to funding.
©Copyright Wavehill Ltd 2011
Page | 6
On-going evaluation of the COASTAL project Report 2: October 2011
Recommendation 3 Issue Work Star should be used pervasively across COASTAL to demonstrate soft outcomes from delivery agents and progress towards employability to funders and partners. Evidence • Work Star is not being used as a regular progress measure by delivery agents. • Only 16% of participants have had two Work Star interviews and 3% three interviews. Recommendation The Work Star needs to be used more frequently, with a regular and repeated cycle of assessment taking place, and actions developed from this assessment. It is suggested that each participant undertakes an assessment every 120 days (4 months) and that this data is analysed by Wavehill and then sent as an information package to all delivery agents. The data will aggregate the progression across all participants, and then for each of the ten COASTAL partners. This will then allow each COASTAL partner to compare their data to the whole of the COASTAL project.
Recommendation 4 Issue The core concept of COASTAL being about employability and employment of participants needs emphasis. Evidence • Interviews with stakeholders identified a drift towards provision of care settings for participants. Recommendation Some ‘mission drift’ is noted amongst COASTAL stakeholders. This may be partly as employment outcomes are, so far, very few amongst the COASTAL projects. However, the WEFO / ESF contract is explicit in its requirement of a focus on employability and employment. It is also clear from participant interviews, and speaking to stakeholders, that some COASTAL participants are not ready for employment, and probably never will be. These participants require a social/day-care model, not an employability employment model.
©Copyright Wavehill Ltd 2011
Page | 7
On-going evaluation of the COASTAL project Report 2: October 2011
Recommendation 5 Issue Social firms may be required to create employment opportunities for COASTAL participants inside an Intermediate Labour Market model. Evidence • Low employment opportunities being found from private and public sector employers. • Successful models such as FRAME and PACK-IT. Recommendation The partners may wish to visit social firms from within Wales, but also wider afield to examine how social care, employability and employment and social firms can work together to provide service models with employment outcomes. If the partners wish, Wavehill can suggest several options for a COASTAL delegate to visit.
Recommendation 6 Issue The participation rate of males is greater than for females and the revised (2011) targets anticipate a 37% Female to 63% Male ratio. Evidence • Males are 2.7 times more likely to participate in COASTAL than Females. Recommendation The issue of inclusion equality was discussed in the 2010 report and still requires some attention. Why are males 2.7 times more likely to participate in COASTAL than females? The cause may be systemic, in that for some reason the COASTAL project is more likely to appeal to males than females, or it could be sociological, as parents and carers may feel that searching for employment is a more male activity than female and so less females are presented to COASTAL for inclusion. The COASTAL team should be aware that there is a gender based equalities impact assessment in place for the Convergence ESF area.
©Copyright Wavehill Ltd 2011
Page | 8
On-going evaluation of the COASTAL project Report 2: October 2011
1.Introduction and Context This is the 2011 and second report for the on-going evaluation of the COASTAL project which will be undertaken over a five year period alongside the delivery of the project. The evaluation is being undertaken by the social research and evaluation specialists Wavehill 1. This first section will provide the context for the remainder of the report by introducing the COASTAL project as well as the evaluation.
1.1. The COASTAL project COASTAL is a £52million Regional Strategic Project covering the six Local Authority (LA) areas of Bridgend, Neath Port Talbot, Swansea, Carmarthenshire, Pembrokeshire and Ceredigion, aimed at the promotion of vocational guidance, employment, skills training and lifelong learning opportunities for individuals who are currently economically inactive as a result of illness, disability, (mental illness, learning disability, physical disability, sensory impairment) substance misuse problems and/or the serious social disadvantage associated with the transition from long-term care into adulthood. Bridgend and Ceredigion Local Authority partner areas were not active in service delivery to June 2011; however, delivery of services has now started in both these areas. In addition to the Local Authority teams, there are four ‘cross-regional’ partners within COASTAL, taking on a dual role of both COASTAL partner and service delivery: • • • •
West Glamorgan Council on Alcohol and Drug Abuse (WGCADA) Swansea Drugs Project ‘PRISM’ – The Mid & West Wales Alcohol & Drug Advisory Service Community Chaplaincy – HM’s Prison Swansea
COASTAL also offers support, advice and guidance to employers, in order to raise their awareness of illness and disability issues and to assist them in engaging with and, hopefully employing, COASTAL participants. The headline targets of the COASTAL project are described in the table below. Note that the current targets and the proposed revised targets have been included.
1
www.wavehill.com
©Copyright Wavehill Ltd 2011
Page | 9
On-going evaluation of the COASTAL project Report 2: October 2011
Table 1: COASTAL project outputs and results; including original (old) targets and revised and proposed (new) targets 2011. Output Category
Target (old)
Target (new)
Total participants (all economically inactive)
9,020
8,500
Female participants
4,100
3,160
NEET participants
740
675
BME participants
225
180
Older participants
2,542
2,380
Participants with a work-limiting health condition or disability
8,610
8,113
20
10
Target (old)
Target (new)
5,412
6,500
2,870
1,000
1,763
2,750
8,570
8,100
Employers assisted or financially supported Result Category Participants gaining qualifications Participants entering employment - Economically inactive and unemployed Participants entering further learning Economically inactive and unemployed Participants gaining other positive outcomes Economically inactive and unemployed Source: COASTAL project proposed revised business plan 2011
Further information about the COASTAL project is available from the project’s website: www.coastalproject.co.uk
ŠCopyright Wavehill Ltd 2011
Page | 10
On-going evaluation of the COASTAL project Report 2: October 2011
1.2. The evaluation 1.2.1. Aims and objectives Over its lifetime, the evaluation will seek to answer four questions: • • • •
How have participants benefited from the support provided? How effectively has the project been delivered? How has the project changed the way in which support is provided in each area? What has been the cost-benefit and impact of the project?
This is the second of a number of reports that will be produced as the evaluation progresses. It focuses on the perception differences between what stakeholders perceive their involvement in COASTAL to be and the economic and employment focussed outcomes anticipated by the WEFO funding under ESF Theme 1- Priority 2.
1.2.2. Fieldwork undertaken for this report The report itself will be split into two parts. Firstly, this section includes interviews with 20 strategic level staff and related stakeholders, generally heads or directors of social care services for local authorities, with responsibility for their element of the COASTAL project. The agreed method for the 2011 fieldwork is to create 40 case studies of COASTAL participants, randomly selected to give an unbiased picture of the type of clients, what benefit they derive from participation in COASTAL and the change in their personal propensity to find employment. These will also be supported with around 20 interviews with delivery agent staff from 10 COASTAL delivery agents. The timing of the 40 case studies and supporting interviews with delivery agent staff will be the basis for the next evaluation report which will be produced in early 2012.
1.3. Structure of the report The remainder of this report is structured as follows: • • •
Section 2 - discusses the outputs, results and impacts of the COASTAL project to date; Section 3 - considers the strategic stakeholders perceptions of the COASTAL project to date; Section 4 - draws together the key findings of the reports and presents a number of recommendations.
©Copyright Wavehill Ltd 2011
Page | 11
On-going evaluation of the COASTAL project Report 2: October 2011
2.Outputs, results, impacts and outcomes This section will explore the outputs, results and impacts of the COASTAL project to date (June 2011). Due to the affects of a weak economy and potentially less participants due to Work Programme, COASTAL has made an application to reduce the number of outputs and results expected from the COASTAL project. Reduction has been applied for and the board is awaiting formal confirmation from WEFO. At the time of writing formal approval has not yet been received by COASTAL, although informal approval has, and so these output targets are included for reference only in this report.
Key findings • • •
The rate at which employment and employability results are being realised needs to increase significantly if the targets set for the COASTAL project are going to be achieved. The COASTAL Project Board needs to consider options for increasing the focus on achieving employment outcomes. To meet the new revised targets of 8,500 participants by December 2013 COASTAL needs to work with 630 new participants per quarter. With relatively few participants moving out of the COASTAL model into employment or learning and training this participation target is doubtful.
2.1. Participation Outputs and Results The following table sets out the targets that have been set for the COASTAL project in terms of outputs and results. It also notes the numbers achieved to date (as of 30th June 2011) and what percentage of the target that represents. Definitions for the outputs and results can be found in Appendix 2.
©Copyright Wavehill Ltd 2011
Page | 12
On-going evaluation of the COASTAL project Report 2: October 2011
Table 2: COASTAL project outputs and results; targets achieved to date including both old and new (2011) targets to June 2011
Target (old)
Target (new)
% of target (old) achieved
% of target (new) achieved
2021
9,020
8,500
22%
24%
698 1977 33 150
4,100 740 225 2,542
3,160
2,380
17% 267% 15% 6%
22% 293% 18% 6%
1121
8,610
8,113
13%
14%
Employers assisted or financially supported
0
20
10
0%
0%
Result Category
to June 2011
Target (old)
Target (new)
% of target (old) achieved
% of target (new) achieved
Participants gaining qualifications
114
5,412
6,500
2%
2%
Participants entering employment - Economically inactive and unemployed
31
2,870
1,000
1%
3%
Participants entering further learning - Economically inactive and unemployed
70
1,763
2,750
4%
3%
6,724
8,570
8,100
23%
19%
Output Category Total participants (all economically inactive) Female participants NEET participants BME participants Older participants Participants with work-limiting health condition or disability
Participants gaining other positive outcomes Economically inactive and unemployed
675 180
Source: COASTAL project management data up to the end of June 2011
2.1.1. Change in Participant involvement and outcome targets. The original participation and outcome targets proposed and agreed between COASTAL and WEFO at the start of the COASTAL project have been realised to be too high, partly due to the impact that Work Programme is having by reducing the number of potential participants for COASTAL to engage onto its project. In terms of overall participants the total has been reduced from 9,020 to 8,500. The most significant reduction in targets is for participants entering employment. This has a proposed reduction of 65%, from 2,870 to 1,000. There are several factors operating that have reduced the potential to deliver jobs to COASTAL participants, and these are explored later in this report, but in summary they are a weak labour market, a weak economy, and delivery agents not focussing on economic outcomes for participants. It is also worth noting that the ‘participants gaining qualifications’ target has increased as lower level skills that move participants closer to employment can now be counted and ‘participants entering further learning’.
©Copyright Wavehill Ltd 2011
Page | 13
On-going evaluation of the COASTAL project Report 2: October 2011
Table 3: COASTAL project original participation and output targets compared with 2011 revised targets. Original Target 9,020
2011 Revision 8,500
4,100
3,160
-23%
NEET participants
740
675
-9%
BME participants
225
180
-20%
Older participants Participants with work-limiting health condition or disability Employers assisted or financially supported
2,542
2,380
-6%
8,610
8,113
-6%
20
-50%
Results
Target
% change
Participants gaining qualifications Participants entering employment - Economically inactive and unemployed Participants entering further learning - Economically inactive and unemployed Participants gaining other positive outcomes Economically inactive and unemployed Employers adopting or improving equality and diversity strategies and monitoring systems
5,412
10 2011 Revision 6,500
2,870
1,000
-65%
1,763
2,750
+56%
8,570
8,100
-5%
20
10
-50%
Outputs Total participants (all economically inactive) Female participants
% change -6%
+20%
Source: COASTAL project management data up to the end of June 2011
2.1.2. Analysis of Participation Outputs and Results The original estimates of potential outputs and results made within a more sympathetic economic climate and labour market, where finding employment opportunities for people with long term unemployment histories, alcohol or drug issues, mental health and learning disabilities was easier. The original participation outputs and results targets have proven too high for COASTAL to achieve. This is demonstrated by the recent reduction in results targets from, for example 2,870 employment outcomes to 1,000, and the probable difficulty noted in this report in achieving even these reduced targets. This is not unusual. Tendering is a competitive process, and in some cases a beauty competition that requires the tenderer to become the most attractive funding option and high participation outputs and results make projects appear attractive and good value. Finally the various partners in the project were all anticipated to have delivered some results by this stage of the project cycle, and notably Ceredigion and Bridgend have not made any contribution to date as they have not started delivery. The COASTAL Business Plan (page 3.) noted that in Swansea between 2001 and 2007, projects working with the same COASTAL cohort had experienced a ~ 10% employment rate and ~ 62% qualification rate.
ŠCopyright Wavehill Ltd 2011
Page | 14
On-going evaluation of the COASTAL project Report 2: October 2011
In the 2010 evaluation report the following table was used to illustrate the short fall in participant numbers. The graph suggested that the total number of participants at the end of the programme would be around 3,520 instead of the 9,020 within the WEFO funding agreement. Graph 1: The target and actual / forecast number of participants supported by the COASTAL project during its lifetime – 2010
Number of participants
10,000 9,000 8,000 7,000
Target number of participants
6,000 5,000 4,000 3,000
Actual / forecast number of participants
2,000 1,000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Quarter number
Source: COASTAL project management data up to the end of June 2011
New targets have now been suggested and a similar straight line extrapolation has been calculated for this report (Graph 2). The graph again predicts a shortfall; of approximately 3,000 participants. This assumes that COASTAL works at the current rate of 321 new participants per quarter. To reach the revised target of 8,500 participants 630 new participants will need to be included in COASTAL for each quarter between September 2011 and December 2013. With few participants moving out of COASTAL and into other more independent settings, or out of COASTAL and into more dependent day care service settings, the concern is that it is not likely that COASTAL will meet the outcome target for participants. These revised rates demonstrate the importance of the contribution as a result of the roll-out of services from both Bridgend and Ceredigion, which are not yet included in these calculations as they were not active (i.e. providing support) at the time of this analysis.
ŠCopyright Wavehill Ltd 2011
Page | 15
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 2: Graph showing revised targets and the cumulative progress to date as two separate lines. The solid line shows the anticipated targets towards Dec 2013 and the dashed line progress at the current rate. 9,000 8,000 7,000 6,000 5,000 4,000 3,000 2,000 1,000 1
2
3
4
5
6
7
8
9
10 11 12 13
14 15 16 17 18 19 20
Cumulative target required to reach 2011 revised target (8,500) Cumulative performance extrapolation to Dec 2013 (8,500) Source: COASTAL project management data up to the end of June 2011
2.1.3. Analysis by Local Authority and cross-regional delivery agents Graph 3 illustrates how the number of participants is distributed amongst the Local Authority partner and cross regional delivery agents that make up COASTAL. The graph also shows that none of the participants have yet been registered by Bridgend County Council or Ceredigion County Council as COASTAL was not active to June 2011, however, delivery of services has now started in both these areas. Graph 3: A breakdown of COASTAL participants per Local Authority and cross-regional delivery agents 25%
21% 18%
20%
16%
15% 10%
8%
8%
8%
10%
10%
5% 0%
0%
0%
N= 2,021 (June 2011)
ŠCopyright Wavehill Ltd 2011
Page | 16
On-going evaluation of the COASTAL project Report 2: October 2011
The main contribution in terms of participation has come from the Swansea, Neath Port Talbot and Pembrokeshire Local Authority areas. However, the COASTAL project is not about participation, it is about the achievement of greater independence through obtaining qualifications, employment and other outcomes such as volunteering. Day Care settings have a high participation rate, but do not focus on achieving the employability and employment results that the COASTAL delivery agents have contracted to. The chart (Graph 4) shows two indicators for each project partner. The blue bar represents the wide definition in outcome achievement across the 10 projects. The red bar is the proportion of participants that have gained the narrower employability outcome results. Once the category which records ‘other’ outcomes is removed, only one partner (Pembrokeshire) has recorded more than 20% of results for its participants. To put this into context, the revised targets (2011) expect 76% of all participants to gain a qualification, 12% to enter employment, and 32% to enter further learning. Graph 4: Histogram showing the proportion of positive results across the 10 COASTAL partner regions and cross area delivery agents. The blue region showing ‘all’ results and the red region showing ‘employability and employment’ results. The graph demonstrates the need to create more employability (red) results. 120%
113%
All Results 94%
100%
96%
92%
99% 88%
87%
All Results except 'other' outcomes
80% 60% 37%
40%
23%
20%
13% 3%
0%
2%
9%
13%
8%
8%
0% 0% 0% 0%
N= 2,021 (June2011) *Note a participant may undertake employment activities, training, and work towards a qualification, which will register as multiple results, hence some delivery agents (Swansea Chaplincy) show more than 100% of results as a proportion of their participants. The COASTAL model plans to continue its activities post ESF funding, and will possibly then be paid on an outcomes based model, similar to other outcomes based social support models, notably the DWP Work Programme. If this is the case then it will require adequate ‘positive outcomes’ for the model to sustain its staff and costs.
©Copyright Wavehill Ltd 2011
Page | 17
On-going evaluation of the COASTAL project Report 2: October 2011
2.1.4. A review of separate WEFO result categories The following four graphs review the WEFO results categories separately, but disaggregated by cross area delivery agent and project partner. Graph 5 shows the qualifications gained by participants across the project partners and cross area delivery agents. It is clear that Pembrokeshire County Council, via its FRAME and Norman Industries delivery agents, have created an environment for qualifications; 63% of COASTAL project participants gaining qualifications come from the Pembrokeshire area. It is however noted that there has been a technical issue with the level of qualification being set at NVQ Level 2 and that this has now been reduced to NVQ Level 1 to accommodate the learning levels of some COASTAL participants. Furthermore, confirmation of certification can take up to a year in some extreme cases, and seven months for OCN accreditation, so many qualification results may not yet be recorded. Graph 5: Distribution of COASTAL participants gaining qualifications per project partner / area delivery agent. 70% 60% 50%
63%
40% 30% 20% 10% 0%
0%
0%
0%
5%
8%
7%
6% 0%
0%
(N= 114) - In total 6% (114/2021) of participants have recorded qualifications. The category for further learning (Graph 6) shows that some further learning is taking place across the project partners and cross area delivery agents, with 3% (70/2021) participants recording further learning results. Thirty-seven percent of COASTAL project participants undertaking further learning are however from the Swansea area.
ŠCopyright Wavehill Ltd 2011
Page | 18
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 6: Distribution of COASTAL participants undertaking further learning per project partner / area delivery agent. 37%
40% 35% 30% 25% 20% 15% 10% 5% 0%
23% 15% 8%
8% 6%
1%
0%
0%
0%
(N= 70) - 3% (70/2021) of participants recorded further learning results. The category for participants into employment shows that, for the COASTAL project as a whole, under 2% (36/2021) of participants have recorded employment results. The Swansea Prison Chaplaincy has recorded the most employment outcomes; 49% of COASTAL participants that have recorded employment outcomes are from that element of the project. Graph 7: Distribution of COASTAL participants exiting into employment per project partner / area delivery agent. 60% 50%
49%
40% 30% 16%
20%
11%
10% 0%
11%
8% 0%
3%
0%
0%
0%
(N= 36) - 2% (36/2021) of participants have recorded employment results.
ŠCopyright Wavehill Ltd 2011
Page | 19
On-going evaluation of the COASTAL project Report 2: October 2011
By far the highest category recorded is for ‘other’ outcomes. These are specified by WEFO 2 as; … the number of participants gaining intermediary outcomes as a result of participation in an ESF-funded project. Intermediary outcomes, such as completing courses, entering voluntary work, or attending a job interview, are those which are achieved as part of the journey to achieving final outcomes, such as entering paid employment or gaining qualifications. It is important to note that the ‘other’ outcomes category is recorded as ‘part of the journey …’ and not the journey itself. This again places an emphasis on using a tool such as Work Star to evidence and measure the journey, but also that the journey is towards more tangible employment and employability results. In total 78% (1702/2021) of participants have recorded other outcomes and it is important to the success of COASTAL to identify how these move participants closer to employment and employability results. Wavehill will give some focus to this issue when it visits the individual projects in the next stage of this evaluation. Graph 8: Proportion of participants with other positive outcomes per project partner / area delivery agent.
25% 20% 15% 10% 5% 0%
22% 16% 10%
10%
18%
10% 3%
3%
0%
0%
(N= 1,572) - In total 78% (1572/2021) of participants have recorded other outcomes. The analysis of the results categories above demonstrate a wide range of outcomes to date and considerable variance between the Local Authority areas. For example, Pembrokeshire is finding some employment outcomes, attributable in part to delivery agents FRAME and Norman Industries. Other COASTAL partners are finding employment outcomes more difficult to achieve. Accordingly, it may be that COASTAL, as a collective, or the individual delivery agents need to extend and emphasise the role of employer’s liaison officer, responsible for identifying businesses willing to work with COASTAL participants. The role could be to work between the COASTAL teams and the local employers to negotiate employment places and employment experience.
2
ESF Indicator Definitions, 2009
©Copyright Wavehill Ltd 2011
Page | 20
On-going evaluation of the COASTAL project Report 2: October 2011
This system worked well within the Cer Amdani! project run by MENCAP across South Wales. Using employer engagement created an internal employment rate of around 26%, approximately 400% better than the background employment rate of 6.4% in England. If this rate was replicated into the COASTAL project then employment outcomes of around 500 would have been achieved to date. To summarise, the employment and employability outcomes are currently far lower than WEFO has contracted with COASTAL. Although progression in the participants is being recorded as ‘Other Outcomes’ it is not clear how far these activities move the participants towards the employability and employment results. It is possible that the majority of the COASTAL participants are far from the employment market and in most cases may never find work or gain qualifications to increase employability and so may not be the best cohort to participate in a programme that places an emphasis on employability and employment. A more comprehensive analysis will be made in the next report, now that the evaluators are able to link participant data to Work Star data and other outcomes. A greater emphasis on identifying employment opportunities by officers dedicated to this role may benefit COASTAL participants by finding more full-time and part-time employment opportunities.
2.1.5. Characteristics of participants An analysis of the characteristics of the participants (please refer to Appendix 2) shows that: • • • • •
65% (1,323/2,021) are male, 35% (698/2,021) female; 23% (471/2,021) are in the age group 15-24 years, and 69% (471/2,021) 25 to 54 years; 45% (900/2,021) are disabled; 1.6% (33/2,021) are from a BME group; and 36% (727/2,021) had no qualifications at the time they entered the project.
Perhaps the most notable of the above statistics is that 65% of participants were, as of the 30th June 2011, male and 35% female, which does not reflect either demographics in Wales or the target participation rate of 45% of participants being female (please refer to Table 1). The Swansea Prison Chaplaincy – is only open to men as HMP Swansea is a prison for men and 8% of the participants to date have come from the Chaplaincy project. However even adjusting for this by subtracting the Chaplaincy project gives a Male to Female distribution of 62%:38%. The issue of inclusion equality was discussed in the 2010 report and still requires some attention. Why are males 2.7 times more likely to participate in COASTAL than females? The cause may be systemic, in that for some reason the COASTAL project is more likely to appeal to males than females, or it could be sociological, as parents and carers may feel that searching for employment is a more male activity than female and so less females are presented to COASTAL for inclusion. The COASTAL team should be aware that there is a gender based equalities impact assessment in place for the Convergence ESF area 3.
3
http://wales.gov.uk/docs/det/consultation/110221genderequalityconvergence.pdf
©Copyright Wavehill Ltd 2011
Page | 21
On-going evaluation of the COASTAL project Report 2: October 2011
Analysis by Swansea University 4 of the 2001 Labour Force Survey shows that males (21.25%) are more likely to be considered disabled then females (18.42%) in the work force. It is also worth noting that 1.6% of current participants are from BME groups, against a target of 2.5% that has been set for COASTAL (2.5% - see Table 1). This participation rate will also need to be reviewed using the race equality impact assessment guide 5.
2.2. Impacts and outcomes 2.2.1. WEFO monitoring requirements WEFO has set specific questions that need to be addressed as part of the assessment of the impacts of projects that are funded under Priority 2 of the ESF Convergence programme (please refer to section 1.1.1). The questions that are applicable to COASTAL are: • •
How many (net) participants have entered employment as a result of ESF assistance? How many of these participants are still in employment 12 months after receiving ESF assistance?
Figure 2; Copy of WEFO Guidance on how to measure the employment impact of the COASTAL project
Participants in employment at 12 months Definition: The net number of participants in employment 12 months following participation in an ESF-funded project. Net: achieved through Structural Fund activity and adjusted for deadweight, displacement, leakage and multiplier effects. Evidence: Evaluation Evaluation questions: ∗ How many (net) participants have entered employment as a result of ESF assistance? ∗ How many of these participants are still in employment 12 months after receiving ESF assistance? ∗ How many participants who had or were at risk of developing a work limiting health condition have remained in employment as a result of ESF assistance? ∗ How many of these participants are still in employment 12 months after receiving ESF assistance?
4
Disability, Gender and the Labour Market in Wales (Sloane, 2004)
5
http://wales.gov.uk/docs/wefo/publications/developingguidance/raceanddisability/090112reiaconvergenceen .pdf ©Copyright Wavehill Ltd 2011
Page | 22
On-going evaluation of the COASTAL project Report 2: October 2011
Currently 36 COASTAL participants have entered employment. The contributions by project are laid out in table 4 below. It is clear that the individual delivery agents now need to make substantial progress in moving participants on through the project and out into employment outcomes. Table 4; Total participants entering employment as a result of the COASTAL ESF assistance Entering Employment Swansea Prison Chaplaincy PRISM WGCADA Swansea Drugs Project Carmarthenshire CC C&C Swansea Neath Port Talbot CBC Pembrokeshire CC Ceredigion CC Bridgend CC Source: Wavehill 2011
18 0 4 3 1 1 4 5 0 0
48.6% 0.0% 10.8% 8.1% 2.7% 2.7% 10.8% 16.2% 0.0% 0.0%
The COASTAL project partners and delivery agents need to make significant progress towards developing employability and employment if the revised result of 1,000 participants entering employment is to be met. The target may look large, however, shared across the delivery agents it is more achievable. Each quarter 107 employment places must be found to meet the WEFO results target. This equates to around 3.5 jobs per month per project. Whether this is achievable or not needs further discussion with the project staff and a review of approaches to sourcing employment opportunities from the management team. An employment target of 1,000 from 8,500 participants equates to an 11.75% employment rate within COASTAL. To give this some context the Bevan Foundation (20096) wrote “In 2009, the proportion of disabled people who were working was very much lower in Wales than in Great Britain – just 38.9% were employed in Wales compared with 47.9%.” Although the COASTAL cohort is not solely made up of disabled people, approximately 50% are disabled. Another report that can inform the rate of employment from COASTAL is a recently published study on Employment Support Allowance cohort (DWP, 2011)7. This report covers ESA claimants in England, not Wales, and the cohort is only similar, rather than exactly the same as the COASTAL cohort, but the context is useful. The key point is that around 15% of ESA claimants who were not working prior to ESA registration were in work six months later.
6
http://www.ey2010.bevanfoundation.org/8.html
7
Routes onto Employment and Support Allowance (2011); Paul Sissons, Helen Barnes and Helen Stevens; DWP.
©Copyright Wavehill Ltd 2011
Page | 23
On-going evaluation of the COASTAL project Report 2: October 2011
Fig 3; Insert from DWP cohort study of ESA claimants 2011 Overall, people who were not working immediately prior to claiming ESA were much less likely than people who had been in employment to have returned to work. By the time of the baseline survey, only six per cent of those coming from non-work backgrounds prior to their claim reported being in work and working, compared to 30 per cent of people who had been in employment before claiming ESA. … By the follow-up survey, 15% of claimants from non-work backgrounds were in employment and working compared with 35% of claimants who were in work prior to their ESA claim. … Overall exactly a quarter (25%) of the whole ESA sample were in work and working at the followup survey. Source: Routes onto Employment and Support Allowance (2011); Paul Sissons, Helen Barnes and Helen Stevens; DWP.
A recent study by Wavehill (2011) into the Merthyr based Family Support Service for long term unemployed people found an employment rate of 18% (14/77) from within the survey cohort, although this was not a statistically significant survey.
Fig 4; Reference: Research Review of the impact of Welfare to Work Policy in Merthyr Tydfil, Blaenau Gwent and Rhondda Cynon Taff; Worklessness Sub-Group Merthyr Tydfil; October 2011 Interviewed 77 out of 200 vulnerable and workless individuals working with the Family Support Service in Merthyr April – May 2011. 14 had achieved employment (6 full-time, 7 part-time, 1 on work placement) with 2 not having worked at all before. 10 of the 14 say that their experience of the Family Support Service has made a lot of difference in them getting a job. Most findings focussed on softer indicators as a prelude to achieving employment. (See Table 5 next page).
©Copyright Wavehill Ltd 2011
Page | 24
On-going evaluation of the COASTAL project Report 2: October 2011
Table 5; Impact of Support Received on Employability Worse (%)
Better (%)
Much better (%)
All Better (%)
Base
Your confidence to look for a job
0
33
23
56
48
Your confidence to apply for jobs
2
26
26
51
47
Your prospects for getting a job
4
32
22
54
50
0
33
32
65
66
1
42
21
63
76
Employability
Your prospects for improving qualifications
your
Your understanding of your strengths and weaknesses Source: Wavehill 2011
Finally, an excellent source of comparative data comes from the 2009 ESF Leavers Survey8. A key finding was that 15% of economically inactive participants found employment prior to their ESF intervention, and 30% after ESF intervention; although 67% of jobs were classified as part-time. Seventy-five per cent of the ESF cohort gained a qualification (to NVQ Level 2 or above). This suggests a correlation between achieving qualifications and gaining employment outcomes for previously economically inactive ESF participants. The probabilities of ESF participants being employed after participation in ESF Priority 2 and Priority 3 projects were; Table 6; A comparison of findings from the ESF Leavers Survey 2009 and the COASTAL cohort ESF Employment Model Males are less likely to find work than females (0.89) People with work limiting illness are 65% less likely to work (B=0.35)
COASTAL context COASTAL has more males (65%) than females
COASTAL works with 45% disabled (B=0.45) ∗ possibly more as disability is self reported on enrolment People gaining an NVQ Level 2 via ESF are 55% COASTAL has gained 1% (27 qualifications) at more likely to find employment than those with NVQ 2 or higher. no ESF sponsored qualifications. People employed prior to ESF participation are 7 COASTAL participants are 93% unemployed prior times more likely to work post ESF. to starting and 74% with unemployment greater than 1 year. Source: ESF Leavers Survey 2009 and COSTAL cohort In all categories in Table 6 above, the COASTAL participants are less likely to work than the ESF survey and so it is much less likely that COASTAL participants will find employment outcomes compared to the ESF Priority 2 and 3 survey group.
8
The 2009 European Social Fund Leavers Survey, WISERD, (2010)
ŠCopyright Wavehill Ltd 2011
Page | 25
On-going evaluation of the COASTAL project Report 2: October 2011
To summarise, the labour market has deteriorated since 2009 (MENCAP) but has probably not changed much since July-Sept 2011 (DWP) and April-May 2011 (Merthyr) so it could be concluded that an employment rate of around 12% is achievable. The comparison with the ESF leavers’ survey shows that the COASTAL cohort is less likely to find employment outcomes. However, the greater employment rate from the MENCAP Cer Amdani! project suggests that a greater emphasis to engage local employers may be beneficial.
2.2.2. Impact on employers As noted in Table 1, one of the targets of COASTAL is to provide assistance to 10 (revised target) employers in the region in order to raise awareness of illness and disability issues and to assist them in engaging with and, hopefully employing, project participants. The main tool is to develop procedures and plans for employers, so that they can understand the issues and legislation around employing adults with learning disabilities and other health and mental issues. Some progress has now been made on this element, primarily with public sector employers rather than private businesses. For this report we will not undertake any further analysis of this element as although some plans are being developed, none are complete yet and so it is probably too soon to measure if these policies and procedures increase employers’ propensity to employ COASTAL participants. The COASTAL web site (www.coastalproject.co.uk) has several downloadable tools available to help organisations develop better informed equality and diversity schemes and procedures.
2.2.3. Baseline economic data Baseline data has been updated in Appendix 1. The main issues are that net business creation is currently negative; more businesses are registering closure, than new ones being started. The labour market remains weak, and the existing claimant count method of measuring unemployment may be distorted due to the introduction of Employment Support Allowance.
2.2.4. Soft outcomes and distance travelled by participants - The Work Star Model Soft outcomes in the context of COASTAL are the changes in behaviour, skills, communication, attitude, personal stability and so on that demonstrates that a participant has made progress whilst participating in COASTAL. The overall positive changes are often called the ‘distance travelled’ by an individual. The changes are often quite subjective and can fluctuate from day to day, especially in more chaotic, less stable individuals. As previously discussed, it is important to measure the progress made by participants towards greater employability, in order to demonstrate that the range of COASTAL delivery agents benefit the participants. These ‘soft’ outcome measures are also of interest to WEFO and can be used to demonstrate the effectiveness of the COASTAL project. As hard results are very low across COASTAL this places a greater emphasis on demonstrating success by using soft outcomes. A tool that is often used to identify the amount of change across these subjective dimensions is a star diagram. Each arm of the star identifies a dimension for measurement, and each arm has a scale to score that dimension, the Work Star uses a 10 point scale from 1 to 10.
©Copyright Wavehill Ltd 2011
Page | 26
On-going evaluation of the COASTAL project Report 2: October 2011
In the case of COASTAL, soft outcomes are being used to monitor how participants move towards a situation where they are ready to move into employment and the Work Star is being used to collect the necessary information from participants. The Work Star sets out five stages within a ‘journey into work’ that an individual will take if and when they progress: a) b) c) d) e)
Not thinking about work: substantial barriers to work or advancement and you can't way round them (Score: 1-2) Thinking about work: substantial barriers but you are working out how to address (Score: 3-4) Making progress: some barriers overcome but others are still there (Score: 5-6) Work-ready with support: barriers mostly overcome or can be worked around; you support to maintain progress (Score: 7-8) Self-reliance: in work, work-ready, or engaged in work-related training. By then you need support from the programme (Score: 9-10)
see a them
need don't
As part of their monitoring procedures, the COASTAL delivery agents are tracking participants as they move through these stages. At the time the analysis for this report was undertaken, the Work Star data represents the opinions of 1,375 participants, however, only 324 have been assessed twice, and 56 more than twice. This greatly limits the accuracy and use of the data in this analysis. To demonstrate the overall ‘soft impact’ of the COASTAL delivery agents, the mean change has been calculated from Work Star assessment one to two, and then two to three, and an overall change. The change to assessment three is not statistically reliable as so few participants (56) have actually had a third assessment but does provide some indication of the distance travelled by the participants in question. Table 7 shows the baseline or starting point of the Work Star data for each of the categories monitored: • • • • • • •
Job-specific skills: the extent to which participants have the skills and experience necessary to get a job. Aspiration and motivation: whether participants know what sort of work that want to do, are motivated to do it and believe that it is possible for them. Job-search skills: the skills participants need to find and get a job or training – researching opportunities, confidence on the phone using a computer, feeling able to present their strengths in a CV or interview, etc. Stability: improvements to the day to day pattern of participants lives; it covers issues relating to drugs and alcohol, dept problems, insecure housing, etc. – issues that may affect participants ability to get a job or further education/training. Basic skills: literacy, IT, numeracy, use of the telephone, etc. Social skills for work: how participants relate to others and meet expectations in a work or learning situation. It includes, getting on with people, self-confidence, turning up on time, behaving appropriately, etc. Challenges: the practical issues that participants may see as barriers to participating in work or training; childcare responsibilities, health issues, disability, age, loss of benefits, etc.
©Copyright Wavehill Ltd 2011
Page | 27
On-going evaluation of the COASTAL project Report 2: October 2011
Table 7; Work Star data, baseline 2011 N Basic skills 1246 Stability 1363 Social skills for work 1375 Job specific skills 1364 Aspiration and motivation 1375 Challenges 1363 Job search skills 1364 Source: Wavehill / Work Star 2011
Mean 6.0 5.8 5.7 5.0 5.0 4.8 4.5
Std. Deviation 2.7 2.5 2.4 2.2 2.2 2.3 2.4
Low σ
High σ
3.3 3.3 3.3 2.8 2.8 2.5 2.1
8.7 8.3 8.1 7.3 7.2 7.2 6.9
The range of opinion for each of the Work Star categories is quite high, and so the standard deviation of each mean value is wide. Taking ‘basic skills’ as an example, the average (mean) response is a score of 6.0, but the standard deviation range of σ=2.7 means that approximately 68% of all views or scores fall between 3.3 and 8.7 – which ranges between the stages (b) Thinking about work and (d) Work ready with support in the Work Star ‘journey to work’ (see above). The mean is lowest for the categories ‘challenges’ and ‘job search skills’ suggesting that those are the areas were participants will need to make greatest progress before they are work-ready. Graph 9 shows the percentage change in participants’ views. The COASTAL project is having an effect on the participants. The green line shows a positive change in all areas except for ‘basic skills’. The blue line shows that most of the change takes place between assessment one and two, and less so between two and three, and there is some reversal of personal assessment between assessments two and three. Overall ‘aspiration and motivation’ (+22%) is the category that shows the greatest change, with ‘stability’ (+14%), and ‘job specific skills’ (+11%) next. The categories showing the least progress are ‘basic skills’ (-1%) which appears to make no difference across the cohort, ‘challenges’ (+3%) and ‘social skills for work’ (+6%). It appears that more work needs to be done in the ‘basic skills’ area although the ‘baseline mean’ is highest for this category suggesting that other categories may need to be prioritised. It must also be noted that some delivery agents have not entered any data into the Work Star, and some very little data, so these findings are therefore not wholly representative.
©Copyright Wavehill Ltd 2011
Page | 28
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 9: ‘Spider’ chart showing the categories in the Work Star tool with the greatest rate of change using paired mean scores
Stability
Aspiration and Motivation 25% 20% 15% 10% 5% 0% -5% -10%
Basic Skills
% change 1 to 2 % change 2 to 3
Social Skills for Work
Job Specific Skills
Challenges
Job Search Skills
Source: Wavehill/Work Star 2011 Table 8; Table showing the mean change between Work Star assessments
Aspiration and motivation Basic skills Challenges Job search skills Job specific skills Social skills for work Stability Source: Wavehill/Work Star 2011
% change 1 to 2 15% -2% 9% 10% 14% 7% 15%
% change 2 to 3 6% 2% -5% -1% -2% 0% 0%
% overall change 22% -1% 3% 8% 11% 6% 14%
The Work Star needs to be used more frequently, with a regular and repeated cycle of assessment taking place, and actions developed from this assessment. It is suggested that each participant undertakes an assessment every 120 days (4 months) and that this data is analysed by Wavehill and then sent as an information package to all projects. The data will aggregate the progression across all participants, and then for each of the ten partners provide a tailored analysis by project for each. This will allow each partner to compare their data to the whole of COASTAL. This is not intended to be used as a league table of results, but to give some clarity and quantitative view about what is happening within the projects to the participants. Currently there is a ‘black-box’ approach to the COASTAL projects, as each project is undertaking its own activities in its own way in order to suit its particular participants and their needs. The Work Star analysis will hopefully allow some comparison of approaches, identify good practise and promote the capacity to share these methods.
©Copyright Wavehill Ltd 2011
Page | 29
On-going evaluation of the COASTAL project Report 2: October 2011
Also the Work Star data needs to be integrated with the monitoring data so that a more accurate analysis of the benefits of participation with delivery agents can be recorded. The analysis will then support greater detail and clarity. As an example, the graph below reviews the Work Star progress made disaggregated by gender. This shows that males benefit more from Job Specific Skills and Social Skills for Work, but females in the other categories. Again, this analysis must be treated with caution as the data is not comprehensive enough to make any statistically certain statements. Also the scoring mechanism by different Work Star users is not yet uniform, so one respondent may score a 4 and another a 6 but both for a similar interpretation of the category, Basic Skills for example. Finally an enhanced training programme is recommended for delivery agents, so they understand how to use Work Star correctly. Graph 10: Graph showing Work Star categories disaggregated by gender. 70% 60% 50%
58%
57%
53%
49%
43%
40% 30%
47%
44% 37%
31% 29% 20% 20%
20%
20% 16%
Female Male
10% 0% Aspiration Basic Skills Challenges Job Search Job Specific Social Skills and Skills Skills for Work Motivation
Stability
Source: Wavehill / Work Star 2011 To summarise, the Work Star data is currently not robust enough to be useful for the evaluation of soft outcomes within COASTAL. The frequency of assessments must be made using a regular time period and every four months is suggested. Linking the data to other demographic information such as previous work history, age group, disability category, and so on will, as in the ESF Leavers Survey, provide far more useful data and powerful analysis.
2.2.5. Benefits identified by participants The 2011 method proposed using 40 case studies to show the range of participants, their experiences on COASTAL and benefits and outcomes from participation. The method of selecting appropriate case studies was discussed by the evaluators and the partnership board and a nonbiasing randomised selection method was proposed. Essentially this method prevents project managers presenting for case study their most positive participants, which in turn may provide a positive bias on participant outcomes and opinions. The selection of participants is negotiated, so a participant is empowered to decide whether they wish to be involved in a case study, or not. This process of selection and consent has taken considerably longer than originally anticipated and so the participant’s views as case studies will not be included in this report, and will be published as an separate report early in 2012.
ŠCopyright Wavehill Ltd 2011
Page | 30
On-going evaluation of the COASTAL project Report 2: October 2011
3. Views of Strategic Stakeholders; 2011 Key findings • • • • • • • • •
A funding risk exists to the COASTAL partnership due to the low employment and employability outcomes and results. Work Programme issues and uncertainty used a lot of the COASTAL team time and resources in 2011. What additional support can be offered to COASTAL clients as they leave COASTAL and move into employment? Work Star data could be centralised and analysed more frequently by Wavehill. Some ‘mission shift’ away from employment towards externalisation of services has been observed in stakeholder interviews. COASTAL could develop social firms similar to FRAME in order to create employment opportunities, and possibly sustainability, but outside of ESF funding. The employment market for COASTAL participants is weak and this is probably affecting employment outcomes. What is the correct balance between COASTAL as an Employment project and COASTAL as a Social Care project? Greater collaboration, especially between project delivery teams, should be encouraged.
This section discusses the views of the strategic stakeholders of the COASTAL project. The interviews were undertaken during July and August 2011. The range of stakeholders includes senior management (strategic level stakeholders) and senior staff who are providing support to participants (referred to in the following discussion as ‘delivery agents’). The discussions were led by a topic guide, the main direction of the topics being; • • • •
How effective is the project being delivered to date now it has entered the post start-up stage How have participants benefitted? How has the project changed the way in which support is provided in your delivery area? Who now provides the delivery, has this changed in structure, from Local Authorities to Social Enterprises / Third Sector for employing and training people, for example?
The key findings of the interviews are discussed below. These views will be supplemented with approximately 20-30 interviews with staff from across the delivery agents. The interviews will take place parallel to the beneficiary interviews where possible and so will also be included in the next report due in early 2012. The interviews will include a focus on the important issue of how much work with participants is within the context of employability and employment and how much is developing stability and skills, and other factors.
©Copyright Wavehill Ltd 2011
Page | 31
On-going evaluation of the COASTAL project Report 2: October 2011
3.1. Work Programme The new Work Programme introduced by the DWP during 2011 has used up much time and resources from April to July 2011 on establishing how COASTAL would be affected by the changes it brings. Without reporting the nuances of the various interpretations of how COASTAL may have been affected by Work Programme, the final understanding is that some COASTAL clients currently on Incapacity Benefit (IB) and transferring onto Employment & Support Allowance (ESA) may not be eligible for ESF support. Furthermore the Work Choice programme has also made some participants ineligible for inclusion on the COASTAL project. The estimated reduction in the number of COASTAL participants varies between partners and delivery agents, with several estimates at around the -10% level, but with two partners noting that they have around 40% - 50% of participants on IB/ESA and so the effect may be greater. The targets for the COASTAL programme have been reduced from 9,200 to 8,500 to reflect the reduction in participant qualification for COASTAL. The overall effect on COASTAL as an employment focused project may not be too large as the greater issue is not currently with the supply of participants into the COASTAL project, but transition within the delivery agents’ projects and results into employability and employment.
3.2. Progress ‘Work Star’ data is not being captured or used by all projects As previously discussed, the Work Star progression tool was chosen by COASTAL from several progression tools as the best tool available to track soft outcomes. Yet, it is not being fully used throughout the COSTAL project. Other potential tools have been suggested by stakeholders and one similar progression tool is used within the Swansea Chaplaincy. The stakeholder interviews demonstrate that Work Star in concept is understood, in that it collects the data to evidence the changes across participant groups, however there is a wide range of reactions to it. Six stakeholders supported the Work Star, four were indifferent to it, three were negative about its use and two stated they would not use it at all. One stakeholder mentioned that Work Star was inappropriate for its participants because it had the word ‘work’ in its title. However, when the Work Star was presented to the project teams at the November 2011 conference, the reaction to it was more positive, and attitudes may have shifted as more training and support in its use and analysis has now been offered. WEFO will recognise soft outcomes as contributing to the overall understanding of a project. Currently the COASTAL project has very low employment and employability outcomes and so it is doubly important to demonstrate the progress made by COASTAL participants, and their increasing employability. The Work Star should therefore be updated on a regular cycle (120 days or 4 months has been suggested by Wavehill) in order to demonstrate progression. The issue of data management and collection in general and Work Star specifically was a common theme throughout the stakeholder interviews. There were several suggestions that workshops or collective discussions should be undertaken to negotiate the central collection and analysis of data. It may also be beneficial to discuss the centralisation and analysis of data. This will allow the evaluators, central COASTAL team, stakeholders and delivery agents to discuss a unified solution.
©Copyright Wavehill Ltd 2011
Page | 32
On-going evaluation of the COASTAL project Report 2: October 2011
The analysis of the Work Star and other data should be for a clear purpose, so that outcomes and impact can be measured effectively. The key question of which groups benefit the most from COASTAL and which the least is difficult to inform without a quantitative data source that cuts across all stages of a participant’s progress through COASTAL. Understanding which individuals are most likely to get employment or near employment outcomes and who are not is also important if the COASTAL project is to be successful within its ESF contract.
3.3. An increased emphasis on the externalisation of social care services and independence within participants The original emphasis of the COASTAL project was to move service delivery models away from long term social care settings into creating a different paradigm of learning, skills and employment for COASTAL participants. The business plan emphasises the need to develop employability skills and employment outcomes are currently targeted at 12% (1,000/8,500). During the interviews with stakeholders a small shift was noted in the responses to the question about the core role of COASTAL; there was less focus on employment and employability and more with developing independence in participants and externalising services. The externalisation of services is the movement of statutory care services from within statutory and local authorities to external service providers, often charities and social firms. The authority then remains as a contract holder, monitoring the quality and delivery of the service. This is a very current debate amongst stakeholders as externalisation is seen by some as a method of introducing more competition between service providers and as a means to reduce the financial costs of providing social care from within Local Authorities and Health Services. The COASTAL project does encourage project partners to consider the externalisation of services. COASTAL allows for service delivery by competitive tendering of service provision, in theory at least opening up the possibility of service provision from any EU member state. However, this is seen as a secondary outcome of the COASTAL model and not its primary function. The primary function is to work with participants, who may traditionally have been cared for within social/day-care settings, to undertake employability skills development, learning and to find employment. The externalisation of services are, it seems, being used by some partners to justify the participation in the COASTAL partnership, rather than the core concept based around employment outcomes. Inclusion in the COASTAL partnership does benefit the partners financially as their social-care service budget is multiplied by the ESF contribution, but ESF Priority 2 requires a focus on economic outcomes above social care outcomes and ‘Making the Connections’ collaborative delivery structures. Some ‘mission drift’ is noted amongst COASTAL stakeholders. This may be partly as employment outcomes are so far very few amongst the COASTAL projects. However, the WEFO / ESF contract is explicit in its requirement of a focus on employability and employment. It is also clear from participant interviews, and speaking to stakeholders, that some COASTAL participants are not ready for employment, and probably never will be. These participants require a social/day-care model not an employability employment model. Finally, stakeholders were more likely to emphasise the participation targets than their employment outcome targets. This possibly explains the position of many COASTAL partners, as they are primarily concerned with the provision of a service and setting for their clients, rather than delivering employment and employability outcomes.
©Copyright Wavehill Ltd 2011
Page | 33
On-going evaluation of the COASTAL project Report 2: October 2011
3.4. New Service Models – externalised Social Firms The Pembrokeshire participants benefit from the externalised services of FRAME and Norman Industries. With such a weak labour market and so few COASTAL participants reaching employment outcomes, the partners need to identify methods of creating employment outcomes. A shared vision of creating employment opportunities through social firms may be a positive outcome for COASTAL at a strategic level. Although ESF cannot be used to create social firms, the partnership can be used to shape the strategy and encourage discussions with potential social firm suppliers. The interviews with stakeholders support this, five interviewees stated that the development of social firms to deliver externalised services, provide apprenticeships and ultimately employment, is an opportunity for the partners. The partners may wish to visit social firms from within Wales, but also wider afield to examine how social care, employability and employment and social firms can work together to provide service models with employment outcomes. If the partners wish, Wavehill can suggest several options for a COASTAL delegate to visit.
3.5. The balance between social care services and employment services COASTAL is delivered differently from area to area and approaches to delivery between each Local Authority partner and delivery agent can vary. However, there is a dichotomy emerging within the COASTAL project as a whole. The funding and the business plan are based within the economic context of participants moving from COASTAL into employment. This core reason behind the whole ESF fund across Europe ‘… was set up to improve employment opportunities in the European Union and so help raise standards of living. It aims to help people fulfil their potential by giving them better skills and better job prospects.’ (DWP, 2011). Priority 2: Increasing employment and tackling economic inactivity to raise levels of employment and economic activity, and secure higher participation in the labour market.“ WEFO The COASTAL participants are in many cases a very long way from the labour market and, as demonstrated by the baseline data collected using the Work Star discussed in the previous section, many have a very low probability of finding mainstream employment opportunities in the near future. This places them in the category of not being able to work or not thinking about work. This was illustrated using the graphic below in the 2010 report. Figure 5: An illustration of the continuum of participants COASTAL will work with from 2010 evaluation report.
Not thinking about work Participants at this point need in-depth support
©Copyright Wavehill Ltd 2011
Self-reliance In-work Work ready Participants at this point require less support
Page | 34
On-going evaluation of the COASTAL project Report 2: October 2011
This was also further illustrated by the progression model (below), which was included as figure 6 in the 2010 report. The point was made that much participant activity is based within overcoming barriers and less within the categories of employability. Figure 6; From the 2010 evaluation report illustrating categories of participant assistance and development with COASTAL
Feedback from stakeholders in 2010 and 2011 suggests that the majority of those participants that the COASTAL project is currently working with are overcoming barriers to employment. This places a question about the propensity of the COASTAL participants to increase economic activity and increase employment under the Priority 2 theme from which COASTAL is match funded. The evidence for this comes from the proportion of COASTAL participants that are reaching ESF outcome and result targets of entering learning, gaining qualifications, finding employment and gaining skills that increase employability, measured as ‘other outcomes’. The proportions of participants reaching these results are very low, less than 3% progressing into qualifications, further learning or employment as a proportion of all participants. This suggests that the large majority of the COASTAL cohorts still requires in-depth support and are probably attending some COASTAL delivery agents primarily for social care, rather than for employability. However the origination of many of the participants (from social/day-care settings) and their distance from the labour market suggests that the majority of COASTAL participants will not progress into employment outcomes and may not record employability results such as qualifications or further training. “In the first two years the aim was to enrol all known client groups, people who go to Day Centres” Source; Stakeholder interview 2011 The revised outcome targets (2011) propose around 1,000 employment outcomes from 8,500 participants. This is a 12% employment rate across COASTAL still leaving nearly 90% of participants without employment and held within the various COASTAL delivery agents.
©Copyright Wavehill Ltd 2011
Page | 35
On-going evaluation of the COASTAL project Report 2: October 2011
This leads to a fundamental question about what the role of COASTAL is; 1. Is COASTAL a social/day-care project where around 10% of participants gain employment and the rest undertake personal and skills development; or; 2. Is COASTAL a project that gets participants to gain in confidence, enhance skills and training and then ultimately employment? The stakeholder interviews identify COASTAL more as a social care model. This issue of the balance between providing stabilisation and care for participants and progression towards employability and employment results was discussed in the 2010 evaluation report. The comments below support both sides of this discussion.
Figure 7; Extracts from stakeholder interviews 2011 1. The funding allows us to develop the current services in the way they work and having more resources via match funding to allow us to do more with the clients. We provide social care but in a different way now, developing day services, then progression into COASTAL activities, training, volunteering, etc. It allows progression and this is the main concept for us. We measure the benefits as more independence and less dependency. 2. It provides opportunities for those that did not have opportunities before. Some people have moved on with their lives considerably and although it is still small numbers that have achieved it is still way beyond what statutory organisations would have achieved, some have better emotional well-being and others have jobs as an outcome. 3. I think you do have to accept that if this service is an alternative to Day Services then some participants may have been in Day Care for years and years and will be used to just achieving an outcome ‌ I think some participants will always need support and should we be saying it has to be a work environment, can't it just be that they end up in a nice and safe environment?
The ESF contract and the core management team hold the opinion that COASTAL is about moving participants into employability and employment results, whereas the stakeholders interviewed appeared to consider the project is more about personal skills enhancement which lead to increased employability, testing models for the externalisation of services, and in some cases social care provision.
ŠCopyright Wavehill Ltd 2011
Page | 36
On-going evaluation of the COASTAL project Report 2: October 2011
Figure 8; Extract from the COASTAL Business Plan (Business Plan Aims section) illustrating the employment focus of the project aims Aim 2; To develop a strategic direction to offer opportunities for learning and work experience to move disadvantaged citizens from our target group, [i.e. individuals who are currently economically inactive as a result of illness, disability, (Mental Illness, Learning Disability, Physical Disability, Sensory Impairment) substance misuse problems and/or the serious social disadvantage associated with the transition from long-term care into adulthood], from being economically inactive to active in the labour market. Aim 5; To support and enable access to mainstream employment and learning opportunities with support where necessary. Aim 6; To gain the support of employers to provide opportunities, by offering support, information and guidance in all aspects of engagement with our participants. Aim 7; To promote permanent employment, paid at the same rate of pay and same conditions as any other employee. Aim 9; To assist our target participants to obtain and retain work by developing employment skills required by local employers.
Figure 9; Extract from the 2010 evaluation report in relation to the key role of COASTAL Their concern was that this focus on moving participants into employment (rather than overcoming the barriers to employment) made the project irrelevant to many of the participants that they worked with. Further, there was a concern that participating in the project could even have a negative impact on some individuals by jeopardising the progress that they had previously made. “There’s no hope of some of the people that we work with getting a job” – stakeholder at a project delivery level “You can’t push the people that we work with that hard” – stakeholder at a project delivery level
The correct balance between COASTAL as an employment project and COASTAL as a Social Care project needs to be agreed on between the partners as there are some significant differences in opinion and potentially a funding risk.
©Copyright Wavehill Ltd 2011
Page | 37
On-going evaluation of the COASTAL project Report 2: October 2011
3.6. Weak employment demand Interviews with stakeholders clearly identified the weak labour market as both a weakness and a threat to COASTAL. The economy in the UK and in West Wales is weak and deteriorating. The recent monthly announcement of the state of the economy and labour market in South West Wales (Statistics Wales, 20119) stated that; • • •
South West Wales had the lowest employment rate – 63.7% vs. 66.3% for Wales The second highest unemployment rate – 8.5% vs. 8.5% for Wales But positively the second highest average weekly earnings – wage index of 87.5 vs. 86.2 for Wales
Finding employment opportunities for COASTAL participants is undoubtedly difficult, especially as some sectors such as retail, where traditionally COASTAL clients have found work are particularly bad. The evaluation should now place some focus on; 1. How delivery agent teams are finding work opportunities for their clients. 2. Where delivery agent teams are finding other near employment opportunities, such as work experience and volunteering. 3. How delivery agent teams promote employability learning, skills development/training and qualifications.
3.7. Competition for ESF participants Where several ESF funded projects have been commissioned to provide similar services to a similar client group this has created inter-project (delivery agent) competition for ESF referrals. ESF rules only allow one individual to have ESF funded activities recorded against their name, and so it is important that ESF projects do not duplicate services. The stakeholders do not recognise this as an issue across the COASTAL projects and have identified some referrals coming from Genesis, Work Ways and Want 2 Work. The stakeholders do not notice any duplication or competition for participants. However, referrals have fallen from an estimated 10 a month at the start of delivery, down to only 1 or 2 a month now, and this may be a result of not being allowed to share outcomes with ESF/WEFO.
9
Regional Economic & Labour Market Profile – South West Wales – September 2011
©Copyright Wavehill Ltd 2011
Page | 38
On-going evaluation of the COASTAL project Report 2: October 2011
3.8. Cross (Local Authority) border services The stakeholder interviews identified the need to share good practise and resources across Local Authority borders. One criticism is that service still stops at the Local Authority boundaries and some clients are tied to these geographies and cannot select the service that is best for them, or are not offered the range of services across the COASTAL partnership to choose from. Clearly issues of transport and travel will restrict many participants to their nearest service but offering access to services more widely demonstrates the effectiveness of collaborative partnerships. This is reiterated in the Framework for Sustainable Social Services in Wales policy document. Also, the highly influential Beecham Report and the subsequent policy paper ‘Making the Connections; Delivering Services Across Boundaries’ promotes shared and collaborative delivery, as do the Local Service Boards for social care services. The very fact that COASTAL exists proves that such collaborations can be formed. The balance to this comment is that four delivery agents are delivering services not based within Local Authority borders and as services increasingly externalise it is anticipated that more will do the same.
3.9. Procurement of services and suppliers is complex In the 2010 report the issue of the procurement requirements for contracts being complex were covered in some detail. However, the same issues were again mentioned by four stakeholders during interviews as a weakness of the COASTAL project.
3.10. Collaboration and sharing of good practise The stakeholder interviews identified that although collaboration between partners is taking place at the strategic level through various COASTAL meetings. The working practices and day to day work between delivery agents are not widely compared or shared. Collaborative projects become most effective when good practice is transferred across the partnership, affecting change to make other partners more effective in terms of service delivery and ultimately to the benefit of the participants. A Development Officer Group has been established although it is not clear yet what they have so far developed or collaborated on. The evaluation visits to delivery agents are due to commence shortly and will identify areas where collaboration and sharing can take place so that delivery staff can share their experiences and good practise. A new mandate to demonstrate collaboration and sharing between delivery agents on a bimonthly frequency has been agreed. Collaboration does not need to be limited to COASTAL partners, but could include Work Programme teams and Job Centre plus staff too, so that a better understanding is created between these service providers, all who are essentially trying to achieve the same outcomes across similar client groups within the same geographical areas. If the Development Officer Group wishes, Wavehill can suggest some ideas to form into actions to increase the exchange of ideas and collaboration. Collaborative groups and partnerships follow the policy direction from the Welsh Government under the Making the Connections policy.
ŠCopyright Wavehill Ltd 2011
Page | 39
On-going evaluation of the COASTAL project Report 2: October 2011
3.11. A critical response to this report A criticism of this report has been recognised (and valued) that “although only 10% of Participants are expected to achieve employment outcomes, what about the service to the other 90% of participants.” We acknowledge this but would stress that it is not possible to capture all views and opinions in every report when using a rolling evaluation programme because different waves capture different opinions. It is therefore worth noting that reports produced by the evaluation in 2012 will consider the following questions: • • • • • •
What “other positive outcomes” have been achieved and does this show the progression WEFO would want to see? Is this [above] reflected in the Work Star – is the service provision effective? Is training being delivered and achieved? Is training effective in moving participants towards employability? Are participants leaving the project? Where do participants go when they leave the project – is it further learning?
©Copyright Wavehill Ltd 2011
Page | 40
On-going evaluation of the COASTAL project Report 2: October 2011
Appendix 1: Updated baseline data Please note – the baseline data has been updated using a data series from Statistics Wales, and so will be comparable in the future, and in fact accessible at any time on line from the Statistics Wales web site.
Per cent unemployed claimants
Graph 11: Claimant Count – the claimant count peaked in August 2009. Claimants may have fallen due to a systemic change in workless benefits rather than an improvement in labour market demand. Claimant count based unemployment rate. 5 4 3 2 1 0 Wales
Ceredigion
Pembs
Carms
Swansea
Neath Port Talbot
Bridgend
August 2007
2.1
1.3
1.3
1.9
2.2
2.3
2.2
August 2008
2.4
1.2
1.5
2
2.4
2.5
2.3
August 2009
4.2
1.8
3.2
3.3
4
4.4
4.7
August 2010
3.7
1.7
3.2
3
3.5
3.6
3.9
August 2011
4
1.9
3.1
3.2
3.8
4
4.2
Source: Statistics Wales; October 2011
Per cent labour market activity rates
Graph 12: Disabled Employment Rate 2007-2009 – The employment rate amongst disabled people in September 2011 is around 9% for Wales but significantly higher in Swansea, Bridgend and Neath PT Labour market data for disabled persons - September 2011. 70 60 50 40 30 20 10 0 Wales
West Wales & Valleys
Ceredigion
Pembs
Carms
Swansea
Neath Port Talbot
Bridgend
Economic inactivity level
45.7
43
43.9
49.6
43
49.2
36
43.8
Economic activity rate
41.7
39.3
40.9
48.6
39.6
43.7
32
37.7
Employment rate
8.8
8.7
0
0
8.1
11.2
11.1
14
Unemployment rate
54.3
57
56.1
50.4
57
50.8
64
56.2
Source: Statistics Wales; October 2011
©Copyright Wavehill Ltd 2011
Page | 41
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 13: Net Birth of Enterprises 2007-2009 – net enterprise births and deaths is a strong but lagging indicator of economic activity. The graph clearly shows that the Welsh economy is losing businesses in 2009, was mixed in 2008, and creating businesses in 2007. Net business births and deaths by Local Authourity area 2007-2009 150 115 100
90 75
75 55
50
50
45
40
0 0
2007 net
-30 -50
-40
-40
2008 net 2009 net
-60 -90
-100
-90
-120 -150
-135 -150
-200
Source: Source: Statistics Wales; October 2011
ŠCopyright Wavehill Ltd 2011
Page | 42
On-going evaluation of the COASTAL project Report 2: October 2011
Appendix 2: Analysis of COASTAL project management data The following analysis is based on the project management data for COASTAL as at the end of the quarter ending the 30th June 2011.
Output and result definitions Output / results Participants
Definition The number of individuals participating in an ESF funded project. Participation should be linked to specific outcomes and require a meaningful level of engagement, for example a training course. Therefore, this excludes individuals attending conferences or individuals who simply receive information.
Employers assisted or The number of employers that receive assistance or financial support, financially supported through this Programme. Assistance: advice, guidance and information which can be delivered through the following media: face-to-face, telephone, web-based dialogue, conference, seminar, workshop, or networks (OffPAT, 2005, p.8). For example, provision of advice and guidance on managing health conditions for employers. Financial support: Receiving a grant or loan. For example, financial support for workforce development by employers. Participants qualifications
gaining The number of participants gaining a full, accredited qualification as a result of participation in an ESF-funded project.
Participants employment
entering The number of participants entering employment as a result of participation in an ESF-funded project. Employment includes self-employment and can be full-time or part-time. However, employment must involve a minimum of 16 hours work a week and must be paid employment. Projects must only report against this indicator if the participant enters employment within six months of completing provision. This indicator only applies to those participants who were not in employment upon commencement of their participation in an ESF-funded project.
ŠCopyright Wavehill Ltd 2011
Page | 43
On-going evaluation of the COASTAL project Report 2: October 2011
Output / results Definition Participants entering The number of participants entering further learning as a result of further learning participation in an ESF-funded project. Further learning can include formal academic or vocational education and less formal skills development training (including soft outcomes). For example, a young person classed as NEET (16-18 year olds not in education, employment or training) entering vocational training after receiving intensive careers advice funded by this Programme; or, an individual moving into formal training following outreach work funded by this Programme. Projects must only report against this indicator if the participant enters further learning within six months of completing provision. This indicator only applies to those participants who were not in education or training upon commencement of their participation in an ESF-funded project. Participants other outcomes
gaining The number of participants gaining intermediary outcomes as a result of positive participation in an ESF-funded project. Intermediary outcomes, such as completing courses, entering voluntary work, or attending a job interview, are those which are achieved as part of the journey to achieving final outcomes, such as entering paid employment or gaining qualifications.
Employers adopting or improving equality and diversity strategies and monitoring systems
The number of employers adopting a strategy, which outlines the key priorities for action by the employer and its staff to promote equality and diversity and challenge discrimination (GLA, 2005), and monitoring progress against these priorities. The equality strategies and monitoring systems must have been adopted or improved as a result of Structural Fund assistance or financial support.
ŠCopyright Wavehill Ltd 2011
Page | 44
On-going evaluation of the COASTAL project Report 2: October 2011
Analysis of Coastal participants Graph 14: Distribution of COASTAL participants by the L.A. in which they are resident
860
498 418 336
64
9
5
4
N=2,194 (please note total percentage doesn’t equate to 100 due to rounding)
Graph 14a: Distribution of COASTAL participants per project 25%
21% 18%
20% 16% 15% 10%
8%
8%
8%
10%
10%
5% 0%
0%
0%
N=2194
©Copyright Wavehill Ltd 2011
Page | 45
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 14b: Distribution of Coastal participants entering employment 0.80%
0.50% 0.50% 0.30% 0.20%
0.20% 0.10%
0%
0%
0%
N=2,194 Graph 15: Gender of participants
35% Male Female 65%
N=2,194
ŠCopyright Wavehill Ltd 2011
Page | 46
On-going evaluation of the COASTAL project Report 2: October 2011
Table 9: Gender of participants by project Project Swansea Prison Chaplaincy PRISM WGCADA Swansea Drugs Project Carmarthenshire CC C&C Swansea Neath Port Talbot CBC Pembrokeshire CC Ceredigion CC Bridgend CC TOTAL N=2,194
Male 178 124 132 136 135 281 202 248 0 0 1436
Female 0 61 45 78 85 187 150 152 0 0 758
Graph 16: Existing qualifications of participants 40%
36% 33%
35% 30% 25% 20%
17%
15% 10%
7% 4%
5%
3%
0% None
Below NQF 2 At NQF 2
At NQF 3
At NQF 4-6 At NQF 7-8
N=2,194 (please note total percentage doesn’t equate to 100 due to rounding)
©Copyright Wavehill Ltd 2011
Page | 47
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 17: Age group of participants 80% 68%
70% 60% 50% 40% 30%
24%
20% 7.5%
10%
0% 0% 15-24
25-54
55-64
65+
N=2,194 (please note total percentage doesn’t equate to 100 due to rounding)
Graph 18: Disability status of participants
47% 53%
Yes No
N=2,192
©Copyright Wavehill Ltd 2011
Page | 48
On-going evaluation of the COASTAL project Report 2: October 2011
Graph 19: Ethnicity of participants: BME – yes or no
2%
Yes No
98%
N=2,194
Graph 20: Employment status of participants 57%
60.00% 50.00% 40.00% 30.00%
23% 18%
20.00% 10.00%
0.80%
0.10%
1%
0.00%
N=2,194(please note total percentage doesn’t equate to 100 due to rounding)
©Copyright Wavehill Ltd 2011
Page | 49
On-going evaluation of the COASTAL project Report 2: October 2011
wavehill
ymchwil gwerthuso arolygon research evaluation surveys
Tel / Ffôn: 01545 571 711 Email / Ebost: wavehill@wavehill.com Website / Y We: www.wavehill.com Wavehill Ltd, 8 Water Street / 8 Heol y Dŵr, Aberaeron, Ceredigion, SA46 0DG
©Copyright Wavehill Ltd 2011
Page | 50