Ongoing evaluation of the Taith i Waith / New Work Connections project by Wavehill

Ongoing evaluation of the Taith i Waith / New Work Connections project Report 2: October 2012

Report prepared by: Endaf Griffiths Louise Petrie Karen Bourne Richard Brooks Simon Tanner Sarah Bulloch Any queries or questions about this report should be referred in the first instance to Endaf Griffiths at Wavehill: t: 01545 571711 | e: endaf.griffiths@wavehill.com Client contact: Melanie Evans, Regional Manager, New Work Connections t: 01745 353494 | e: melanie.evans@denbighshire.gov.uk Report version: FINAL

Evaluation of the New Work Connections Project | Report 2: October 2012

Contents Executive summary ...................................................................................................... 3 1. 2.

Introduction .......................................................................................................... 8 Overview of the project and the purpose of the evaluation ...................................... 9 2.1. 2.2. 2.3.

Analysis of project expenditure, outputs and results to date ...................................13 3.1. 3.2. 3.3.

Introduction ..................................................................................................................... 26 Overview of soft outcomes combined .............................................................................. 27 Variation in soft outcomes ............................................................................................... 28

Strengths, weaknesses and lessons learnt: management and delivery of the project.32 5.1. 5.2. 5.3. 5.4. 5.5. 5.6. 5.7. 5.8. 5.9. 5.10. 5.11. 5.12.

Introduction ..................................................................................................................... 13 Expenditure ..................................................................................................................... 14 Outputs and results .......................................................................................................... 16

Analysis of soft outcomes......................................................................................26 4.1. 4.2. 4.3.

Introduction ....................................................................................................................... 9 The project......................................................................................................................... 9 The evaluation ................................................................................................................. 12

Introduction ..................................................................................................................... 32 The type of support provided ........................................................................................... 33 Adding value to other provision ....................................................................................... 34 Working in partnership with other provision .................................................................... 34 Differences between the local authority areas ................................................................. 36 Perception of the objectives of the project and the support that it could provide ............ 37 Slow start and winding down period ................................................................................ 38 A long term process and continuity of support ................................................................. 39 Administration / paperwork / bureaucracy....................................................................... 39 The emphasis on ‘employment’ targets / performance indicators ................................ 41 The introduction of the Work Programme .................................................................... 42 The concern and influence of parents and guardians .................................................... 43

Case studies: the project at work ...........................................................................44 6.1. 6.2. 6.3. 6.4. 6.5.

Introduction ..................................................................................................................... 44 Anglesey .......................................................................................................................... 44 Gwynedd ......................................................................................................................... 46 Conwy .............................................................................................................................. 47 Denbighshire .................................................................................................................... 48

©Copyright Wavehill Ltd. 2012

Page | 1

Evaluation of the New Work Connections Project | Report 2: October 2012

Thematic analysis of participant interviews ............................................................51 7.1. 7.2. 7.3. 7.4. 7.5. 7.6. 7.7. 7.8. 7.9. 7.10. 7.11. 7.12. 7.13.

Review of literature on the link between improved confidence and employability ....76 8.1. 8.2. 8.3. 8.4.

Introduction ..................................................................................................................... 51 Background information about the participants interviewed ............................................ 52 The type of support participants had been receiving ........................................................ 54 Anticipated outcomes ...................................................................................................... 56 Perceived main benefits ................................................................................................... 57 Gaps in support ................................................................................................................ 64 Other support being accessed by participant.................................................................... 65 The main thing learnt which will help participants to get a job ......................................... 66 Distance from the labour market ..................................................................................... 67 Ways in which participants were helped to get a job .................................................... 69 Would participants recommend the project to others? ................................................ 71 Three words to describe the project ............................................................................. 72 Final comments made by interviewees. ........................................................................ 74

Introduction ..................................................................................................................... 76 Employer demands for skills and competencies ............................................................... 76 How improved confidence can improve the employability of an individual....................... 78 Conclusion ....................................................................................................................... 79

Emerging findings and recommendations ...............................................................80 9.1. 9.2. 9.3. 9.4.

Introduction ..................................................................................................................... 80 Expenditure, outputs and results to date.......................................................................... 80 Management and delivery of the project ......................................................................... 81 Outcomes of the support provided................................................................................... 83

Appendix 1: Local Authority level budget and targets ....................................................84

ÂŠCopyright Wavehill Ltd. 2012

Page | 2

Evaluation of the New Work Connections Project | Report 2: October 2012

Executive summary Taith i Waith / New Work Connections (hereafter referred to as NWC) is a £20million, four-year European funded project aimed at supporting people who, through a range of disadvantages, are economically inactive or unemployed. The project has been active since March 2010 and covers the four North Wales local authority areas of Anglesey, Conwy, Denbighshire and Gwynedd. This is the second report of the independent and ongoing evaluation of the project, which is being undertaken alongside its delivery over two years by social research and evaluation specialists Wavehill. Project expenditure, outputs and results to date This section analyses the project management data for NWC up to the end of May 2012. It includes an analysis of the progress of the project as a whole and of the activities in the four local authority areas within the project. The key findings are: •

• • • •

•

Taking into account the fact that claimed expenditure is likely to lag behind actual expenditure, due to the need to ensure that all the necessary audit evidence has been collected, claimed expenditure to date shows that under 30% of the approved budget has been utilised (£5,999,527). Given that the project is over half way into its 4-year lifetime, this is probably less than we would have anticipated although our experience shows that project expenditure often accelerates during the latter years. Expenditure has also been slower than anticipated due to delays in approving a re-profiling of expenditure and outputs / results on the part of WEFO. Expenditure to date is lower than forecast / profiled in each local authority area with the variance being highest in Anglesey (29% lower than profiled) and Denbighshire (32% lower) and lowest in Gwynedd (4% lower). The number of participants engaged to date is 44% of the target for the lifetime of the project (1,983/4,555); this is as per the latest profile submitted to WEFO. The project has however made limited progress to date in terms of a number of its other targets for outputs and results. The conversion rate (participants / results) is to date lower than will be necessary to achieve the targets set. For example, to achieve the target for the number of participants entering employment (896), a conversion rate of 20% is required (896/4,555). The current conversion rate is 4% (70/1,983). To date, the outputs and results claimed by the project have been largely dependent on those generated by two of the four local authority areas: Conwy and Denbighshire. The differences in the delivery model, emphasis and targets in each area needs to be taken into account (please refer to Report 1 of the evaluation and Appendix 1 of this report). However, the analysis shows that, at a ‘whole project level’, the outcomes of the project to date have been greater in those two areas. The characteristics of NWC participants illustrates the challenges faced by the project in terms of progressing participants towards and into employment. For example, 41% have no qualifications and 47% are economically inactive when entering the project.

©Copyright Wavehill Ltd. 2012

Page | 3

Evaluation of the New Work Connections Project | Report 2: October 2012

•

Neither the budget nor targets for the number of participants are evenly distributed amongst the Local Authority areas; whilst Gwynedd has 17% of the budget (total cost) of the project allocated to it, it has 27% of the target for the total number of participants engaged. The opposite is true in Conwy, which has 23% of the budget but 11% of the participants’ target. This leads to substantial differences in the forecast cost per participant in those areas; £9,968 in Conwy and £2,799 in Gwynedd. It is important to recognise that the approach in each local authority area is different which makes direct comparison of measures, such as cost per participant, difficult and possibly misleading. One possible explanation is however that local authorities have either been overoptimistic or under-optimistic in terms of what they have forecast / profiled can be achieved with the resources that are available. This is something that the evaluation will explore further as it progresses.

Soft outcomes Evaluation reports often refer to ‘soft outcomes’ and the ‘distance travelled’ by those being supported by a project. They represent a way of monitoring the benefits to participants in a project which conventional outputs such as the numbers gaining a qualification or employment (often referred to as ‘hard outputs’), may not capture. In the case of NWC, soft outcomes are being used to monitor how participants move towards a situation where they are ready to move into employment and the Work Star is being used to collect the necessary information from participants. Work Star is one of a series of ‘Outcome Stars’ developed by Triangle Consulting1. Analysis of the soft outcomes data collected found that: • • • •

Only a relatively small sub-group (255 individuals) of the total number of current participants have to date provided two sets of soft outcomes data. This significantly impacts upon the extent and nature of the analysis that is possible at this stage. For those participants for whom we have data, on average, the group’s soft outcomes as a whole improved by 12 percentage points, which is a statistically significant increase. 80% of those for whom we have data experienced positive outcomes on average across the seven dimensions of the Work Star. This leaves 20% of individuals who reported either no change or negative progress over the time of the intervention. Distance travelled varies considerably across the 255 individuals. It ranges from -1.9 to 5.71, indicating that there was a wide range of different experiences across the project.

Strengths, weaknesses and lessons learnt: management and delivery of the project With a particular emphasis on what was identified as strengths, weaknesses and lessons learnt, this section of the report discusses key aspects of the management and delivery of NWC discussed by staff when participating in ‘pause & review’ sessions held in each local authority area and on a regional level in July 2012. We also draw upon the discussions Wavehill researchers had with NWC staff during the days they spent ‘shadowing’ NWC staff in each local authority area during August 2012.

For further information please refer to: http://www.outcomesstar.org.uk/

©Copyright Wavehill Ltd. 2012

Page | 4

Evaluation of the New Work Connections Project | Report 2: October 2012

The key findings are: •

• •

•

• • •

•

The way in which support is provided (in particular, the ‘personal’ and ‘one-to-one’ nature of that support); and the opportunities the project offers to participants (training, workexperience, etc.) are seen as the key strengths of the project. This is also key to how the project ‘adds value’ to the other support available in the region. The skills, knowledge and experience of staff is considered to be critical to delivering the project, especially in terms of their ability to build relationships with and gain the confidence of the participants. As already noted, there are significant differences in the way NWC is delivered in each local authority area. Generally, this is considered to be a strength of the project due to the fact that it has allowed areas to build on previous provision and the structures that were in place in that particular area rather than impose a standard model across the region as a whole. But, it also leads to inconsistencies within the project and can be challenging to administer. One of the opportunities that the different structures and models that exist within the project provides is the opportunity to share experiences and learning and there are structures in place within NWC to facilitate just that. It is however apparent from the feedback provided via the pause & review sessions that there is a perception that there should be a greater emphasis on cooperation and sharing of ideas. Given the range of tools and mechanisms that are in place, this suggests that those mechanisms are not being utilised as effectively as they potentially could be and should, therefore, be reviewed. NWC works in close partnership with other providers working in the same area. The partnership approach is most obvious (and formalised) in Gwynedd where all ESF projects are located within the same part of the local authority and in Conwy where NWC is part of a wider programme branded as Let’s Get Working. The Regional Team developed a formal ‘agreement of outcomes’ document which can be used to set out how projects separate / ‘share’ outputs and results; something that was identified as a potential barrier to partnership working and an example of ‘good practice’ from within NWC. NWC management needs to take into account the fact that the project will inevitably experience a ‘winding down’ period as it moves towards the end of its lifetime unless there is a clear plan in place to continue the service beyond its current funding. A related issue which has been highlighted as being of concern to NWC staff is the fact that many of those being supported by NWC are being taken on a ‘journey’ towards work that will, for many, take a long time; longer that the lifetime of the NWC project in many cases. The individuals concerned are also often those furthest away from the labour market and change (more specifically, a withdrawal) in the support provided could lead to any progress that has been made being lost. The administrative burden on project staff was a constant theme in the ‘pause & review’ sessions and discussions with staff. Whilst administrative issues are probably inevitable within any project of the scales and scope of NWC, the fact that issues and problems are still being identified by staff suggests that the actions that have been undertaken to try and tackle the issue may not have been effective to date. It is however important to note that, in many cases, this is the first time the staff concerned have been involved in the delivery of an European funded project. Accordingly, they are not familiar with such an administration / monitoring process and are therefore uncomfortable with it. It is important to acknowledge that many of the changes to the administrative processes that have been introduced as the project has progressed are due to changes introduced by WEFO; for example, changes to the definitions of performance indicators. It is not, therefore, something that the project could control. Those changes have however been challenging from a project management perspective.

©Copyright Wavehill Ltd. 2012

Page | 5

Evaluation of the New Work Connections Project | Report 2: October 2012

•

The introduction of the Work Programme and the administrative work it generated provided an example of one of the benefits of having a regional team overseeing the delivery of NWC. The team led the discussions with WEFO on the matter on behalf of the four local authority areas and was able to disseminate advice and guidance across the region.

Interviews with participants and case study visits The aim of this element of the evaluation is to increase our understanding of the benefits that the individuals attribute to their participation in NWC. Telephone interviews were undertaken during June and July 2012 with 100 participants in the project selected at random from a database of participants who agreed to take part in the evaluation. The key findings of the interviews were: •

• •

•

Participants understand the focus within the project on getting people into work. But, the primary benefit of NWC apparent to interviewees was the ‘stability’ it brings to their lives and specifically the skills and opportunity to socialise and ‘get out of the house’. The confidence that participants develop as a result of their participation in the project was a common theme within interviews. It is apparent that the role of the NWC project worker / case worker / mentor is central to the project and was regularly cited as the main or best element of the NWC experience by participants. Analysis of the interviews demonstrates the wide range of participants that the projects supported; ranging from those who are relatively close to the labour market, to those who need a considerable amount of support before they could be considered to be ‘work ready’. This is further evidence of the fact that the project is working with some of those who are furthest away from the labour market and, hence, the most challenging in terms of achieving positive employment related outcomes.

Evaluation team researchers also visited each local authority area during August 2012 in order to meet with NWC staff informally, shadowing them as they undertook their work and met with participants. The case studies illustrate the ‘personal’ service that NWC participants receive. They also demonstrate the vital role played by the case officers / mentors and the value of the relationship that they develop with the participants. Review of literature on the link between improved confidence and employability There is reference throughout the report to the significant impact the support provided has on the confidence of participants. In order to explore this further we have conducted a literature review to identify whether there is evidence of how such increases in confidence may be contributing to improved employment prospects for these beneficiaries.

©Copyright Wavehill Ltd. 2012

Page | 6

Evaluation of the New Work Connections Project | Report 2: October 2012

That review highlights two key areas where such changes may result in positive outcomes for beneficiaries from their improved confidence. Firstly, related to the skills and competencies that potential employers may be looking for when recruiting staff (what may be termed as the ‘demandside’), and secondly, how improved confidence levels improves participants potential employability. Recommendations 1. It is recommended that a review of forecast / profiled project expenditure, outputs and results is undertaken with a view to ensuring that the targets set are realistic and achievable within the lifetime of the project. This review should take into account the fact that economic conditions continue to be challenging. The focus of the project on those who are furthest away from the labour market also needs to be taken into account and reviewed. If the conclusion is that the current targets are not achievable, two potential actions should be considered (in consultation with WEFO): a) Reducing the targets so that they are achievable and realistic whilst continuing to work with the same client group; and / or b) Devising a strategy whereby the project will target specific participants with a view to achieving the targets that have been set. 2. We support the work that is already ongoing to develop ideas, options, proposals and plans for a successor to the NWC with a view to minimising the ‘transition period’ between the current and subsequent project / support mechanism. This process should continue to be prioritised. 3. NWC management should review the mechanisms in place to facilitate networking and sharing of experiences amongst NWC staff across both on a region basis and on a local level. Where appropriate, developments to the NWC service provided in the individual Local Authority areas should be encouraged with a view to developing a more consistent approach across the region. 4. Building on the work that has already been undertaken to date and the recommendation above, NWC management should consider the potential to establish a working group that brings together staff from across the project to review the project administrative process and, if possible, identify areas where further improvements could be made without compromising the need to meet audit requirements as set out by WEFO. 5. NWC management should continue to take steps to ensure that soft outcomes (i.e. Work Star) data is collected from as many participants as possible so that the data can be fed into the evaluation process. 6. The evaluation should continue to undertake interviews with a sample of NWC participants that will explore both the social and employment outcomes of the support being provided. The evaluation will also continue to explore the link and relationship between ‘social’ and ‘employment’ outcomes.

©Copyright Wavehill Ltd. 2012

Page | 7

Evaluation of the New Work Connections Project | Report 2: October 2012

1. Introduction Taith i Waith / New Work Connections is a four-year European funded project aimed at supporting people who, through a range of disadvantages, are economically inactive or unemployed. The project has been active since March 2010 and covers the four North Wales local authority areas of Anglesey, Conwy, Denbighshire and Gwynedd. This is the second report of the independent and ongoing evaluation of the project, which is being undertaken alongside its delivery over two years by social research and evaluation specialists Wavehill. The report is set out as follows: • • • • • • • •

Section 2 provides a very brief overview of the Taith i Waith / New Work Connections project and the purpose of the evaluation; Section 3 analyses the progress made by the project in terms of its expenditure, outputs and results achieved to date; Section 4 analyses the data on the progress made by participants as recorded by the method being used to monitor ‘soft outcomes’ - Work Star; Section 5 reviews the management and delivery of the project to date drawing with a particular emphasis on the strengths and weaknesses identified by staff; Section 6 includes case study examples of the Taith i Waith / New Work Connections project in practice; Section 7 is thematic analysis of the comments made by 100 participants in the project during interviews with the evaluation team; Section 8 reviews some of the literature on the links between social or soft outcomes and employment outcomes; and Finally, Section 9 concludes the report by drawing together the key findings of the evaluation to date and making a number of recommendations for consideration by those managing the project.

It is important to stress that this is an interim report of an ongoing evaluation. The findings of this report should therefore be considered as being interim and all issues and matters discussed will be subject to further research and discussion as the evaluation progresses.

©Copyright Wavehill Ltd. 2012

Page | 8

Evaluation of the New Work Connections Project | Report 2: October 2012

2. Overview of the project and the purpose of the evaluation Section summary • • • •

Taith i Waith / New Work Connections is a £20 million four-year project aimed at supporting people who, through a range of disadvantages, are economically inactive or unemployed. The project is part-funded by the European Social Fund (ESF) programme in Wales and covers the four North Wales local authority areas of Anglesey, Conwy, Denbighshire and Gwynedd. The principal aim of the evaluation is to determine whether NWC has achieved its objectives, including its effectiveness, impact and sustainability. The work undertaken by the evaluation team to date includes analysis of the monitoring data collected, discussions with staff and interviews with 100 project participants.

2.1.

Introduction

This section very briefly provides the background and context for the remainder of the report by introducing the Taith i Waith / New Work Connections project together with the aims and objectives of the evaluation. For further detail, please refer to ‘Report 1’ of the evaluation which sets out the delivery structures for the project in greater detail including a review of the different delivery structures in each local authority area 2.

2.2.

The project

Taith i Waith / New Work Connections (hereafter referred to as NWC) is a European funded project aimed at supporting people who, through a range of project disadvantages, are economically inactive or unemployed. The project has been active since March 2010 and covers the four North Wales local authority areas of Anglesey, Conwy, Denbighshire and Gwynedd as shown on the map on the right. The services that local authority Social Services Departments provide do not normally include support to aid individuals to pursue training and employment and NWC was developed to address that gap.

The report is available here: http://issuu.com/wavehill/docs/wh_-_nwc_evaluation_-_report_1_-_final

©Copyright Wavehill Ltd. 2012

Page | 9

Evaluation of the New Work Connections Project | Report 2: October 2012

The eligibility criteria used for participation in the project is as follows: Individuals must not be: • •

Referred onto the Work Programme 3 A Work Choice participant 4

Individuals must be: • • •

Sixteen years old or over and living in the local authority area Economically inactive or unemployed Legally resident in the UK and able to take employment in a European Union Member State

Plus at least one from the list below: • • • • • • • • • • •

Facing significant challenges obtaining employment, education and training Have a work limiting health condition Aged 16-18 and not in education, employment or training (NEET) Have a physical disability or sensory impairment Have a learning disability or learning difficulty Have mental health needs Older people (50 or over) Lone parent with sole caring responsibility for a child/children Women returning to work Have caring responsibilities Black Minority Ethnic (BME)

The project team also reserves the right to use its own discretion to deal with individual cases. The following are described as ‘principles’ of eligible activity: • • • • •

Directly concerned with bringing the participant closer to the labour market Requires a meaningful level of engagement; that is not merely transfer of information or attendance at a conference Participant hours are recorded Participant outcomes are recorded Provision is not one which the government is statutorily committed to provide (whether or not such provision is available/accessible for the participant)

Whilst the delivery of the project varies between the four local authority areas concerned, the range of activities provided to participants by NWC includes: • • • •

Initial engagement – a single point of access to accept referrals into the project and register potential participants; Undertaking detailed assessments of individual participants and determining the most appropriate service based on these; Preparing individual education, training and employment plans; Providing intensive support regarding job finding and preparation for jobs;

A Department for Work and Pensions scheme that provides tailored support for claimants who need more help to undertake active and effective job-seeking. http://www.dwp.gov.uk/policy/welfare-reform/the-work-programme/ 4 A government supported employment programme designed specifically for people who due to their disability, may find it difficult to find or keep a job. http://www.direct.gov.uk/en/DisabledPeople/Employmentsupport/workschemesandprogrammes/DG_187696

©Copyright Wavehill Ltd. 2012

Page | 10

Evaluation of the New Work Connections Project | Report 2: October 2012

• • • • • •

Providing packages of support such as financial assistance or help with clothing and/or equipment; Providing expert advice and support on the financial benefits of being employed and dealing with concerns about moving from reliance on benefits; Supporting, mentoring and tracking each participant throughout his or her progress in the project, and maintaining regular contact; Supporting individuals at each step as they progress on a journey towards education, training and / or employment; Encouraging and supporting employers to provide more job opportunities for people from disadvantaged groups; and Encouraging and supporting employers to adopt or improve equality and diversity strategies and monitoring systems.

The business plan submitted to the Wales European Funding Office (WEFO)5 states that the project is innovative because: • • • • • •

This will be the first time that local authority Social Services Departments from North Wales have worked together in addressing the employment needs of disadvantaged groups; There will be much greater emphasis on helping older people and people who have a learning disability to move on to mainstream employment; The project wants to achieve a much higher profile for addressing employment requirements when people with social care needs become engaged with Social Services and partner agencies; There will be a more flexible, community based approach to learning/employment which will engage those excluded from mainstream provision; The project intends to recruit a number of older people as mentors to support young people with pursuing particular trades or employment; and The project will introduce a change of culture for many participants from one of dependency to one of sustainable personal development.

As shown in the table below, NWC is part funded by the European Social Fund (ESF) programme in Wales. Table 2.1: NWC project costs, ESF and match-funding Total Project Cost ESF funding Match funding

Funds £20,468,700 £10,747,678 £9,721,022

Source: NWC Regional Team

Sources of ‘match-funding’ for the NWC project consist of: • • •

funding provided by the four local authorities through their own direct service provision; funding brought to the project by service providers, as part of the procurement requirements; and the time of volunteers engaged in external service provision.

WEFO is part of the Welsh Government and manages the delivery of EU Structural Funds programmes in Wales. http://wefo.wales.gov.uk

©Copyright Wavehill Ltd. 2012

Page | 11

Evaluation of the New Work Connections Project | Report 2: October 2012

2.3.

The evaluation

The principal aim of the evaluation of NWC is to determine whether the project has achieved its objectives, including its effectiveness, impact and sustainability. The evaluation is split into two separate yet overlapping parts: • •

Process: How well is the project being managed and delivered? What, if anything, can be improved? What have been the main lessons learnt? Outcomes: What has the project achieved? What has been the impact of the project on the participants and employers engaged? Most importantly, has the project achieved its aims and objectives?

The methodology being used is based on the concept of viewing evaluation as a learning process and creating a strong link between the evaluation and activity – i.e. evaluation can have a direct and early effect on improving performance. The research undertaken for the preparation of this report has included: • • • • •

An analysis of the latest management and monitoring data for the project including expenditure, outputs and results; Analysis of the data captured using the ‘Work Star’ method for monitoring change in the individuals supported by the project (participants); Visits to each of the local authority areas by the evaluation team during which they shadowed and observed the work of NWC staff; A review of the findings of ‘pause and review’ sessions undertaken in each of the local authority areas and for the region as a whole; and Telephone interviews with a random sample of 100 project participants’ that explored a range of issues including the impact of the support provided.

©Copyright Wavehill Ltd. 2012

Page | 12

Evaluation of the New Work Connections Project | Report 2: October 2012

3. Analysis of project expenditure, outputs and results to date Section summary •

• • • • • •

•

Reported expenditure to date (May 2012) showed that under 30% of the approved budget has been utilised to date. Given that the project is over half way into its four-year lifetime, this is probably less than we would have anticipated although our experience shows that project expenditure often accelerates during the latter years. Expenditure to date is lower than forecast / profiled in each local authority area with the variance being highest in Anglesey (29%) and Denbighshire (32%) and lowest in Gwynedd (4%). The number of participants engaged to date is 44% of the target for the lifetime of the project; this is as per the latest profile submitted to WEFO. The project has however made limited progress to date in terms of a number of its other targets for outputs and results. The conversion rate (participants / results) is to date lower than will be necessary to achieve the targets set. The characteristics of NWC participants illustrates the challenges faced by the project in terms of progressing participants towards and into employment. For example, 41% have no qualifications and 47% are economically inactive. Neither the budget nor the target for the number of participants are evenly distributed amongst the local authority areas. Most notably, whilst Gwynedd has 17% of the budget (total cost) of the project allocated to it, it has 27% of the target for the total number of participants engaged. The opposite is true in Conwy which has 23% of the budget but 11% of the participants target. This leads to substantial differences in the cost per participant in those areas; £9,968 in Conwy and £2,799 in Gwynedd. To date, the outputs and results claimed by the project have been largely dependent on those generated by two of the four local authority areas: Conwy and Denbighshire.

3.1.

Introduction

This section analyses the project management data for NWC up to the end of May 2012. It includes an analysis of the progress of the project as a whole and of the activities in the four local authority areas within the project. The monitoring of European funded projects in Wales is focused on a set of outputs (activities) and results (that happen directly as a result of the outputs undertaken) recorded and reported to WEFO on a quarterly basis.

©Copyright Wavehill Ltd. 2012

Page | 13

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 3.1: NWC headline input, outputs and result performance indicators

Outputs

•£22million Expenditure (input)

•Participants engaged (economically inactive and unemployed) •Employers assisted or financially supported

•Participants entering employment •Participants gaining qualifications •Participants entering further learning •Participants gaining other positive outcomes •Participants receiving support with caring responsibilities •Employers adopting or improving equality and diversity strategies and monitoring systems

Results

3.2.

Expenditure

Expenditure within the project and its individual parts provides an indication of the level of activity ongoing within the project as a whole and in the individual local authority areas. It is however important to note that this is reported expenditure which (due to the need to ensure that the necessary audit evidence is in place before it is reported) can be slightly misleading. Nevertheless, it is an important indicator of progress. The following table noted NWC project expenditure for the period up to the end of May 2012. Table 3.1: NWC project expenditure; approved total project costs and expenditure to date Expenditure Total Project Cost ESF funding Match funding

Approved total project costs £20,468,700 £10,747,678 £9,721,022

Expenditure to date £5,999,527 £2,482,764 £ 3,516,762

% of approved expenditure utilised 29% 23% 36%

Source: NWC Regional Team

As shown, less than 30% of the approved budget has been utilised to date. Given that the project is over half way into its four-year lifetime, this is probably less than we would have anticipated although our experience shows that project expenditure often accelerates during the latter years. This is in fact reflected in the target set by the project for total expenditure to date: £7,545,506 (37% of the total approved project cost). Total expenditure to date is however 20% less than that target. In expenditure terms, the project is therefore underperforming suggesting that activity to date has been less than had been anticipated.

©Copyright Wavehill Ltd. 2012

Page | 14

Evaluation of the New Work Connections Project | Report 2: October 2012

It is important to note here that one of the factors that has contributed to this under-spend is the delay in approving the re-profiled expenditure forecasts submitted by the project to WEFO in July 20116 but not approved for almost a year. Under-spend continued to build during that period and this has contributed to the current position. Now that the re-profile has been approved, the Regional Team anticipate that the shortfall discussed here will reduce. The following table details expenditure to date (total) per local authority area. It shows that whilst the expenditure is lower than forecast in each area, the difference is considerably higher in Anglesey and Denbighshire. Table 3.2: NWC project expenditure per Local Authority; forecast vs. actual expenditure to date Expenditure Anglesey Gwynedd Conwy Denbighshire

Forecast expenditure to date £1,667,232 £1,494,146 £2,318,328 £2,065,799

Actual expenditure to date £1,191,099 £1,432,188 £1,970,798 £1,405,441

Difference 29% 4% 15% 32%

Source: NWC Regional Team

NWC has been subject to two re-profiling exercises firstly in February 2011 and then again in July 2011. The first reprofile was largely the result of delays in WEFO’s final approval of the project and the subsequent knock-on impact of the recruitment of new staff and so on. The primary cause of the second re-profile was changes to the context within which the project was being delivered including the recession and introduction of the Department for Work and Pension’s Work Programme. These re-profiling exercises affected both the project budget and the targets that had been set.

©Copyright Wavehill Ltd. 2012

Page | 15

Evaluation of the New Work Connections Project | Report 2: October 2012

3.3.

Outputs and results

3.3.1. Project as a whole The table below shows the progress made by the project to date in terms of the headline outputs and results. Table 3.3: NWC project headline performance indicators Target

Achieved to date*

% of target achieved

Total Participants (economically inactive and unemployed)

4,555

1,983

44%

Employers assisted or financially supported

896 2,553 907 4,056

70 123 27 1,441

8% 5% 3% 36%

290

Results

Outputs

Performance indicators

Participants entering employment Participants gaining qualifications Participants entering further learning Participants gaining other positive outcomes Participants receiving support with caring responsibilities Employers adopting or improving equality and diversity strategies and monitoring systems

Source: NWC Regional Team *claimed rather than actual7

In terms of the number of participants engaged, 44% of the target had been achieved as of the end of July 2012. This is higher than had been profiled to have been achieved to date which is a positive indication of progress. Project staff report that employers are being assisted on a regular basis. However, only three ‘employers assisted or financially supported’ have been claimed to date which is clearly of concern given that the project is over half way through its four-year lifetime. In terms of results, the best performing indicator is the number of participants gaining ‘other positive outcomes’ at 36% achieved. Very little progress has been made against the other result targets. In our experience as evaluators, projects often achieve a significant proportion of their results later in the lifetime of the project; participants need to be engaged and supported before results can be achieved. However, progress obviously needs to be closely monitored. With a view to giving the project the best possible opportunity to achieve its targets, we would suggest that the number of participants being engaged needs to reach the target number as soon as possible in the lifetime of the project in order to allow sufficient time in which to achieve the desired results.

As with expenditure, the outputs and results claimed will be lower than those actually achieved to date as there is a delay in reporting due to the need to ensure that all the necessary audit evidence to be able to claim the output or result is in place. ©Copyright Wavehill Ltd. 2012

Page | 16

Evaluation of the New Work Connections Project | Report 2: October 2012

Converting outputs into results We can explore the above issue further by considering the rate at which outputs (primarily in terms of the number of participants being engaged) are being ‘converted’ to results. Table 3.4(a) below shows the percentage of participants engaged that need to be ‘converted’ to results in order to achieve the targets set for NWC. Table 3.4(b) shows the rate at which the current number of participants have been converted to results. Table 3.4(a): NWC project headline performance indicator targets; results and percentage of outputs

Results

Outputs

Performance indicators Total Participants (economically inactive and unemployed) Employers assisted or financially supported Participants entering employment Participants gaining qualifications Participants entering further learning Participants gaining other positive outcomes Participants receiving support with caring responsibilities Employers adopting or improving equality and diversity strategies and monitoring systems

Target 4,555

Results as a % of outputs

60 896 2,553 907 4,056 290

20% 56% 20% 89% 6%

72%

Source: Analysis of data provided by the NWC Regional Team

Table 3.4(b): NWC project headline performance indicator claimed to date; results and percentage of outputs

Results

Outputs

Claimed 1,983

Results as a % of outputs

3 70 123 27 1,441 12

4% 6% 1% 73% 6%

Source: Analysis of data provided by the NWC Regional Team

The tables show that the conversion rate at the current time is lower than the percentage required to meet the targets set in all but one instance. Again, our previous experience suggests that these ‘conversion rates’ will increase as the project progresses / the longer participants are being actively supported by the project. However, it also underlines the need to engage participants in the project as soon as possible in order to allow adequate time for the results to be generated.

©Copyright Wavehill Ltd. 2012

Page | 17

Evaluation of the New Work Connections Project | Report 2: October 2012

Recruitment of participants The graph below plots the actual number of participants engaged by NWC against the target for the number of participants engaged. It shows that the number of participants actually engaged by the project was on or above 100% of the target set for each quarter until the December 2011 to February 2012 quarter when the number achieved fell to 96% of the target and the March 2012 to May 2012 quarter when it fell to 77% of the target. Figure 3.2: Actual number of participants vs. target number of participants 140.0

% of target achieved

120.0 100.0 80.0 60.0 40.0 20.0 0.0 Mar 10 May 10

Jun 10 Aug 10

Sept 10 - Dec 10 - Mar 11 Nov 10 Feb 11 May 11

Jun 11 Aug 11

Sept 11 - Dec 11 Nov 11 Feb 12

Mar 12 May 12

Source: NWC regional team

It is important to note that the analysis is somewhat misleading in that the quarterly targets have been revised to reflect the re-profiling exercise undertaken in July 2011 during which targets were revised to reflect the experience / performance of the project to date (as discussed in Report 1). However, the concern is that the number of participants engaged has been below the re-profiled target for two quarters in a row. As previously noted, the delay in approving the July 2011 re-profile will have contributed to this ‘slow down’. However, it is an issue that should continue to be monitored on an ongoing basis to ensure that the project ‘catches-up’ now that the re-profile has been approved.

Characteristics of participants The following is a very brief analysis of the key characteristics of the individuals being supported by NWC. The average (mean) age of NWC participants (as at the end of May 2012) was 40 with a range of 16 to 95. Of the 1,983 participants recorded: • •

24% (476) are over 50 years old; and 19% are under 25.

Just 1% (19) of participants are under 18 years of age.

©Copyright Wavehill Ltd. 2012

Page | 18

Evaluation of the New Work Connections Project | Report 2: October 2012

The fact that some participants are older than might be expected in a project of this nature is partly explained by the fact that older people are encouraged to participate in the project as a means of â&#x20AC;&#x2DC;mentoringâ&#x20AC;&#x2122; younger participants. In terms of gender, 55% (1,086) of participants to date were male and 45% female (897) In terms of employment status, there is a fairly even split between the percentage of participants who are economically inactive (48% - 957) and unemployed (52% - 1,025). As illustrated by the graphic below, the largest group of participants within the NWC project are those with no qualifications at the time they were engaged by the project. A small proportion of participants are however relatively highly qualified. Figure 3.3: The qualifications held by NWC participants 45%

41%

40% 35% 30%

26%

25% 18%

20% 15% 10%

0% None

Below NQF 2

NQF 2

NQF 3

NQF 4-6

NQF 7-8

N=1,983

Achieving results The following is an analysis of the data that is available on the participants who are recorded within the management data as having achieved positive results. It may be of interest to see who these individuals were (in terms of their key characteristics) in order to identify whether this result is more accessible to particular participants. Volunteering Currently, 169 out of the 1,983 individuals in the programme are registered as having obtained a volunteer placement. This represents 8.5% of the participants.

ÂŠCopyright Wavehill Ltd. 2012

Page | 19

Evaluation of the New Work Connections Project | Report 2: October 2012

Men and women were approximately evenly represented among those who gained volunteering opportunities (77 out of 169 individuals are women). Looking at individuals’ employment status prior to being engaged by NWC, the data shows an almost equal number of economically inactive individuals as unemployed individuals (48% and 52%, respectively). Examining the representation of these two groups within those who gained access to volunteering opportunities, we currently see an over-representation of those who had previously been economically inactive. These individuals represent 75% (129/169) of those who gained this hard outcome. Lone parents represent 9% of the current participants in the programme and they represent 7% (11 out of 169) of those who gained access to volunteering placements. This indicates that lone parents are not currently underrepresented in terms of accessing this hard outcome. Currently, 32% of the participants in the programme are registered as disabled, yet this group represent 61% of individuals who have gained access to volunteering placements (103 of 169). The over 65 year old group are, however, somewhat under-represented among the group who gained access to volunteer placements. They currently represent 5% of the overall participants in the programme, yet less than 1% (1 out of 169) of those who gained access to a volunteer placement. Nineteen per cent of individuals in the overall programme were between 16 and 24 years old at the start of the initiative, and this group also represent 20% (33 out of 169) of those who accessed volunteering placements. Interview opportunities Of the 1,983 participants in the project at the time of writing, 79 (4%) had achieved the ‘interview opportunity’ result. Among those who accessed the interview opportunity, the gender balance was equal, with 40 men and 39 women. Similarly, there is no evident discrepancy between those who were economically inactive prior to the programme and those who were registered unemployed in terms of their access to interview, with 46% (36/79) having been economically inactive and 54% having been unemployed. Currently, 9% of the participants of the programme are lone parents. Eleven out of the 79 individuals who gained interviews were lone parents, indicating that lone parents are well represented within this group. Thirty-two per cent of the participants overall are disabled whilst 44% (35/79) of those gaining an interview are disabled. This indicates that disabled people are not under-represented within those currently gaining access to this hard outcome. About 5% of the current participants are over 65 years of age, however none of these individuals are registered as having had any interviews during the course of the initiative. The other age categories are broadly proportionally represented amongst those having been offered an interview. Participants gaining qualifications Of the 1,983 participants in the project at the time of writing, 124 (6.3%) had achieved the ‘gained qualifications’ result. Among those who gained qualifications, there were, proportionally, slightly more women (70) than men (54). There is substantial discrepancy between those who were economically inactive prior to the programme and those who were registered unemployed in terms of their gaining qualifications, with 70.2% (87/124) having been economically inactive and 29.8% having been unemployed. Currently, 9% of the participants of the programme are lone parents. Twenty-three out of the 124 individuals (18.5%) who gained qualifications were lone parents, indicating that lone parents are slightly over-represented within this group. Thirty-two per cent of the participants overall are disabled whilst 55.6% (69/124) of those gaining qualifications are disabled. This indicates that disabled people are well represented within those currently achieving this hard outcome. About 5% of the current participants are over 65 years of age, however only one of these individuals is registered as having gained qualifications during the course of the initiative.

©Copyright Wavehill Ltd. 2012

Page | 20

Evaluation of the New Work Connections Project | Report 2: October 2012

Participants entering further learning Of the 1,983 participants in the project at the time of writing, 27 (1.4%) had entered further education. This is a very small number, so the following conclusions need to be treated with caution. Among those who entered further education, the gender balance was equal, with 14 men and 13 women. There is some discrepancy between those who were economically inactive prior to the programme and those who were registered unemployed in terms of their entering further education, with 21 out of the 27 having been economically inactive and 6 having been unemployed. Currently, 9% of the participants of the programme are lone parents. Eight out of the 27 individuals who entered further education were lone parents, indicating that lone parents are well represented within this group. Thirty-two per cent of the participants overall are disabled whilst 16 out of the 27 of those entering further education are disabled. This indicates that disabled people are not underrepresented within those currently achieving this hard outcome. About 5% of the current participants are over 65 years of age, however none of these individuals are registered as having entered further education during the course of the initiative. Participants entering employment Of the 1,983 participants in the project at the time of writing, 75 (3.8%) had entered employment. Slightly more women entered employment (43) than did men (32). There is no evident discrepancy between those who were economically inactive prior to the programme and those who were registered unemployed in terms of their entering employment, with 33 out of 75 having been economically inactive and 42 out of 75 having been unemployed. Currently, 9% of the participants of the programme are lone parents. Nineteen out of the 75 individuals who entered employment were lone parents, indicating that lone parents are very well represented within this group. Thirty-two per cent of the participants overall are disabled whilst 25.3% (19/75) of those entering employment are disabled. This indicates that disabled people are slightly under-represented within those currently achieving this hard outcome. About 5% of the current participants are over 65 years of age, however only one of these individuals is registered as having entered employment during the course of the initiative.

ÂŠCopyright Wavehill Ltd. 2012

Page | 21

Evaluation of the New Work Connections Project | Report 2: October 2012

3.3.2. Per local authority area analysis The first report of the evaluation discussed the differences in terms of how NWC was being delivered in the four local authority areas; that discussion is, therefore not repeated here. However, for ease of reference, the budget (i.e. total cost) and targets for individual local authority areas have been included in Appendix 1 of this report and table 3.5 below highlights some important differences between the areas in terms of budgets and outputs & results. Table 3.5: Total cost, participant numbers, and result conversion rates (CR) per local authority area.

Anglesey Gwynedd Conwy Denbighshire PROJECT OVERALL

Total cost (£)

Total participants

Cost per participant (£)

Entering employment CR

Gaining qualifications CR

Entering further learning CR

Other positive outcomes CR

5,184,643

1,229

4,219

20%

57%

18%

75%

3,431,992

1,226

2,799

19%

56%

21%

99%

4,784,861

480

9,968

20%

57%

18%

75%

7,067,205

1,620

4,362

19%

62%

15%

100%

20,468,700

4,555

4,494

20%

56%

20%

89%

Source: Analysis of data provided by the NWC regional team

As illustrated by the graph below, as well as in the table above, neither the budget nor the target for the number of participants are evenly distributed amongst the local authority areas. Most notably, whilst Gwynedd has 17% of the budget (total cost) of the project allocated to it, it has 27% of the target for the total number of participants engaged. The opposite is true in Conwy which has 23% of the budget but 11% of the participants target. This leads to substantial differences in the cost per participant in those areas; £9,968 in Conwy and £2,799 in Gwynedd. There are also differences in the ‘conversion rates’ that it is anticipated the project will achieve in the different areas. Figure 3.4: The distribution of the NWC budget (total cost) and target for the total number of participants per local authority as a percentage. Total cost

Total participants

40%

35% 36%

35% 30%

25% 27%

27% 23%

25% 20%

17%

15%

11%

10% 5% 0% Anglesea

Gwynedd

Conwy

Denbighshire

Source: Analysis of data provided by the NWC regional team

©Copyright Wavehill Ltd. 2012

Page | 22

Evaluation of the New Work Connections Project | Report 2: October 2012

The obvious question is whether these difference in cost per participant in the different areas is justified. This is however a difficult (and complicated) question to answer as each area has developed its NWC project / provision based on, and to add-value to, the structures that already existed pre-NWC in those areas. As discussed in Report 1, the approach in each area is therefore different. For example, unlike the other areas, NWC in Gwynedd is focused exclusively on individuals with predominantly mental health problems. This makes direct comparison of measures such as cost per participant difficult. One possible explanation is however that local authorities have either been over-optimistic or under-optimistic in terms of what they have forecast / profiled can be achieved with the resources that are available. This is something that the evaluation will explore further as it progresses.

Engaging participants The following graphic plots the figures for the number of participants engaged as compared to the target / profile in percentage terms for the five most recent quarters split by the four local authority areas. If all was going as planned / forecast, the lines for the individual local authority areas would follow the horizontal 100% line highlighted by the red circle. This would mean that the number of participants being engaged was as profiled / forecast. Again, the graph is somewhat misleading as it is based on the re-profiled targets. But, the difference in ‘performance’ in the four areas (in terms of the number of participants being engaged as compared to the set profile / target) is clear with Conwy and Denbighshire’s numbers being significantly higher than profiled (the 100% line) and Gwynedd and Anglesey being well below the profile suggesting that the recent ‘under-performance’ is most apparent in those two areas. Figure 3.5: Actual number of participants vs. target number of participants split by local authority area 350.0 300.0 250.0 Gwynedd

200.0

Conwy

150.0

Anglesey 100.0

Denbighsire

50.0 0.0 Mar 11 - May Jun 11 - Aug Sept 11 - Nov Dec 11 - Feb Mar 12 - May 11 11 11 12 12 Source: NWC regional team

Again, the fact that the project differs in each area needs to be taken into account. But, given that the profiles were set by the local authorities themselves, the variance in ‘performance’ needs to be considered. Is there a reason why two areas are apparently over performing and two are underperforming in terms of the targets set for participant engagement? Are there any lessons learnt that can potentially be shared between the areas? ©Copyright Wavehill Ltd. 2012

Page | 23

Evaluation of the New Work Connections Project | Report 2: October 2012

Outputs and results The following graphs analyse some of the key outputs and results for the project to date. They show that, to date, the outputs and results claimed by the project have been largely dependent on those generated by two of the four local authority areas: Conwy and Denbighshire. For example, as shown in the graph below, to date, 68% of the NWC participants gaining a qualification and 40% of those gaining employment have been located in Conwy. Again, the differences in the delivery model, emphasis and targets in each area needs to be taken into account (please refer to Report 1 and Appendix 1). However, the analysis shows that, at a ‘whole project level’ the outcomes of the project to date have been greater in two of the four areas. Figure 3.6: Participants gaining qualifications per local authority area as a percentage of the total claimed by the project to date 80% 68%

70% 60% 50% 40%

29%

30% 20% 10%

0% Gwynedd

Conwy

Anglesey

Denbighshire

N=85

Figure 3.7: Participants entering employment per local authority area as a percentage of the total claimed by the project to date 60% 48%

50% 40% 40% 30% 20% 11% 10% 0% 0% Gwynedd

Conwy

Anglesey

Denbighshire

N=62

©Copyright Wavehill Ltd. 2012

Page | 24

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 3.8: Participants entering further learning per local authority area as a percentage of the total claimed by the project to date 50%

44%

45% 40%

33%

35% 30% 25% 20%

17%

15% 10%

5% 0% Gwynedd

Conwy

Anglesey

Denbighshire

N=18

Figure 3.9: Participants gaining other positive outcomes per local authority area as a percentage of the total claimed by the project to date 70% 62% 60% 50% 40% 30%

23%

20% 10%

10% 0% Gwynedd

Conwy

Anglesey

Denbighshire

N=1,332

ÂŠCopyright Wavehill Ltd. 2012

Page | 25

Evaluation of the New Work Connections Project | Report 2: October 2012

4. Analysis of soft outcomes Section summary • • • • •

Soft outcomes are being used to monitor how participant’s progress towards a situation where they are ready to move into employment and The Work Star is being used to collect the necessary information from participants. Only a relatively small sub-group (255 individuals) of the total number of current participants have to date provided two sets of soft outcomes data. This significantly impacts upon the extent and nature of the analysis that is possible at this stage. For those participants for whom we have data, on average, the group’s soft outcomes as a whole improved by 12 percentage points, which is a statistically significant increase. 80% of those for whom we have data experienced positive outcomes on average across the seven dimensions of the Work Star. This leaves 20% of individuals who reported either no change or negative progress over the time of the intervention. Distance travelled varies considerably across the 255 individuals. It ranges from -1.9 to 5.71, indicating that there were a wide range of different experiences across the project.

4.1.

Introduction

Evaluation reports often refer to ‘soft outcomes’ and the ‘distance travelled’ by those being supported by a project. They represent a way of monitoring the benefits to participants in a project which conventional outputs such as the numbers gaining a qualification or employment (often referred to as ‘hard outputs’), may not capture. In the case of NWC, soft outcomes are being used to monitor how participant’s progress towards a situation where they are ready to move into employment and the Work Star is being used to collect the necessary information from participants. Work Star is one of a series of ‘Outcome Stars’ developed by Triangle Consulting 8. The seven key areas of development that the Work Star measures are: • • • • • •

Job-specific skills Aspirations and Motivations Job-search skills Stability (this measures day-to-day stability of an individual’s life and takes into account factors like debt, alcohol or drug use and housing issues) Basic skills (e.g. literacy, English, numeracy) Social skills for work (working with others, self-confidence, timeliness, work attire)

Along these seven key dimensions, the Work Star sets out five stages within a ‘journey into work’ that an individual will take if and when they progress. Each dimension is measured on a 10 point scale:

For further information please refer to: http://www.outcomesstar.org.uk/

©Copyright Wavehill Ltd. 2012

Page | 26

Evaluation of the New Work Connections Project | Report 2: October 2012 a) b) c) d) e)

Not thinking about work (represented by a score of 1-2 on the Star): substantial barriers to work or advancement and you can’t see a way around them ; Thinking about work (score of 3-4): substantial barriers but you are working out how to address them; Making progress (score of 5-6): some barriers overcome but others are still there; Work-ready with support (score of 7-8): barriers mostly overcome or can be worked around; you need support to maintain progress; Self-reliance (score of 9-10): in work, work-ready, or engaged in work-related training. By then you don’t need support from the programme.

As part of their monitoring procedures, the NWC teams are tracking participants as they move through these stages on the seven key areas and are entering that data into the monitoring database. The evaluation will use that information, alongside the other monitoring data being recorded, as part of the process of measuring the impact of the project. Only a relatively small sub-group (255 individuals) of the total number of current participants have provided soft outcomes data into the system to date. This significantly impacts upon the extent and nature of the evaluation that is possible. For example, it is not advisable, with this relatively small sample, to make use of inferential statistics that examine associations between, for example, hard and soft outcomes, whilst taking into account multiple characteristics of individuals. Instead, at this stage the analysis is focused on the 255 individuals that provided responses at this point in the evaluation, rather than trying to extrapolate their progress to participants overall. This section looks at the start points and distance travelled across all the seven dimensions of the Work Star together, as well as in relation to each of the dimensions separately. It then seeks to identify the characteristics of groups making negative progress or no progress and those making positive progress.

4.2.

Overview of soft outcomes combined

Table 4.1 shows the average score across the seven soft skills dimensions and each dimension separately for the start of the initiative, the current point in the evaluation and, as an extension of this, the average distance travelled for the group of 255 individuals so far. Remembering that the soft outcomes are measured on a scale of 1-10, an increase of 1 score represents an increase of 10 percentage points. On average, the group’s soft outcomes as a whole improved by 12 percentage points, which is a significant increase when tested using a paired samples t-test 9. Looking at the dimensions separately, the data indicates that this group entered the programme with relatively high basic skills and social skills, and with relatively low job search skills and abilities to overcome challenges. Interestingly, looking at the distance travelled, progress was highest in relation to job skills, job search skills and motivation, and lowest in relation to basic skills and stability scores. This indicates that the programme is, on average, addressing the areas that are least developed on entry into the initiative. The data shows that positive change was made, on average, across all the seven dimensions of soft outcomes. Paired samples t-tests confirm that this positive difference is statistically significant.

A paired-samples t-test tests the difference between two scores an individual gives and tells us whether, on average, the group examined had significantly different scores between the two time points. The test requires the data to be normally distributed, which it is. ©Copyright Wavehill Ltd. 2012

Page | 27

Evaluation of the New Work Connections Project | Report 2: October 2012

Table 4.1: The average score across the seven soft skills dimensions and each dimension separately for the start of the initiative, the current point in the evaluation and the average distance travelled

All seven dimensions Motivation Social Skills Challenges Job Skills Job Search Stability Basic Skills

Start point 5.7 5.6 6.2 5.2 5.3 5.1 6.1 6.4

Most recent 6.9 7.0 7.4 6.5 6.8 6.5 6.9 7.2

Average distance travelled 1.2 1.4 1.1 1.3 1.5 1.4 0.8 0.8

A number of the tables that follow below show the average distance travelled over the seven combined dimensions (on a scale of 1-10) but comparing particular groups of participants. Overall, the data suggests that the more ‘vulnerable groups’ (disabled, over 65s, lone parents) are making as much, if not slightly more progress than the ‘less vulnerable’ groups. A definitive test of these differences is not advisable at this stage given the limitations of the data. This is however again something that we would hope to address as the evaluation progresses.

4.3.

Variation in soft outcomes

The above analysis of soft outcomes examines the average across the 255 individuals for whom Work Star data is available. Concentrating on the average provides a sense of ‘most people’s experiences’ but it does not allow for reflections on those who differ from ‘most people’. The next section of the analysis recognises that progress across the soft outcomes varied around the average. It divides the group of 255 individuals into two groups: those for whom we see positive outcomes and those for whom we see no change in soft outcomes or negative change. Distance travelled varies considerably across the 255 individuals. It ranges from -1.9 to 5.71, indicating that there were a wide range of different experiences across the project. Table 4.2 and figure 4.1 below show the distribution of distance travelled across the 255 individuals Table 4.2: Distance travelled across the seven dimensions of the Work Star; frequency and percentage. Distance travelled 10 progress of -2 to -1.5 progress of -1.49 to -1 progress of -0.99 to -0.5 progress of -0.49 to 0 progress of 0.1 to 0.5 progress of 0.51 to 1 progress of 1.1 to 1.5

Number of individuals 2 7 7 35 26 50 42

Percentage of 255 8 2.8 2.8 13.9 10.4 19.6 16.5

This is calculated by summing the participant’s scores on all the seven Work Star dimensions and then dividing this by 7 in order to return the answer to the original scale (1-10).

©Copyright Wavehill Ltd. 2012

Page | 28

Evaluation of the New Work Connections Project | Report 2: October 2012

Number of individuals 29 17 23 4 4 2 5 1 1 255

Distance travelled 11 progress of 1.5 to 2 progress of 2.1 to 2.5 progress of 2.5 to 3 progress of 3.1 to 3.5 progress of 3.5 to 4 progress of 4.1 to 4.5 progress of 4.5 to 5 progress of 5.1 to 5.5 progress of 5.5 to 6 Total

Percentage of 255 11.6 6.8 9.2 1.6 1.6 0.8 2 0.4 0.4 100

Figure 4.1: Distance travelled across the seven dimensions of the Work Star; frequency. 60

42 35

23 17

20 10

Grouping individuals who reported no soft skills progress or negative progress, and comparing them to those individuals who reported positive progress, we see that 204 of the 255 individuals (80%) for whom we have data experienced positive outcomes on average across the seven dimensions. This leaves 20% of individuals who reported either no change or negative progress over the time of the intervention. The tables that follow examine the characteristics of these two groups. In summary, they show that a lack of progress in terms of soft outcomes does not seem to be related with gender, age, disability, lone parenthood or previous employment status. Or, in other words, the analysis has not found anything to suggest that NWC is having less / no impact on a particular group / cohort of individuals.

This is calculated by summing the participantâ&#x20AC;&#x2122;s scores on all the seven Work Star dimensions and then dividing this by 7 in order to return the answer to the original scale (1-10).

ÂŠCopyright Wavehill Ltd. 2012

Page | 29

Evaluation of the New Work Connections Project | Report 2: October 2012

Table 4.3: Gender and progress

Negative or no progress Positive progress Total

Count % Count % Count %

Female 21 42% 91 47% 112 48%

Male 29 58% 102 53% 131 56%

Total 50 100% 193 100% 234 100%

The table above indicates that men and women are largely equally represented within the group that made no or negative progress, as they are in the group that made positive progress. Table 4.4: Employment status on entry to programme and progress Employment Status on entry to programme Economically Long term Unemployed Total inactive unemployed 37 5 8 50 Count Negative or no progress 74% 10% 16% 100% % 146 18 29 193 Count Positive progress 76% 9% 15% 100% % 183 23 37 243 Count Total 75% 9% 15% 100% % As shown in the table above, the presence of progress is also not associated to previous employment status. Similar proportions of both the group that made progress and those that did not are represented in each of the employment status categories. Table 4.5: Parental status and progress

Negative or no progress Positive progress

Total

Count % Count % Count %

Parent status Not lone parent Lone parent 45 5 90% 10% 178 15 92% 8% 223 20 92% 8%

Total 50 100% 193 100% 243 100%

Again, the table above shows that progress is not associated to previous lone parent status. Similar proportions of both the group that made progress and those that did not are represented in each of the parental categories.

ÂŠCopyright Wavehill Ltd. 2012

Page | 30

Evaluation of the New Work Connections Project | Report 2: October 2012

Table 4.6: Age and progress

Negative or no progress Positive progress

Total

Count % Count % Count %

16-24 9 18% 44 23% 53 22%

25-54 35 70% 129 67% 164 67%

Age Range 55-64 6 12% 17 9% 23 9%

>65 0 0% 3 2% 3 1%

Total 50 100% 193 100% 243 100%

The presence of progress is also not associated to age. Similar proportions of both the group that made progress and those that did not are represented in each of the age groups. Table 4.7: Disability and progress

Negative or no progress Positive progress

Total

Count % Count % Count %

Not disabled 21 42% 75 39% 96 40%

Disabled Disabled 29 58% 118 61% 147 61%

Total 50 100% 193 100% 243 100%

The presence of progress is also not associated to disability.

ÂŠCopyright Wavehill Ltd. 2012

Page | 31

Evaluation of the New Work Connections Project | Report 2: October 2012

5. Strengths, weaknesses and lessons learnt: management and delivery of the project Section summary •

• •

•

• • • •

5.1.

Introduction

With a particular emphasis on what was identified as strengths, weaknesses and lessons learnt, this section of the report discusses key aspects of the management and delivery of NWC discussed by staff when participating in ‘pause & review’ sessions held in each local authority area and on a regional level in July 2012. We also draw upon the discussions Wavehill researchers had with NWC staff during the days they spent ‘shadowing’ NWC staff in each local authority area during August 2012.

©Copyright Wavehill Ltd. 2012

Page | 32

Evaluation of the New Work Connections Project | Report 2: October 2012

5.2.

The type of support provided

Much of the discussion about the strengths of NWC as a project focused on: • •

The way in which support is provided (in particular, the ‘personal’ and ‘one-to-one’ nature of the support being provided); and The opportunities that NWC offers to participants (training, work-experience, etc.) and especially the fact that those opportunities are made available to those who are unlikely to be offered those same opportunities from other sources.

The following are typical examples of the comments made: •

“The biggest thing I see is someone coming in who is wary/closed off and after a while you can see a difference as you then see them interacting, they can sit and talk to people, we can sit and talk to them, make them believe they can do it and when they do they are so proud. Their voice counts and we give them the belief to achieve what they want. We have small enough groups to give people time, to make sure we get to know people and can build trust and they can then come to us with any problems. We come in at the same level and build their trust.”

•

“Some participants are wary at first and embarrassed because they are living in poverty but we build trust and it ends up as a good service. I see some once a week, others once a fortnight or once a month.”

•

“The individual one-to-one support people have, the feedback we’ve had is that they don’t get it anywhere else. In the Job Centre you are just a number and it’s not personal. Here it is all confidence building and motivational and they feel like somebody cares. The funding for training we’ve got, a lot of projects haven’t got this.”

The skills, knowledge and experience of staff is considered to be critical to delivering the project, especially in terms of their ability to build relationships with and gain the confidence of the participants. This was something which was also very apparent when Wavehill researchers ‘shadowed’ NWC staff as they met with participants; a significant amount of time and effort goes into building a relationship and level of trust with participants (please refer to Section 6). It was also a key finding of the interviews with participants (Section 7). This however obviously comes at a cost and the question that the evaluation will seek to answer as it progresses is whether the benefit generated justifies the cost. It is also important to acknowledge that any project that is dependent to any significant extent on the knowledge, experience and skills of its staff, is at risk; what if those staff leave or become unavailable? This risk is especially apparent in European projects which have a limited lifetime and, accordingly, run the risk of staff leaving as the project draws to a close. Our experience also shows that projects can find it difficult to replace staff that leave part-way through the delivery of a project due to the limited length of contract which they can offer potential employees. This issue discussed further later in this section. One of the weaknesses identified during the ‘pause & review’ sessions was that ESF funding cannot be used to provide training for project staff; this has clearly been a significant frustration for NWC management who would have liked to use project funding to fund training for staff.

©Copyright Wavehill Ltd. 2012

Page | 33

Evaluation of the New Work Connections Project | Report 2: October 2012

This has however been deemed as being ineligible by WEFO 12. Essentially this has restricted the project’s ability to build upon the skills and abilities of their staff, increasing their dependence on the skills and abilities within current staff and exacerbating the associated risks.

5.3.

Adding value to other provision

One of the key questions for any European funded project is how does the support or service provided add value to other provision? The main thing NWC staff referred to when discussing this issue was again the depth / intensity and range of support that the project can provide to participants, the following comments being typical: •

“It is really hands on with us and we can work with a broader range of people and we’ve got more money to use than others. They can do a wider variety of things here and we give them space to explore. One lad who was written off in the day centre is now one of our best workers. We have flexibility, we’ll meet participants where it suits them.”

•

“If we were not here social workers and CPN's would have no-one to refer to, so with the amount of caseloads they have, they would have no time to offer help with courses or voluntary placements... There is stuff around but participants need support to attend so if NWC goes I don't know what will happen. There are support workers under Social Services but their role is more about taking participants to shop and do banking, or to hospital appointments, etc.”

Staff also point to the fact that NWC adds value to other projects by offering services to those considered as being ‘hard to reach’ and those who are furthest from the labour market. This however leads to an element of ‘conflict’ within the project; what should be the priority of the project – to work with those who are furthest away from the labour market and most in need of support or to work with those who are more likely to help the project achieve its targets? This question is discussed further later in this section.

5.4.

Working in partnership with other provision

It is noticeable that one of the key ways in which NWC ensures that it complements and adds value to other provision is by seeking to work in partnership with other providers, projects and schemes. Indeed this was broadly identified as one of the strengths of the project. The manner of the cooperation however differs from area to area which is a reflection of the different approaches taken in each area (please refer to Section 5.5). Different local authority areas have different approaches to partnership working with Gwynedd and Conwy offering examples of a more formal approach. In Gwynedd, all ESF projects sit within the same local authority service in order to encourage an integrated approach. Further, the broader work experience and link with employers on the skills shortage element of the project is within the remit of the same service. In Conwy, NWC is part of a wider programme branded as Let’s Get Working. As well as NWC this programme also includes Genesis Wales (Welsh Government)13, the Engagement Gateway (WCVA) 14, and Families First (Welsh Government) 15. 12

ESF expenditure guidance states that staff training costs can be claimed only if an individual has specific needs for training that could not have been identified before the project began. This needs to relate to the acquiring of specialist knowledge. 13

http://wales.gov.uk/topics/childrenyoungpeople/parenting/help/genesiswales2/?lang=en http://www.wcva.org.uk/europe/index.cfm?sub=4&display_sitetextid=226 15 http://wales.gov.uk/topics/childrenyoungpeople/parenting/help/familiesfirst/;jsessionid=340qPwxGhrP5ShrknQMRg9j6LKJRTPNm1QLR 099ZNqQgrZJhGMHg!639923638?lang=en 14

©Copyright Wavehill Ltd. 2012

Page | 34

Evaluation of the New Work Connections Project | Report 2: October 2012

The graphic below (Figure 5.1), developed by the programme, illustrates how the elements of the programme complement one another, and allow a participant to enter the journey towards employment at whichever stage is appropriate to their individual needs. Figure 5.1: The Genesis Wales, Engagement Gateway and NWC journey in Conwy

The approach is less formalised in other areas but it is apparent that there are links in place with other projects. •

“We work well as a team, we have good communication and strong links with Job Centre Plus and Social Services, we also get a few referrals from GP’s and the pain clinic. People can self-refer downstairs in the IT suite and we can help them with CV’s, work skills and IT skills”

This relationship with other projects and schemes is important in a situation where there are a range of other projects potentially working with the same client group as NWC. To illustrate this point, the table that follows lists projects funded by WEFO from the same measure and theme as NWC (as of 1st October 2012). The feedback from NWC staff is that the project has working relationships with these and other projects working in the same area and field. The Regional Team have also developed a formal ‘agreement of outcomes’ document which can be used to set out how projects are separate going to ‘share’ outputs and results; something that was identified as a potential barrier to partnership working and an example of ‘good practice’ from within NWC.

©Copyright Wavehill Ltd. 2012

Page | 35

Evaluation of the New Work Connections Project | Report 2: October 2012

Table 5.1: ESF, Priority 2, Theme 1 projects active in North Wales Project Gateway BME Employment For All Want2Work Connect All Age Skillbuild for the Unemployed Inactive and NEET ReAct ESF Convergence P2 STRIDES Alliance Partnership New Work Connections Genesis Cymru Wales 2 Convergence Princes Trust Enterprise Programme Conv P2 WCVA ILM Convergence Careers Information Advice and Guidance CIAG The Peer Mentoring Programme Convergence Priority 2 Increasing BME Employment, Tackling BME Economic Inactivity New Day Convergence P2

Sponsor Wales Council for Voluntary Action All Wales Ethnic Minority Association Jobcentre Plus WG DCELLS Lifelong Learning and Skills WG DCELLS Lifelong Learning and Skills New Sandfields Aberavon Denbighshire County Borough Council WG DCELLS Lifelong Learning and Skills The Princes Trust Wales Council for Voluntary Action Careers Wales Association WAG Social Justice SOVA National Offenders Management Services

Source: WEFO website

5.5.

Differences between the local authority areas

As previously noted, there are significant differences in the way NWC is delivered in each local authority area 16. Generally, NWC staff consider this to be a strength of the project due to the fact that it has allowed areas to build on previous provision and the structures that were in place in that particular area rather than impose a standard model across the region as a whole. The logic of this argument is clear; imposing a standard delivery model across a region would clearly be difficult. However, the variance within the project clearly caused some difficulty in terms of managing the project. For example, there is inconsistency in the services and support available to participants across the region and, on a practical level, it is apparent that rolling out a consistent monitoring system has been challenging. The evaluation will consider the impact of these differences as it progresses by, for example, considering whether the approach /model in one area is generating ‘better’ or different results to that in another area. The review of outputs and results in the previous section is however already starting to identify differences between the local authority areas. One of the major potential opportunities that the different structures and models that exist within the project provides is the opportunity to share experiences and learning. There are structures in place within NWC to facilitate just that; the Regional Team has introduced a number of mechanism for sharing information and set up working groups that bring together staff from the different areas.

Details of the differences between the Local Authority areas are set out in detail in Report 1 of the evaluation, available here: http://issuu.com/wavehill/docs/wh_-_nwc_evaluation_-_report_1_-_final

©Copyright Wavehill Ltd. 2012

Page | 36

Evaluation of the New Work Connections Project | Report 2: October 2012

For example: • • • • • • • •

Regional Newsletter Regional Update Report (monthly) Regional Progress Report (quarterly) ‘Top tips’ are provided to the teams ‘Practitioners Group’ set up which is a group of staff (fluid) who meet on a quarterly basis to share their work Desktop processes are gathered from each authority and shared WEFO updates are shared with the Local Managers who are expected to communicate them to their project staff A Regional website is in development and each authority is given their own section to highlight their services

It is however apparent from the feedback provided via the pause & review meetings that there is a perception that there should be a greater emphasis on cooperation and sharing of ideas as the project progresses and especially as it moves towards the ‘exit strategy’ phase. Given the range of tools and mechanisms that are in place to encourage communication and cooperation across the region, this suggests that those mechanisms are not being utilised as effectively as they potentially could be.

5.6.

Perception of the objectives of the project and the support that it could provide

Being clear about the objectives and purpose of NWC was identified as a key lesson learnt during the pause & review sessions. In particular, it is apparent that a few stakeholders in some areas (primarily those outside the project but also some within the project) were perceived as being unclear about the focus on moving people towards and into employment believing that NWC was a continuation of previous social service provision. The ‘lesson learnt’ identified by staff was a need for more ‘internal’ marketing and promotion of the project within local authorities and stakeholders within the sector in general. •

“People assumed that NWC was there just to provide additional activities for people supported by Social Services; something for them to do on a Friday. This is a misunderstanding of what the project is trying to achieve.”

This is an important issue, as one of the objectives of NWC is to ‘modernise’ the services being provided by introducing a greater focus on progressing participants towards and into employment. The fact that such a misconception exists in some quarters is therefore not surprising. It is however something that needs to be monitored as the project progresses to assess whether views (and understanding) of the project have changed. This is something that the evaluation will consider as it progresses by interviewing stakeholders directly and indirectly involved with NWC. One of the factors identified by the discussion as to what had led to any misconception was that there was a ‘rush’ to get the project up and running once it had been approved by WEFO in order to make up for the time that had been lost awaiting approval of the project. One of the consequences of this was that, in order to recruit participants quickly, the focus was on engaging with existing provision and services without necessarily firstly effectively introducing the aims and objectives of NWC. This ‘rush’ to get up and running quickly is discussed further below.

©Copyright Wavehill Ltd. 2012

Page | 37

Evaluation of the New Work Connections Project | Report 2: October 2012

5.7.

Slow start and winding down period

In our experience as evaluators, a late or slow start is common amongst European funded projects and NWC was not an exception. Largely due to delays in the approval of the project by WEFO, the project started in the region of a year later than envisaged within the original business plan. One of the consequence of this, as noted above, was (due to a large extent to pressure from WEFO) a rush to get the project up and running and participants engaged as soon as possible once the project had been approved. This had a number of consequences one of which has already been discussed in the section above. Another was that the project was running (i.e. participants were being engaged) before the structures and processes for the administration of the project were fully in place. The result was that there were administrative ‘errors’ which caused problems further down the line which staff were required to address. This has clearly been a frustration to staff and is an important lesson learnt. A pressure to get projects up and running as soon as possible is understandable (funders and others want to see progress being made). However, a rush to progress before the necessary administrative systems and procedures are in place will cause difficulty (and lead to additional work) in the medium to long term. It is also apparent that NWC management needs to take into account the fact that the project will inevitably experience a ‘winding down’ period as it moves towards the end of its lifetime unless there is a clear plan in place to continue the service beyond its current funding. One of the reasons for this is that staff will inevitably look to move on as the end date for a project (and their contract) approaches. The result of this, in addition to the slow start, is that the period during which the project is ‘fully active’ and most likely to achieve its objectives is considerably ‘squeezed’ into the middle portion of the project’s lifetime as illustrated below. Figure 5.2: The squeezed delivery period of projects

Source: Wavehill

©Copyright Wavehill Ltd. 2012

Page | 38

Evaluation of the New Work Connections Project | Report 2: October 2012

Project managers need to take this into account within their planning or make sure that plans are in place to continue the service being provided soon enough to avoid the winding down period. In an ideal scenario, the ‘slow start’ phase of the next project / successor to NWC would in fact start during the ‘fully operational’ phase of the current project although experience suggests that European funding structures and processes tend to make this difficult. An important lesson learnt therefore is that a ‘slow start’ is almost inevitable and needs to be built into the planning process. Staffing a project of the scale of NWC will never happen overnight and there is always a risk that, without key staff being in place, the necessary systems will not be put in place leading inevitably to problems further down the line.

5.8.

A long term process and continuity of support

A related issue which has been highlighted as being of concern to NWC staff is the fact that many of those being supported by NWC are being taken on a ‘journey’ towards work that will, for many, take a long time; longer than the lifetime of the NWC project in many cases. The individuals concerned are also often those furthest away from the labour market and change (more specifically, a withdrawal) in the support provided could lead to progress that has been made being lost. The concern of NWC staff was that the progress that has been made due to NWC (the results generated) could be lost unless there was some link of continuity / continuation of the support being provided. The logic of this argument is clear and some of the evidence gathered from participants by the evaluation supports it (please refer to Sections 6 and 7). Again, this suggests that NWC managers need to have plans in place for the end / next phase of NWC as soon as possible. It is also important to note that the economic and public sector finance ‘climate’ in which NWC (and the original exit strategy) was originally devised was very different to the much more challenging circumstances in which the project is currently operating. Inevitably, this will impact on what the ‘exist strategy’ for NWC will look like.

5.9.

Administration / paperwork / bureaucracy

The administrative burden on project staff was a constant theme in the ‘pause & review’ sessions and discussions with staff, the following comment being typical of the points being made: •

“The paperwork takes a lot of time, the registration process etc there’s XXXX loads of paperwork required for outcomes, there’s a lot of duplication and you can get bogged down and behind. I understand you have to have evidence but it could be simpler.”

Our experience suggests that, to some extent, some administrative burden is inevitable within a European funded project due to the rules that accompany it. The key is however how you manage that burden. The general feedback from NWC staff is that the administrative burden is currently too much. But, there would seem to be a number of factors that have led them to that view. One of these, as previously discussed, is that the administrative processes for the project were not in place before delivery of the project started. Another is that there have been changes to the administrative process and the paperwork that goes with it on an almost ongoing basis since the beginning of the project (or at least, that is the perception of staff).

©Copyright Wavehill Ltd. 2012

Page | 39

Evaluation of the New Work Connections Project | Report 2: October 2012

It is important to acknowledge that many of the changes to the administrative processes that have been introduced as the project has progressed are due to changes introduced by WEFO; for example, changes to the definitions of performance indicators. It is not, therefore, something that the project could control. Those changes have however been challenging from a project management perspective. It is also an indication of the critical (and challenging) role that the Regional Team plays as the ‘link’ between WEFO and the NWC delivery teams. One of the key changes to the structural funds programmes for the 2007-13 programme period compared to its predecessor is the greater role of larger regional projects such as NWC. From a WEFO perspective, one of the benefits of such projects is that it reduces the administrative burden on their team by reducing the number of projects that they have contact with; in the case of NWC, WEFO has contact with one project rather than four as would have probably been the case in the 2000-2006 programme period. However, the administrative burden of working with those ‘four projects’ has not gone away; it is being undertaken by the Regional Team. And, the feedback to date is that undertaking that role has been made very difficult due to a lack of clarity and clear guidance on a range of critical issues from WEFO. It is also relevant to note that, in many cases, this is the first time the staff concerned have been involved in the delivery of an European funded project. Accordingly, they are not familiar with such an administration process and are therefore uncomfortable with it. This is another important lesson learnt and the challenge is to ensure that the knowledge and experience that has developed over the lifetime of the NWC project is maintained or transferred into the next programme period. Again, this is an issue the ‘exit strategy’ (perhaps better described as a ‘transition strategy’) will need to consider. The perception that the administrative burden is too heavy is however something that needs to be addressed by NWC management. It is important to acknowledge that steps have been taken at a regional level to try and address concerns about the administrative burden. For example, staff are encouraged to raise their concerns and issues which are shared by way of a Q&A document across the region so that the other authorities are informed of the question raised and the response provided. There is also an ‘Operational Documents Process and Guidance’ document that explains step by step what needs to be completed, why it needs to be completed, etc. so staff already have an opportunity to raise their concerns and full responses are provided. Whilst administrative issues are probably inevitable within any project of the scale and scope of NWC, the fact that issues and problems are still being identified by staff suggests that the actions that have been undertaken to try and tackle the issues may not have been as effective as they perhaps should have been. The obvious question this raises is what else can be done? One potential way of addressing this question is to create a staff working group that considers how (if at all) the burden on staff can be reduced without compromising the need to meet the audit requirements that come with ESF funding.

©Copyright Wavehill Ltd. 2012

Page | 40

Evaluation of the New Work Connections Project | Report 2: October 2012

5.10. The emphasis on ‘employment’ targets / performance indicators Linked to the above, it is clear that there are mixed views amongst staff regarding the emphasis within the project on performance indicators and achieving the targets that have been set. Again, this is something which is outside the control of NWC management; the performance indicators and targets are a requirement of ESF funding and increasingly, public sector funding more generally. This is in fact generally accepted by those involved in NWC (although sometimes grudgingly). The concern is more around the appropriateness of the targets to the breadth of work being undertaken as part of NWC and the range of participants being supported. The following are typical examples of the views being expressed: •

“I'd prefer it if there was not so much pressure on targets and outputs because it is so difficult to work with our participants. Getting participants to engage is hard enough.”

•

“Soft outcomes I don't think count, they do to us but it's difficult to get soft outcomes to look good on paper. Things like communicating, being dressed nicely, etc. are big outputs but it's difficult to evidence this.”

The following is an interesting alternative view: •

“I think with this project there are a lot of soft outcomes but we need to make more of an effort to get people into work as some people already have these skills, there should be more urgency getting people into employment. It’s okay giving participants skills but we need to find them jobs... The council should have relationships built with local employers, they should have contracts with companies so we can get people jobs. I’ve come from a very targeted environment and here it is more ‘soft’ focused but I think things are changing.”

The key to this issue is perhaps a need to understand and acknowledge the range of participants that the NWC project is working with and the fact that the outcomes (or results) that will (or can) be achieved will be different along this continuum as illustrated below. Figure 5.3: An illustration of the continuum of participants NWC supports

©Copyright Wavehill Ltd. 2012

Page | 41

Evaluation of the New Work Connections Project | Report 2: October 2012

This is an issue which has been raised with WEFO by the Regional Team and as a result the project has been invited to re-profile its targets so that they are realistic and achievable. This enables the project to maintain its focus on supporting those furthest away from the labour market. As is apparent from the analysis of participant characteristics and interviews with 100 participants in this report (Sections 3 and 7 respectively) NWC supports a wide range of individuals, some of whom need a significant amount of intensive support in order to progress and some who require less support to do so. This is illustrated in the graphic above in terms of participants distance from being ‘self-reliant’ and ‘work ready’; some participants will be closer to the right hand side of the continuum than others and the type of support they require and receive will differ accordingly. The objective of the support provided is the same; to move the participant in question towards a situation where they are able to find and hold a job. Their starting point towards that goal is however different and it is important to acknowledge that point when assessing whether or not NWC has been successful. The concern that some NWC staff have is that (with the possible exception of the ‘other soft outcomes’ indicator) the targets that have been set for the projects (and hence the indicators that are being used to assess the success of the project) are relevant to participants towards the right of the continuum. If, as could potentially be the case, most NWC participants are actually towards the left of the continuum, there is a risk that (a) the targets will not be achieved and (b) the true achievements of the project will not be recognised. Such a concern is understandable and is one of the reasons why the Work Star was introduced into the administration of the project. Some staff are however still concerned that the emphasis within Work Star is also too heavily skewed towards ‘work’ rather than monitoring the more social benefits being achieved by the project when participants are furthest to the left of the continuum. To some extent, the evaluation can and will help to address this concern by recording the broader benefits of the project during interviews with a sample of participants. But, the difficulty in terms of ‘imposing’ a consistent monitoring system on a project that works with such a broad range of participants needs to be acknowledged and recorded as an important lesson learnt. The evaluation will consider this point further as it progresses, but consideration should possibly be given in the future – potentially at a programme level - to introducing a broader monitoring system that acknowledges the breadth of work being undertaken and/or splitting the monitoring system into several parts or layers that better acknowledge what the support being provided is trying to achieve.

5.11. The introduction of the Work Programme A number of references is made in the notes of the pause & review sessions to the introduction of the Department for Work and Pensions (DWP) Work Programme and the impact it had on the NWC project primarily as a result of the uncertainty that it introduced for a period of time. As discussed in some detail in Report 1 of the evaluation, the Work Programme is central to the reform of welfare introduced by the UK Government since 2011. It replaces all previous mainstream (Government) support activity, and is, in theory a ‘black box approach’ open to all, although disabled people may get support from the Work Choice programme 17. ESF support (such as that provided through NWC) cannot be used for people eligible for mandatory entry to the Work Programme as it would substitute expenditure which the Member State would make from its own resources. As such, the introduction of the Work Programme required all ESF funded projects to review their activities.

http://www.direct.gov.uk/en/DisabledPeople/Employmentsupport/WorkSchemesAndProgrammes/DG_187696

©Copyright Wavehill Ltd. 2012

Page | 42

Evaluation of the New Work Connections Project | Report 2: October 2012

In the case of NWC, the conclusion of the review was that the introduction of the Work Programme would have minimal impact on the project; the groups of people the project aims to support did not alter (although the eligibility criteria was changed to ensure that there was not overlap) and the service being provided did not change. Nevertheless, the uncertainty generated by the introduction of the Work programme (especially for the many months during which it was unclear what the impact on how NWC operated would be) and the time it took to clarify the situation and approve the re-profiled expenditure and outcome plans undoubtedly had a negative impact on the management and delivery of the project. As such, it is an example of a change to the ‘context’ in which a project is being delivered which could not have been foreseen or planned for. The introduction of the Work Programme and the administrative work it generated has however provided an example of one of the benefits of having a regional team overseeing the delivery of NWC. The regional team had a key role to play during this period in terms of monitoring developments and negotiating with WEFO on behalf of all the project partners. The efficiency of such an approach (as an alternative to potentially having four local authorities or projects undertaking that same role) is obvious. The regional team was also able to co-ordinate positive action to address the administrative challenges introduced into the system as a result of the Work Programme by, for example, developing a partnership agreement between NWC and Job Centre Plus. There was already such an agreement in place in Conwy but it was apparent that the level of support from the Job Centre Plus in each authority was inconsistent and the regional agreement ensured that there was greater collaboration and consistency across the region. As part of this agreement, it is possible to negotiate deferring the referral of NWC participants to the Work Programme in circumstances where that would have a detrimental effect on the ongoing support they are receiving from the project.

5.12. The concern and influence of parents and guardians An interesting issue highlighted during the ‘pause & reviews’ sessions is the influence that parents or guardians can have on their children and their participation in NWC; a number of instances where parents had stopped their child from participating in the project due to concerns about the impact it would have on the benefits they received were identified. According to staff, their main concern was the emphasis within the project on progressing participants towards work. Their concerns had also been exacerbated by the reforms to welfare policy being introduced by the UK Government. Two ‘lessons learnt’ were identified from the above. The first was the need for caution in terms of how the purpose of the project is introduced; a clear link to moving participants into work was not always helpful. The second was the need to provide information about the project to parents and guardians as well as to the participants themselves in some circumstances.

©Copyright Wavehill Ltd. 2012

Page | 43

Evaluation of the New Work Connections Project | Report 2: October 2012

6. Case studies: the project at work Section summary The case studies illustrate the ‘personal’ service that NWC participants receive. They also demonstrate the vital role played by the case officers and the value of the relationship that they develop with the participants.

6.1.

Introduction

This section of the report has been included to provide examples of the support that NWC provides to participants based on visits undertaken by Wavehill researchers to each local authority area in August 2012 during which they met with NWC staff, shadowed them as they undertook their work and met with participants. It is important to stress that these are only examples of the wide range of support that is provided to participants via NWC, but the ‘case studies’ provide an indication of the support that is being provided and some of the challenges that NWC staff face. All the names of participants have been changed in order to maintain their anonymity.

6.2.

Anglesey

6.2.1. Canolfan Byron Canolfan Byron is a purpose built centre offering a wide range of services to promote independent living and for people with disabilities and older adults. The people at the centre all had varying degrees of learning disabilities. The evaluation team met two participants making upholstery at Canolfan Byron. •

Tom is male, 26 years old and was born with a benign brain tumour which caused speech and balance problems. Tom is progressing through NWC, has completed one module and was working in the upholstery workshop to get experience. He is hoping to attend the woodwork centre, 4 days a week, once it is open later in the year but the long term plan for him is to get a paid placement with a small company. The mentor explained that NWC are getting a unit in Holyhead where they can run woodwork and craft courses on ‘rainy’ days when working at the polytunnel project is more difficult.

•

Bryan is male, 25 years old and has mild to moderate learning disabilities. He has been going to Canolfan Byron for a couple of years and attends 4 days a week. He is working in the upholstery workshop to gain experience and is also able to show other participants how to do the various different parts of the upholstery process.

6.2.2. ICT training The team also visited the Communities First project offices at Llangefni where ICT courses are delivered. In attendance were seven participants, six undertaking accredited ICT courses and the other one undertaking a Word Foundation Course, OCN level 1. The courses are run every Wednesday between 10.00am and 12.30pm. The IT equipment belongs to Communities First but the room and equipment is used by NWC participants free of charge.

©Copyright Wavehill Ltd. 2012

Page | 44

Evaluation of the New Work Connections Project | Report 2: October 2012

The following comments were made by the participants in attendance during informal discussions with the researcher: • • • • • • • •

“I am finding the course useful; the Job Centre told me about NWC. I want to be a mechanic ideally but doing this course is something good for my CV as it shows potential employees that I have been doing something.” “When this course finishes I am hoping to do more courses because long term as a job I would like to assemble computers.” “If I was not here I would be at home, it is good to learn skills and meet people.” “I was unemployed and the Job Centre referred me here, I want to enhance my computer skills. I have been unemployed since April this year so am hoping this will help me get a job.” “It’s got me out, away from my home situation, it opens your mind. I’m getting my confidence back and starting to talk to people, it’s getting me ready for work, that’s my main aim.” “You need to get that push as sometimes you feel trapped, you get that here.” “It gets me out of the house, I’ve got bad depression as well as fibromyalgia and if I wasn’t here I’d be sitting at home. It’s getting me to know people again.” “I just enjoy doing this; it’s something that gets me up in the morning as I know I’m coming here. I’d recommend it as I get a lot out of it, if I wasn’t here I’d just be sitting at home not doing a lot. It’s a life saver for me, I can do more now and I’m not in my little bubble.”

6.2.3. Polytunnel project, Gerddi Twt This project is located next-door to a day centre for adults with learning disabilities. There is on average 15 participants in attendance most days who have been referred to NWC through social workers or Community Psychiatric Nurses. The participants in attendance at the polytunnel either have a mental health problem or a learning disability and the mental health participants tend to ‘mentor’ the participants with learning disabilities. On the day that the researchers visited there was also three members of staff and three volunteers in attendance. Wavehill spoke briefly to the project manager who said: •

“People enjoy coming here because they feel safe, it gets people out and less isolated. The project was here before Taith i Waith but not on the same scale. The participants that come here are here for varying amounts of time, some take years to move on. We work alongside Community Psychiatric Nurses which took quite a while for relationships to work but it now works really well and without this project [TiW] what else is there for people?”

Another project worker said: •

“It is getting people out as otherwise they isolate themselves, they feel safe here and we do a range of activities to get people out, gaining confidence etc., and then we can refer them on, it’s like a stepping stone and the length of time they’re here varies person to person.”

The researchers also spoke to a Disability Liaison Officer for Anglesey who is involved in the Lottery funded project ‘Beyond the Barriers’, he said: •

“I work with Taith i Waith and now have someone to refer my clients to or they refer people on to me.”

©Copyright Wavehill Ltd. 2012

Page | 45

Evaluation of the New Work Connections Project | Report 2: October 2012

Comments made by the participants in attendance included: • •

“I come here and I do the ICT course and I am part of the fishing club and I like all of them.” “I enjoy it here, if it wasn’t for these guys I’d still be in bed. They have given me a lot of help in getting my confidence back and if it wasn’t for this place I don’t know where I’d be. Everyone helps everyone else here.”

6.3.

Gwynedd

Wavehill researchers spent an afternoon with a member of the NWC team in Gwynedd who provide participants with one-to-one support with job searches, voluntary work placements, filling out applications (job and further education), confidence building and interview skills. The first appointment was with a participant, male approximately mid-fifties in age who was suffering from mental health problems referred to as Paul below. This was the second time that the case officer had met Paul; the first meeting was to go through the registration process and check eligibility and this second meeting was to go through the Work Star. The case officer started the meeting by discussing Paul’s concerns and anxieties and also working through different types of jobs that may be suitable for him. One of his main concerns was the fact that he did not have access to broadband at home and therefore found it very difficult to send CV’s out to people. He explained that he visits the library but he is only allowed one hour and he also finds it very distracting there. It is also difficult to get to the library as he has to catch a bus. The case officer explained that the library rented out laptops and he could potentially rent a laptop for 3 months. Paul was interested in this prospect and so the case officer said she would find out more information for him. The case officer then asked for an update in terms of jobs vacancies that Paul was aware of and said she would contact the Job Centre and get an up to date list of vacancies for him. She also mentioned the possibility of Paul up-skilling and suggested an ECDL course. Paul was interested in this but felt that if he did do this course (the course would be undertaken from home) then he would be unable to look for work at the same time. The case officer explained that many people are able to do both as the course was very flexible so suggested he think about it and in the mean time she would get more information for him. When completing the Work Star, Paul struggled to score himself against a number of the criteria that were used tending to score himself higher than the discussion suggested was actually the case. Accordingly, it was necessary for the case officer to take the lead in the scoring process. Once the form was completed, the case officer explained that they would review the Work Star again in three months’ time and the discussion progressed to the action plan and three things that had been agreed as actions: • • •

Provide information on ECDL course Help with job searches Look into computer issues at home

Paul agreed the action plan and, referring to the case officer said: “you do everything that is possible for me and I know I can come back or phone you if I need anything and that is very useful”. The researchers also met with a mentor who explained that her job was to help to try and build participants confidence and get them out of the house;

©Copyright Wavehill Ltd. 2012

Page | 46

Evaluation of the New Work Connections Project | Report 2: October 2012

•

“The time I spend with each client varies, I roughly see a client once a week for 2 hours but there is no set period of time, I work with them as long as is needed. It is nice when you see them start to get out and about. I meet them outside their homes, I phone them beforehand. I can have initial meetings at people’s homes or here or at a café – wherever they want. The case officer and I try and do the first visit with a participant together because you can see more when you visit someone’s house but we don’t do a lot of this because of the risks involved.”

6.4.

Conwy

Wavehill researchers ‘shadowed’ a NWC case officer in Conwy as he attended three appointments with participants. All the appointments were ‘outreach’, in participant’s homes.

Appointment 1 The first appointment was with a female in her mid-twenties and unemployed referred to below as Julie. Upon arriving at Julie’s house she explained that something had come up and it was no longer convenient to have the meeting. The case officer agreed to call her to re-arrange and later explained that this was the second time she had re-arranged. Accordingly he would need to have a discussion with her to ascertain how committed she was in terms of receiving support from NWC.

Appointment 2 The second appointment was with a female originally from the Philippines who came to Abergele from the Netherlands in December 2011; referred to below as Sally. Sally is currently working four hours a week over four days in a pub but it costs her £6.00 each day that she goes to work and in addition, the money that she earns is deducted from her partners JSA allowance. She still continues to do the job as she feels it will look good on her CV. Sally and her partner have some quite extensive financial worries in addition to the fact that they have to leave the flat they are currently living in by November 2012. The case officer explained that the meeting would be the 3rd time they had met; he has previously updated her CV and is now trying to help her find a job. The case officer informed Sally that he had discussed her case with the advisor at the Job Centre on her behalf. He also gave her information on a number of job opportunities that he had found and said that he could help her fill in the application forms. •

“I really search for work, I search on the computer, at the Job Centre, in newspapers, and I call into businesses and ask them if they have any jobs – I just want to work. I am happy, I am glad that [the NWC case officer] helps me because he is a good guy, he cares, and no-one else cares. The man in the Job Centre told me that [the NWC case officer] was very good and could help me lots, he said I can give all my problems to [the NWC case officer]” - Sally

©Copyright Wavehill Ltd. 2012

Page | 47

Evaluation of the New Work Connections Project | Report 2: October 2012

Appointment 3 The third appointment was with a female, lone parent with a 6 year old daughter, referred to below as Louise. She had problems previously with an ex-partner who had been abusive and she had had to move to Llandudno for their safety. This had meant moving away from family and friends which had contributed to a total loss of confidence and she had depression. This would be the third time the case officer had met her and she had already been on Conwy NWC’s ‘Let’s get ready to work’ three week course which was one day a week providing support for CV writing, mock interviews and so on. Louise wanted to keep busy so he is going to see if she would like to do another course, due to start in September. The case officer gave Louise leaflets and information he had found on activities that were taking place locally that she could take her daughter to over the summer holidays like the Welsh Mountain Zoo who have cheaper admission for lone parents, a fun day in Llandudno and other things she could do to keep her busy. He told her about the course that he had identified as a result of comments made at a previous meeting. In addition, further to Louise’s suggestion at their previous meeting that she would like to study to be a nutritionist, he had found a person who works for the local authority who is a dietician working with pregnant women who would be happy for her to shadow him so she can see what the work involves. He also told her that she can do an accredited course relating to nutrition. •

“[Before being supported by NWC] I didn’t have a CV or nothing, I’d lost my confidence, I’m more relaxed with [the case officer] coming to the house. It’s helping a lot; I’m only just getting into it, and it is early days. It is great to have someone who can go off and find me suitable courses etc. and signpost me to different bits and pieces.” - Louise

6.5.

Denbighshire

Wavehill researchers ‘shadowed’ a NWC case officer in Denbighshire as she attended two appointments with participants.

Appointment 1 The first appointment was with a male, 19 years old, registered blind, who had been referred to NWC by his social worker and had been receiving support from NWC for 18 months; referred to below as Mark. The case officer explained that Mark had a very poor experience with education in the past and so when he came to NWC he did not want to go into further learning, he wanted to get a job. At that time, Mark had nothing on his CV as he had no work experience and so on. The case officer organised an interview for a voluntary placement at a local third sector organisation in Llandudno. The interview went very well; however, due to a number of issues and no contact from the organisation in question, the participant was left very let down by the whole experience. Mark came to the appointment with his stepfather. They both discussed the lack of contact from the organisation with whom a voluntary placement had been organised with the case officer who agreed to write a letter of complaint on their behalf. The amount of work that had gone into trying to arrange the volunteer placement in Llandudno for the participant in question was very apparent during the discussion as was the subsequent work required to ‘recover’ the situation once the placement did not happen; the case officer had to work hard to maintain Mark’s positivity.

©Copyright Wavehill Ltd. 2012

Page | 48

Evaluation of the New Work Connections Project | Report 2: October 2012

The case officer also spoke to Mark about his change in decision to go to college. Mark explained that he had attended one college interview which had gone well and was due to attend another interview later in the month. The case officer checked if Mark was confident with any travelling that going to college would entail. Mark and the case officer agreed to meet again after the college interview to see how things were going. Before leaving Mark’s stepfather told the Wavehill researcher that the case officer had been very helpful and that he found the treatment of his son by the organisation that had ‘let him down’ in terms of the volunteering opportunity “disgusting” but that he was very grateful for all the support that the case officer was able to give them.

Appointment 2 The second appointment was with a female, 52 years old with mental health issues, referred to below as Hannah. Hannah had been employed on a temporary basis for the past 11 months through an ILM scheme 18 but that post had recently ended. The case officer explained that Hannah was not job seeking at the moment because they needed to work on her confidence. Hannah had been suffering from problems with her benefits; her son was in the army and currently in Afghanistan and the loss of her ILM job, all of which have caused a great deal of anxiety. The case officer and Hannah were meeting once a week to try and build Hannah’s confidence and resolve some of the issues she was encountering. Having completed the Work Star, the case officer said that they needed to focus on stability first, things that they can do to sort out including benefit and housing issues. Once those issues had been resolved then they would organise for Hannah to attend a finance (SAGE) course at the local college. The following is an extract from the conversation between Hannah and the case officer: •

Case officer: “We’ll give ourselves until September to sort a course and we will re-visit the action plan in a few weeks. We will also do a new work star in four weeks’ time and perhaps look at an IT course so you can up skill and we will also look at funding for the SAGE course. The local Women’s Centre is running a confidence course, I have been to the centre and it is a lovely environment, non-judgemental and it would give you the opportunity to meet new people, do you fancy it?”

•

Hannah: “Yes, I moved back to this area having been away for 30 years so it would be a really good chance for me to meet people. It could also be the ripple effect couldn’t it? I need to meet people.”

The case officer called the Women’s Centre there and then and checked availability of the course and booked Hannah on the next available one. She also said that in the meantime she would go with Hannah to the centre to have a cup of tea and a chat so she could see how friendly the place was before going on the course.

ILM stands for Intermediate Labour Market. ILM projects provide paid work experience and training for people facing barriers to employment. The ILM scheme referred to here was a European funded (ESF) project managed by the WCVA: http://www.wcva.org.uk/europe/index.cfm?sub=6&display_sitetextid=1054

©Copyright Wavehill Ltd. 2012

Page | 49

Evaluation of the New Work Connections Project | Report 2: October 2012

The researcher asked Hannah what she thought of the support she had received from NWC: •

“Without [the case officer] and NWC I would be in hospital now, I have depression and anxiety and I have had a breakdown which is something I don’t discuss with anyone else other than [the case officer]. I can’t talk to other people, like the people in the Job Centre, they just give me forms. I went there the other day and I was in tears and then I came here to [the case officer] and I knew I could talk to her about anything and it’s confidential and she would help me.”

©Copyright Wavehill Ltd. 2012

Page | 50

Evaluation of the New Work Connections Project | Report 2: October 2012

7. Thematic analysis of participant interviews Section summary • •

• • •

Telephone interviews were undertaken with 100 individuals, selected at random, who had participated in the project. Participants understand the focus within the project on getting people into work. But, the primary benefit of NWC apparent to interviewees was the ‘stability’ it brings to their lives and specifically the skills and opportunity to socialise and ‘get out of the house’. The confidence that participants develop as a result of their participation in the project was a common theme within interviews. It is apparent that the role of the NWC project worker / case worker is central to the project and was regularly cited as the main or best element of the NWC experience by participants. Analysis of the interviews demonstrates the wide range of participants that the projects supported; ranging from those who are relatively close to the labour market to those who need a considerable amount of support before they could be considered to be ‘work ready’.

7.1.

Introduction

The aim of this element of the evaluation is to increase our understanding of the benefits that the individuals attribute to their participation in NWC. The analysis that follows draws upon telephone interviews undertaken during June and July 2012 with 100 participants in the project selected at random from a database of participants who agreed to take part in the evaluation. Eighty-nine of these interviews were with the participants themselves with 11 interviews undertaken with a parent or guardian on behalf of the participant. In addition to the 100 interviews undertaken we also spoke to a further 19 people who said: • • •

Too busy to take part x 3 Don’t want to take part x 3 Can’t take part because they did not know about NWC or had received very minimal support and felt could offer no feedback x 13

The analysis was undertaken using a system called ‘open coding’ and ‘thematic analysis’; a method whereby the analyst reviews the text and then identifies ‘codes’ within the text. Those codes are then clustered into ‘themes’. For example, the interview text may say: “I like the work experience, I feel it gives me confidence and gives me new skills”. The open codes are to ‘like work experience’; ‘confidence’; ‘new skills’.

©Copyright Wavehill Ltd. 2012

Page | 51

Evaluation of the New Work Connections Project | Report 2: October 2012

The thematic analysis then comes from these identified codes. Codes that mention work experience across several interviews may be: a) b) c) d) e)

“Like work experience” “Work experience helps me understand about work” “Work experience gives me confidence to do things.” “Work experience worries me – I feel scared in case I do something wrong.” “Work experience got me a job! “

The thematic analysis may be that five participants noted that work experience was the main outcome from their participation in the project. But the analysis would also identify that work experience can lead to employment and can build confidence; but it can also be worrying for a participant.

7.2.

Background information about the participants interviewed

Of the 100 participants interviewed in this round, 51 were male, 49 were female and the average (modal) age range was 46-60 years old. The following chart shows the split of interviewees per local authority area. Figure 7.1: The percentage of 2012 interviewees per local authority area

Conwy 32%

Anglesey 20%

Denbighshire 31% Gwynedd 17%

N=100

As illustrated by the graph below, most participants interviewed (69) had been engaged with NWC for less than one year at the time of the interview with 31 having been engaged for less than three months. This will clearly potentially influence the feedback from participants about the perceived outcomes of the support provided, as discussed later.

©Copyright Wavehill Ltd. 2012

Page | 52

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 7.2: The length of time participants interviewed had been supported by the project 35 31

30 25 25 20 15

10 5 0 0 > 3 months

4>6 months

7>12 months

1 year +

N=100

The interviews found that the most common (modal) reason given when participants were asked to explain why they were being supported by NWC was Mental Health Issues (47). The next most common reasons given were Physical Disability (14) and Learning Disability (11). Figure 7.3: The reason given by participants interviewed when asked why they were being supported by the project

Other

On benefits

Long-term unemployment

Learning Difficulties

Physical Disability

Mental Health issues

47 0

N=100

The following are examples of the comments made by the interviewees: Mental Health issues x 47 â&#x20AC;˘

"I'm 43 and for the last 10 years I've been suffering from anxiety which has brought on depression."

ÂŠCopyright Wavehill Ltd. 2012

Page | 53

Evaluation of the New Work Connections Project | Report 2: October 2012

• •

"I'm 26, I've got bi-polar disorder and had a psychosis when I had my last career which was abroad so I had to move back, this was three years ago." "I'm 23, I was referred through the supported housing I live in. I have mental health problems."

Physical Disability x 14 • •

"I'm 42 years old and was injured in an industrial accident about six years ago." "I'm 50, I was signing on at the Job Centre because I've got a disability - I had a fall about 18 months ago and had a haematoma in my leg."

Other x 18 • •

"I'm 45, it was through the Job Centre one of my mates was on it and I asked if I could too." "I go to a day centre called Trefergun where [NWC worker] is placed, I go and do voluntary work in the garden, [NWC worker] started the ball rolling."

7.3.

The type of support participants had been receiving

Those interviewed were asked to describe the type of support that they had received from NWC. The responses focused on employment related outcomes such as developing a CV, and Job Search. However, as important is to build the confidence of the individual so that they are also ready and confident to approach the work place and employment market. Figure 7.4: Response to the question: What kind of support have you received from NWC? Coded 30 25 20 15 25

20 13

0 Help with CV:

Help finding work:

Help with Confidence Sent on computer: building: Course:

None:

Other:

N=100

©Copyright Wavehill Ltd. 2012

Page | 54

Evaluation of the New Work Connections Project | Report 2: October 2012

Help with CV x25 • • •

"Help with my CV, it was brilliant!" "(Support worker) made me a CV." "We've done a CV, and I tell him (support worker) what I like to do."

Help finding work x 24 • • •

"Finding me jobs and going with me to interviews." "Help in how to look for work, how to conduct myself in interviews and sell myself." "At the moment I'm doing gardening on a Wednesday and practical theory for my driving test on Fridays."

Sent on Course x 20 • • • •

"Maths ‘O’ level course." I've done a course building up confidence, he's (NWC worker) helped me search for jobs on the internet, I'm starting a writing course in September at the library. I've also had help with my CV.” “I had help finding a job and with looking for college courses. (NWC support worker) researched suitable courses and went with him to the college for interviews.” "Three day course for over 50's, to tell us how to apply for jobs etc."

Confidence x 20 •

•

“It was just sort of confidence building really, (support worker) was lovely, she had a welcoming feeling about her and gave me the incentive and self-belief to carry on going. I took part in arts and crafts and needlework, the support was like a buddy system really and got me speaking to different people so I gained confidence and belief in my abilities.” “(Support worker) has been there whenever I've needed her, she's been really supportive and been on the phone checking up on me, emailing too to see if I'm okay. She's helped me with my confidence and self-esteem, meeting people, going with me and sorting things out for me like courses, just being there for me.” “[NWC worker] helped me set up a Welsh language course, I also went to confidence classes with her. With [NWC worker] I've been carrying on with the Welsh, and I'm hoping to get back into school next September. I've just been talking to her telling her what I want to do. Going to do an ICT course in September.” “I've done a course building up confidence, he's (NWC worker) helped me search for jobs on the internet, I'm starting a writing course in September at the library. I've also had help with my CV.”

©Copyright Wavehill Ltd. 2012

Page | 55

Evaluation of the New Work Connections Project | Report 2: October 2012

7.4.

Anticipated outcomes

The main outcome anticipated by participants as they enter the project is that they will find employment (55). However, other outcomes such as a ‘confidence boost’ (13) or ‘advice’ (12) were also noted which is an indication of the ‘social’ and ‘employment’ focus of the project. Figure: 7.5: Response to the question: when you first started receiving support through NWC, what were you hoping to get out of it? Coded 60 50 40 30

20 10

N=100

To find work x 55 • • •

"Help with finding work." “I was hoping I'd gain confidence and would break the relationship/cycle with my mental health and not feeling able to work. I wanted to get back to work and be able to find the staying power, the ability to continue with working, to be employable.” "A helping hand back into the world of work."

A confidence boost x 13 • • •

“Confidence, having someone supporting me. My husband has been made redundant, we were inspected by the Job Centre as to what we were doing to find work, it wasn't a great experience.” "It's just to get me out of the house and get more confidence." "To improve my confidence and learn new skills."

Advice x 12 • • •

"An avenue to get back into the community." "Just to build up my confidence and socialise more." “I was lost, I didn't know what I wanted to do or where to go, she (case worker) explored every avenue with me. I'm now starting an access to university course in September.”

©Copyright Wavehill Ltd. 2012

Page | 56

Evaluation of the New Work Connections Project | Report 2: October 2012

7.5.

Perceived main benefits

Interviewees were asked to describe what they perceived to be the main benefits of the support they were receiving from NWC. Their responses were subsequently coded into the following categories19: • • • • • • • • • • •

Job specific skills - skills and experience necessary to get a job Aspiration & motivation – knowing what you want do and believe you can do it Job-search skills – using a computer, preparation of CV, interview skills etc. Stability – improvements to day to day lives Basic skills – literacy, IT, numeracy, use of telephone etc. Social skills for work – self-confidence, turning up on time, behaving appropriately etc. Challenges – barriers to employment / training , health issues, loss of benefits etc. Enter into further learning Other soft skills - making friends, working as part of a team, travel independently etc. Gain qualifications Other – anything not covered by the above.

As shown in the graph below, a range of employment related benefits were identified including ‘job search’ skills (34) and ‘job specific skills’ (14). However, the benefit identified most often was ‘stability’ (39) which suggests that, in addition to the employment related outcomes, there is another dimension based around social and psychological confidence building, self-esteem and stability (39). It could however be argued (as NWC staff do) that participants need to build their social skills and self-esteem and confidence before they can tackle the issue of finding employment successfully; these are in fact therefore also employment related outcomes. Participants with low confidence and self-esteem and social skills (31) may not be able to hold a job for a longer period of time. It is, however an indication of the type of support NWC participants require, at least in the initial stages of their engagement with the project.

It should be noted that the responses of interviewees were coded based on the categories identified within the Work Star in order to allow the findings of the telephone interviews to be compared with the findings of the analysis of Work Star data.

©Copyright Wavehill Ltd. 2012

Page | 57

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 7.6: Response to the question: What are the main benefits you are getting from NWC? Coded

Stability

Job-search skills

social skills

Other soft skills

Aspiration and motivation

Qualifications

Anything else?

Job-specific skills

Enter into further learning

Challenges

Basic skills

2 0

N=211; Note participants could select more than one response.

The following are examples of the comments made by participants during the interviews: • • •

• •

•

“I'm getting more experiences of life by doing courses, I'm meeting other people, gaining more confidence as I'm getting out.” “I built a bit of self-confidence from it by having someone else when I was looking at my employment history saying you can do this, this and this. Help with a CV.” “Being out with other people working with his hands. It's new experiences, he's getting a chance to learn new things working in a safe working environment. It is somewhere for him to go, something for him to do, he's doing some work with his hands and it is getting him out of the house. It is helping his confidence and his communication.” “It gets me out meeting people otherwise I'd be stuck in the house, it's improved my confidence talking to people, I'm learning new skills - gardening and theory for driving and in the winter I'll be doing woodwork.” “It has improved my confidence and phone confidence as I was working on reception at Taith i Waith one day a week, which also helped me with college as I needed to do that as part of my course, learning new skills has helped me improve my communication skills with people, I've done interview skills and it has helped me to go to college.” “I've got a job and I can go places now as I've got more money, it improved my confidence in applying for jobs and meeting people - everything, she [case worker] helped me do a CV. She told me about being reliable for a job, telling me to be on time and do whatever you can.”

©Copyright Wavehill Ltd. 2012

Page | 58

Evaluation of the New Work Connections Project | Report 2: October 2012

•

“I'm now a little more confident as I do volunteering in a kennels one day a week, (worker) from Taith i Waith takes me there and is looking to see if there are any buses so I can go on my own. Working in the kennels is giving me more confidence and exercise. Going to the Byron Centre and walking club gave me somewhere to go, something to do.”

Ten interviewees did not believe that they had benefited in a substantial way from their participation in NWC. • • • •

“I didn't get any benefits - it was meant to be a job club but no one else turned up so it was just me and (NWC support worker) there, I went a couple of times and then (NWC worker) went off on the sick.” “None... I only went for two weeks and stopped going as I don't think it really helped me.” “I didn't get any benefits really... I was still working for the woman until October, which was a couple of months after I first went to NWC. I suppose the job searching skills were helpful.” “Not many... I don't think NWC is right for people with mental health problems. I wasn't looking for work either as I am on my pension, but I felt that as they were only for two hours a week they wouldn't make a difference anyway. They should be for two hours a day. Just finding courses for people is not what people want.”

The benefit gained from participation in NWC is obviously a very important question, and so we have looked deeper into the headline responses above, by analysing each response category further and specifically looking at what had led to those benefits.

©Copyright Wavehill Ltd. 2012

Page | 59

Evaluation of the New Work Connections Project | Report 2: October 2012

7.5.1. Stability, social and other soft skills The category of social skills and stability were prominent amongst the benefits identified by interviewees. In terms of stability; interviewees made reference to having something constructive to do and achieve - often expressed as getting out of the house (20) and building social confidence (socialising 16) - are what could be described as softer benefits of the support being provided. Figure 7.7: Stability benefits identified by participants

Gets me out of the house

Socialising

Something to do

Gave me support

Confidence

Benefits

Payment

Help to trust people

Routine

1 0

N=39 Note participants could select more than one response.

The following are examples of the comments made: •

“Socialisation and realising part of a dream as she has always wanted to do catering and we've never been able before this to find anywhere for her to go. Otherwise on a Wednesday she wouldn't be doing anything, there was a lot of voids in her life. [The support worker] is now part of the family, she is brilliant with her. She was bone idle before, now she's washing up and mixing with local people, she has an extra family and is now meeting up with them on other days so she's socialising more.”

•

“I'd say probably an increase in confidence, it has enhanced my people skills and social skills to a degree as I now feel more comfortable and confident talking to people without feeling anxious and it gave me belief in my abilities, that I am capable and able to do things. I'm in 2 jobs no, working as a support worker and doing admin work so I've come quite a long way.”

•

“Getting out of the house, it's giving me something to do, meeting other people has helped my confidence, I'm more confident now as before I was very shy. I'm in college doing a plastering Diploma and am going to have a work placement when I go back after the summer with a builder. I've done spider grams to see how much more confident I'd got and if my communication skills have improved which they have.”

‘Confidence’ is a common term in the above comments, and, in terms of ‘social skills’, confidence is by far the most common benefit identified.

©Copyright Wavehill Ltd. 2012

Page | 60

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 7.8: Social skills identified by participants 30 25 20 15

10 5 4

Communication

Reliability

0 Confidence

N=31; respondents identified more than one code in their responses

• • •

"My self confidence and self-esteem are getting there now." "It improved my confidence in applying for jobs and meeting people - everything." "It's built up my confidence immensely to know that someone wants me in a working capacity again."

7.5.2. Job specific skills The job specific skills identified by interviewees included general work experience (4), job based training (4) and other vocational experience and skills development. Figure 7.9: Job specific skills identified by interviewees 4 3 2 1

4 2

N = 14

Page | 61

Evaluation of the New Work Connections Project | Report 2: October 2012

Job-based training: •

"Gardening qualifications to help me get a gardening job."

Work experience: •

"Just the experience of working on reception."

Computer skills: •

"More confident with laptop."

7.5.3. Job search skills CV development and preparation (20) was the most common reference within interviewees’ responses in this category. However, it is apparent that support with preparing CVs actually led to a range of other benefits. This may also be because the CV development process identifies what participants are good at and helps build confidence and self-esteem by writing these skills and attributes down on paper. Figure 7.10: Job search skills identified by participants

CV preparation

Help with using a computer/internet

Interview skills

Help with applying for jobs

Help with job search

N=34 Note participants could select more than one response.

•

“Help with filling in application forms, they looked for suitable jobs for me in the paper and on the internet and did a CV for me. They helped me with my confidence as they said there was no good reason why I couldn't find a job as I came across in interviews well and had good qualifications. They made it a lot better, easier for me as before I was quite isolated and wasn't meeting other people and I did there. They helped me see what I was good at, what I enjoyed doing and printed out cards for my landscape gardening business which I started 4 months ago.”

Page | 62

Evaluation of the New Work Connections Project | Report 2: October 2012

•

“I think in a very difficult time of my life it made me realise there were other people in the same situation and stopped me feeling so isolated. It helped me recognise that I had skills that people would need and there was no need to be on the scrap heap. It helped me with my confidence, it laid it out by working on a CV and by showing me my skills and transferable skills and by talking it over with [case officer] it all helped me see the possibility of getting a job and that people wanted those kind of skills. I met quite a lot of people, one of which became a close friend. I did a Back to Work course - confidence building, interview skills etc.”

7.5.4. Aspiration and motivation The following graph shows the most prominent comments made by interviewees in relation to the ‘aspiration and motivation’ benefits identified by interviewees. Figure 7.11: Aspiration and motivation benefits identified by participants 10 9 8 7 6 5 9

4 3

2 1 1

Contacts for work Motivation to find work

Confidence

The help to be shown what you are good at

N = 22

The category of ‘being shown what you are good at’ (9) is essentially confidence building and selfesteem. Motivation to find work: •

"Getting the motivation to continue to look for work and the confidence to apply for jobs."

Confidence: •

"I had lost all confidence, (NWC mentor) has been amazing at spotting my lack of confidence and putting me on a couple of courses."

Page | 63

Evaluation of the New Work Connections Project | Report 2: October 2012

7.5.5. Gaining qualifications There is a very direct correlation between gaining qualifications and finding work. The interviewees mentioned the following qualifications gained (30) specifically within their interviews. Table 7.1: Qualifications gained via NWC as identified by interviewees Courses N Gardening courses 2 Computer course 1 Counselling skills course 1 History course 1 I.T. course 1 Back-to-work course 1 Photoshop course 1 Unspecified course 1 First-aid course 1 ECDL 1 Anatomy and physiology diploma 1 Gone to college 3 N = 30 Note participants could select more than one response. The following are examples of the comments made: • • •

“Qualifications for gardening. Help with my CV, and confidence building.” “I've been put onto courses starting in September that will get my qualifications up to speed.” “Gaining a recognised qualification (ECDL) and being dragged up to date - learning how to use a computer and they've helped me with a CV.”

7.6.

Gaps in support

Most of the participants could not think of other support outside of that offered by NWC that they felt they needed. However, 17 did make some comment which, due to their wide-ranging nature, are noted verbatim below. •

• • •

“I have said if there were any courses that were relevant to me, I'd be happy to go on them, my qualifications are out of date really. I'd also like [support office] to show me exactly what I'm entitled to in terms of help and support.” “A lump sum to start up business. But I'm aware why that wasn't possible. I get a feeling I'm too difficult.” “I'd have liked to speak to someone who knows the work place now and has a bit of compassion for the elderly workforce.” “On my first day in the garage my support worker turned up with me but only stayed quarter of an hour, I was in a totally new situation and feeling so anxious and I feel they should have stayed longer. I have not heard anything from the support worker since.”

Page | 64

Evaluation of the New Work Connections Project | Report 2: October 2012

•

• • • •

• •

•

“Everything, nothing was available though, I've lost my confidence, my mum died last year and I needed someone to help me with my confidence but the girl I was put with couldn't work with more than one person, and made me feel shut out so I left after a couple of weeks.” “I think there could have been a wider range of types of employment, though due to the current climate that's probably not their fault.” “Something like a higher level, say if people have degrees, are literate but just need a bit of confidence building.” “I would have liked to have had more counselling.” “Interview technique and confidence building courses - I wasn't eligible for funding because I wasn't on benefits, and the fact that I had jobs intermittently, [the support worker] wasn't able to put me on track.” “ wanted to find a course.” “The word work in the title, when I first came onto the project I thought it was a government scheme to try and get everyone off benefits and back into work, but it isn't. They understand that for some people just getting onto a training course is a massive step in the right direction, I think they should change the title.” “Funding for a laptop so I could look for work.” “A bit more of a push in the sense of work experience getting into somewhere just to get some kind of work. After 18 months I'd have thought they'd have got me into somewhere even Tesco/Asda stacking shelves to give me something to look forward to, to get up for. I thought they'd have had contacts with local retailers to get people into work placements.” “Financial advice, I would have liked to have seen the benefit of going back to work versus being on benefits. I think a lot of people worry that if they start a job, and then leave because they don't feel ready for it, they may end up worse off within the benefit system.” “I started with one case worker, she left and a [case officer] took over, I don't feel that I finished up getting out of it what I wanted. The level of support wasn't all that I needed as I felt there was a lack of continuity, I felt let down and wouldn't recommend anyone else to go there.” “It was meant to be a job club but no one else turned up so it was just me and [NWC support worker] there and then [NWC support worker] went off on the sick. I would have like to have had a CV put together and some ideas of applying for jobs or courses I could do.”

7.7.

Other support being accessed by participant

Most respondents (69) had not received any alternative or additional support from NWC, although three had with one interviewee being unable to answer. As shown below, the comments made by participants when asked to compare the support that they were receiving from NWC with any other support received. • • • • • • • • • • • • •

A4E Pathways - neither were helpful It was more informal than other courses Agoriad, they weren't better Rathbone, didn't get help from them either Hafod, they were awful Hesketh House were more helpful CAIS, they were better Advisor at the Job Centre Hafal, can't compare Meirionnydd's work was better Mental Health office in Holyhead, they can't be compared Job Centre Genesis, they were similar

Page | 65

Evaluation of the New Work Connections Project | Report 2: October 2012

• • •

CPN NWC are more work focused A rehabilitation place, can't remember name

7.8.

The main thing learnt which will help participants to get a job

The main benefit that the participants interviewed identified when asked to identify the main thing they learnt which would help them get a job was (again) confidence (33); this allows them to build soft skills ready to then work on finding work using the job search and employment training provided. Figure 7.12: Response to the question: Of all the support you have received from NWC, what do you think will be the main thing you learnt which will help you get a job?

Confidence

Other

Nothing

Help with CV

Mock interviews

Routine

Help writing a forwarding letter

1 0

N=91 • • •

“The fact I've got more confidence, I wouldn't go anywhere before and now I will.” “Keeping my mind confident, being positive as it's easy to think negatively when you're applying for jobs all the time and not getting anywhere.” “Now I don't expect to get into work, I suppose being the age I am I've resigned myself to the lack of opportunities there is in the job market now. On May 3rd I became an elected town councillor which has given me a different avenue.”

Page | 66

Evaluation of the New Work Connections Project | Report 2: October 2012

7.9.

Distance from the labour market

Eighty-seven of the 100 participants interviewed were unemployed at the time of the interview and they were asked a series of questions to further explore their progress towards employment. The graph below shows how near or far away from getting and holding down a job participants scored themselves as being on a range of 0 (nowhere near) to 10 (work ready). The responses illustrate the range of participants within the NWC project in this respect. Figure 7.13: Response to the question: How near or far do you personally feel you are from being able to find a job and hold that job down?

10 (work ready)

0 (nowhere near)

13 0

N=82 (5 were unable to respond)

The following graph groups the responses into three groups and is perhaps a better illustration of the point. It is interesting to note that only 24% rated themselves as far from the job market, which shows a level of optimism amongst the participants we interviewed. However, the fact that only 28% consider themselves to be close to getting a job also suggests the challenges that the NWC team face in terms of progressing participants into employment.

Page | 67

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 7.14: Response to the question: How near or far do you personally feel you are from being able to find a job and hold that job down? Grouped and by percentage 48%

50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0%

28% 24%

Far

Middle

Near

N=82

The interviewees in question were subsequently asked to describe what they thought they needed to do to get to a stage where they could get and hold down a job. Interestingly, the main requirements identified were not employment related, but, confidence (25%) and better health (15%). This again shows the demand / need for and value of the soft skills development and how participants need self and social confidence before they can progress towards the employment related skills and training. However, the fact that 16% did not know what support they required to enable them to move closer to the labour market is also an indication of the challenge that NWC faces. Figure 7.15: Response to the question: What do you need to do to get to the stage where you are able to find and hold down a job? 30% 25% 25% 20%

19% 16%

15%

14%

10% 5%

N=80

Page | 68

Evaluation of the New Work Connections Project | Report 2: October 2012

Confidence: • •

"More self-confidence and confidence with my computer skills." "A bit more confidence before I can get a proper job, I'm okay doing volunteering."

Ready now: • •

"I'm ready now, just finished a 3 month work contract." "I am ready to get a job; it's just someone giving me a chance."

Better health: • •

"My health is the main problem." "My depression has got worse since seeing [NWC worker]. I need to sort out my depression.”

7.10. Ways in which participants were helped to get a job Thirteen of the interviewees were employed at the time of the interview and they were asked to explain how, if at all, NWC had helped them to get that job. Yet again, confidence was a prominent comment made. Figure 7.16: In what way did NWC help you get the job you are in now? 8 7 6 5 4 7 3 2

2 1

They didn't

Volunteering

0 CV help

Confidence

Got me a job

N=15; respondents could provide more than one answer.

CV: •

"CV really helped me get the job."

Confidence: •

"Just gaining confidence, personal self-confidence and confidence in my ability."

Page | 69

Evaluation of the New Work Connections Project | Report 2: October 2012

They didn't: •

"They didn't as I didn't benefit from going there, I got this job myself."

Of the 13 participants who had employment, four had full-time jobs and nine part-time jobs. Nine were considered by the participant in question to be permanent jobs, one temporary and three seasonal. The length of time in employment ranged from one participant who was just about to start their job, to 14 months (table 7.2). The key question is obviously how many of the participants that NWC supports into employment remain in employment for a significant period of time and this is an issue that the evaluation will explore further in future reports. When the participants were asked how confident they were that they would be employment in three months’ time, eight were very confident, leaving five who were not. Table 7.2: The length of time which interviewees had been in employment Period Start in a week 2 weeks 1 month 3.5 months 4 months 5 months 6 months 8 months 12 months 14 months

N 1 1 1 1 2 2 2 1 1 1

N=13

Page | 70

Evaluation of the New Work Connections Project | Report 2: October 2012

7.11. Would participants recommend the project to others? Recommendation of any product or service to family and friends is a very strong indicator of satisfaction and, as the graph that follows shows, the majority of those interviewed would indeed do so in the case of NWC. Figure 7.17: Response to the question: Would you recommend NWC to your.... 100 90 80 70 60 50 40 30 20 10 0

Yes

Couldn't Answer

N=100

The interviewees were subsequently asked to give a reason why they would recommend the project. As shown in the following graph, by far the most frequent reason for making a positive recommendation was the NWC ‘case worker’ (39%) which is an indication of the key role that the case worker / support worker play within the project. Figure 7.18: Comments made by interviewees when asked to explain why they would (or would not) recommend NWC to others

Worker was good/helpful

39%

Good experience

18%

Other

12%

Didn't work

12%

Couldn't answer

Have already

Socialising

Confidence

4% 0%

10%

20%

30%

40%

50%

N=85 Note participants could select more than one response.

Page | 71

Evaluation of the New Work Connections Project | Report 2: October 2012

Worker was good: • •

"Yes definitely, (name) has been great." "They were very helpful, I'm sure if someone didn't have the problems I have NWC would be very good."

Good experience: • •

“Because they are open and so gentle with you, there's no push to finish things they just let you get on with things, they're so kind.” “Very much because if you're in a situation where you haven't worked for a long time it's very difficult to get on the ladder, very difficult to get a job, employers see you as a basket case and you can't do it without support. They've been there for me, given me the realisation and knowledge that other people are going through this and it's not unusual."

Didn't work: • •

"Maybe I was unlucky with the changeover of support workers but I felt let down." “Because they're unapproachable in a physical way, I haven't got the phone number and tried going there to see someone but when I got there no one would see me.”

7.12. Three words to describe the project As a means of summing up their views on the project the interviewees were asked to note three words that they would use to describe NWC. One hundred and seventy seven words were used the most common of which were: • • • •

Helpful (24) Supportive (18) Friendly (15) Positive (12)

A ‘word-cloud’ is a useful way of visualising this kind of data; the clouds give greater prominence to words that appear more frequently in the source text. The cloud on the following page illustrates the words that participants used to describe NWC 20.

The word-cloud was produced using Wordle: http://www.wordle.net/

Page | 72

Evaluation of the New Work Connections Project | Report 2: October 2012

Figure 7.19: Word-cloud illustrating the words used by participants to describe NWC

Page | 73

Evaluation of the New Work Connections Project | Report 2: October 2012

7.13. Final comments made by interviewees. At the end of the interview, participants were given the opportunity to make any other comments about the project. Thirty-six respondents took the opportunity to do so and the comments made underline the key role that the support workers / case officers play within the project. • • • •

• • • • • • • • • • • • • • • • • • • • •

• • •

“Just finding out what is available for me has been great, I don't know how these different agencies interact with each other.” “NWC and the world works in pigeon holes and I don't fit into any pigeon hole, so life can be difficult.” “I hope it never closes.” “The ladies were very good, they've got a difficult job to do, the first time I met with (name) I had to meet her in the social services offices with a camera in to make sure I was safe to be alone in a room with.” “It's one of the best things that has happened to me in the last 6 years as I thought I'd never work again.” “He does enjoy it, and is quite happy when he returns from a day out.” “Keep going as it does work for some people if you have the right support worker.” “I think it's filled my life, it has been a great help to me and helped me to realise that I still have a place and something to offer.” “They really try to find you something that you'll be able to do and there's a lot of choice of what you can do there.” “I've been to loads of organisations who were meant to be helping people find work and always had to do it myself in the end as they didn't.” “They were absolutely brilliant.” “I think out of all the official people I've seen (support worker) was the nicest, the most helpful and understanding.” “It's well worth it for the work that they do.” “I've recommended them to everyone I've met who is struggling to find work as they are so good at what they do and so kind and supportive.” “They're friendly there.” “I wish I could walk around town with a sandwich board promoting the Woman's Centre and everything they do there.” “I think it's a good project for people who are on their own and need support to help them gain confidence and meet more people.” “I'm much happier than I've ever been before.” “I just hope that it continues and that as many people benefit from it as [participant] has. “I found them really helpful and supportive of my situation.” “Just keeps it going.” “I think they are brilliant, they've done their very best for me.” “It's hard to comment because I've only seen [support worker] once since finishing college, as we'd decided I wouldn't see him until after I'd finished my course.” “I just thought that it was really good and beneficial to me and I think that other people could benefit from it as well.” “It's just great and the people are nice, very helpful, they actually come out to you too, she (support worker) dragged me out when I first went to the project because I didn't want to go, I had no confidence, she took me in the car and if she hadn't have done that I never would have gone.” “(Support worker) the tutor is very kind and helpful.” “They are just all really wonderful people.” “They've been as helpful as they can, there just aren't any jobs out there.”

Page | 74

Evaluation of the New Work Connections Project | Report 2: October 2012

• • •

• • • •

“It's helping, he loves going and it's respite for me as I (his wife) can get on with loads of things while he's there.” “I'm full of praise for (NWC support worker).” “I know it was only a short time I was with them but it was good and if circumstances - my husband's health had of been different I'd have still been there, I would like to go back.” “They'd suggest things and I wasn't panicking as they weren't pushy, it was just nice and a bit of respite for me to go there.” “It's a chance of a lifetime for me to get this knowledge; I wouldn't know where to go to learn about computers if I didn't go there.” “I think it is worthwhile and they're doing a good job.” “I'd like to say (support worker) is really helpful and polite.” “I think it's amazing and he's happy to go, it gives him a structure and a timetable.”

Page | 75

Evaluation of the New Work Connections Project | Report 2: October 2012

8. Review of literature on the link between improved confidence and employability Section summary This brief literature review has shown that soft skills and particularly self-efficacy are a key element of the generic or cross cutting skills employers are looking for. Such skills also facilitate individuals in their application of other practical or technical skills they may possess, but this is only possible where improved confidence levels can be seen to have improved the self-efficacy of the individuals involved.

8.1.

Introduction

There is reference throughout the previous section to the significant impacts participants are identifying the support as having upon their confidence levels. In order to explore this further we have conducted a literature review to identify whether there is evidence of how such increases in confidence may be contributing to improved employment prospects for these beneficiaries. The literature highlights two key areas where such changes may result in positive outcomes for beneficiaries from their improved confidence. Firstly, related to the skills and competencies that potential employers may be looking for when recruiting staff (what may be termed the ‘demandside’), and secondly, changes such improved confidence levels may bring in the beneficiaries which improves their potential employability (or the supply of skills and competencies that employers are seeking).

8.2.

Employer demands for skills and competencies

The National Strategic Skills Audit for Wales published in 2011 by the UK Commission for Employment and Skills (UKCES)21 is the latest published national strategic skills audit for Wales. Drawing on the latest employer survey information 22, it identifies detail of the skills challenges and needs of employers across Wales. Whilst this evidence highlights specific skills needs related to what it terms ‘hard skills’ (such as specific technical skills in IT, software package abilities, multi-media package skills like Photoshop, and engineering or specialist science skills), the audit particularly stresses the overriding importance of what it terms generic / cross cutting skills. 21

See UKCES (2011) ‘Skills for Jobs: National Strategic Skills Audit for Wales 2011 Vol 1’ UKCES, Wath upon Dearne, June 2011. To download a copy go to: http://www.ukces.org.uk/assets/ukces/docs/publications/nssawales-2011-executive-summary.pdf . See also Vol 2 for the data which underpins the audit document at: http://www.ukces.org.uk/assets/ukces/docs/publications/nssa-wales-2011-evidence-report.pdf . 22 The last Future Skills Wales employers survey took place in 2005. The report extrapolates these findings using trends noted from the 2010 UK National Employers Skills Survey and DCELLS 2011 Employers survey UKCES (2011) ‘Skills for Jobs: National Strategic Skills Audit for Wales 2011 Vol 1’ UKCES, Wath upon Dearne, June 2011. To download a copy go to: http://www.ukces.org.uk/assets/ukces/docs/publications/nssa-wales2011-executive-summary.pdf . ©Copyright Wavehill Ltd. 2012

Page | 76

Evaluation of the New Work Connections Project | Report 2: October 2012

As the report makes clear that it does not want to underplay: “the importance of generic / cross-cutting skillsets that are critical to employability and effectiveness in the workplace. Skills such as customer handling, problem-solving and teamworking are pervasive across the economy and current and future requirements in this area, in combination with technical skills demands, are critical to future competitiveness and productivity.” (UKCES 2011, p.723) Indeed, the audit highlights that Welsh evidence shows that shortages of applicants with the relevant levels, or deficiencies in existing staff, in problem-solving, customer handling, and communication skills are as at least as prevalent as specific practical or technical skills. Furthermore, such skills are important features of the jobs available in the low level occupations traditionally filled by those entering the labour market from unemployment, particularly across the retail, hospitality, and social care sectors. These represent skills areas that the Chartered Institute for Personnel and Development (CIPD 24) term ‘soft skills’ which in a 2010 report – Using the head and heart at work: the case for soft skills25 – sought to define what soft skills were, how they could be assessed by employers, and be developed over time. In particular, the CIPD work identifies why its members (the main audience of the report) should pay particular attention to ‘soft skills’ stating that: “A distinctive feature of soft skills is that they facilitate the effective and efficient application of hard skills in workplace settings.” (CIPD 2010, p.14). It is the context in which the report goes on to identify soft skills in terms of three components where they apply to work/employment that is important in considering the role that improved confidence may play in moving individuals into, or towards, employment. The three components are focused upon the individual themselves (Self); the activities they complete whilst employed (Tasks); and their engagement with others, whether customers or work colleagues (People). The reason these are important is because the CIPD work emphasises the critical role played by not only the confidence of the individual involved, but also the related characteristics of self-esteem, motivation, and aspiration. The report makes clear that the changing nature of work where there is an ever greater emphasis upon interconnection and relationship development across all areas of economic activity means that there are increasingly fewer occupations where interpersonal and related soft skills will not be required, and therefore will remain a key consideration of employers when seeking to recruit staff.

UKCES (2011) ‘Skills for Jobs: National Strategic Skills Audit for Wales 2011 Vol 1’ UKCES, Wath upon Dearne, June 2011. To download a copy go to: http://www.ukces.org.uk/assets/ukces/docs/publications/nssa-wales2011-executive-summary.pdf 24 The CIPD is a world leading body offering chartered status to HR and Development professionals from across the world. It has developed a longstanding reputation for producing leading research on the needs of employers when recruiting having access to over 135,000 HR professionals worldwide. 25 CIPD (2010) ‘Using the Head and Heart: the case for Soft Skills’ London. See http://www.cipd.co.uk/NR/rdonlyres/18616949-CF66-47F8-A0887C8FF2D864E0/0/HeadandheartguideFINAL.pdf for copy to download. ©Copyright Wavehill Ltd. 2012

Page | 77

Evaluation of the New Work Connections Project | Report 2: October 2012

The CIPD also note that the development of what it terms ‘less tangible outcomes’ from training or development activities (e.g. less likely to be/or able to be converted into monetary values) in improved confidence, motivation and self-esteem levels can make a significant contribution to the achievement of more measurable outcomes such as securing a job. This final point takes us to the impact that improved confidence levels may have on the individuals involved and their potential to ‘supply’ the kinds of skills and capabilities prospective employers may be looking for when recruiting staff.

8.3.

How improved confidence can improve the employability of an individual

In a range of previous evaluations of programmes working with a similar ‘hard to reach’ or ‘hard to help’ target group conducted by Wavehill, our findings have also shown similar impacts on client confidence levels. What the literature highlights in this area is that such confidence improvements enhance the employability of the individual involved because it improves what may be termed as their self-efficacy. Self-efficacy is a concept developed by Bandura (1997)26 which refers to an individual’s judgement about their skills to successfully perform an activity – for instance completing a job application, job interview, or undertaking job-search activities. This concept was used in a 2011 study by the Institute for Employment Research at the University of Warwick 27 for the Department of Work and Pensions on the job-search activities of benefit claimants. It particularly highlighted the importance of the self-perceptions and self-efficacy of benefit claimants (a client group similar to those participating in the NWC project) in conducting job search activity, but also in achieving a successful outcome through that activity in securing employment. The critical aspect in this and why it is related to confidence improvements noted in NWC beneficiaries is because: “While possessing the skills necessary to fulfil a particular job may be a prerequisite for a successful job ‘match’, so an individual’s judgements and expectations about their capability to perform effectively at each stage of the job-search process are important in getting a job.” (Green et al. 2011, p.4, our emphasis). Consequently, our finding that confidence levels have improved amongst NWC clients means they are therefore tackling their own judgements and expectations about their capabilities and thus increasing their ability to find employment following their work with the project. Green et al (2011) provide particular support for this view, following Bandura (1997), showing how self-efficacy varies on three scales (1) Level; (2) Strength; and (3) Generality. Consequently, low levels of self-efficacy may mean that an individual does not believe they can find a job or even participate in the NWC project. The strength of efficacy beliefs will influence how much ‘support’ or ‘outreach’ might be needed to enable that individual to conduct job search or join the project, whilst the individuals efficacy will also influence the breadth or generality of situations in which they will apply their skills. For instance how can the existing skills of the unemployed or disengaged be applied more widely to other situations such as the world of work.

Bandura, A. (1997). Self-efficacy: The Exercise of Self Control. New York: W. H. Freeman and Company. Green, A. de Hoyos, M, Li, Y and Owen, D. (2011) Job Search Study: Literature review and analysis of the Labour Force Survey. DWP London. Available at: http://research.dwp.gov.uk/asd/asd5/rports20112012/rrep726.pdf 27

Page | 78

Evaluation of the New Work Connections Project | Report 2: October 2012

Green et al (2011) highlight how improved confidence levels will help individuals to have a stronger belief in their own abilities to overcome any obstacles or barriers preventing their search for employment. This is because as Green et al. highlight anxiety or a lack of confidence in individuals can be both a cause and co-effect of low levels of self-efficacy, therefore improved confidence levels are likely to improve an individuals’ belief in their own capabilities and perhaps be able to express these more clearly to a potential employer. This is relevant in the noted case because of the ways in which self-efficacy is identified by individuals, approaches which closely match the key areas of support provided to NWC clients. Green et al. identify that there are four main ways in which individuals are able to recognise their own self-efficacy and these have particular parallels to the ways in which support is provided on the NWC project. The first area is through experiences of successful performance which reinforces and develops expectations amongst those individuals that such success will be achieved in the future as might be experienced through completing a programme of work experience or mock job interviews. Furthermore this can also operate in the opposite direction with experiences of repeated failure tending to lower self-efficacy and thus expectations of performance – for example multiple job application failures. The second area comes through observing others performing successfully such that work with role models can help individuals to believe they possess similar capabilities and therefore improve their level of self-efficacy. This is supported by a third area such as verbal persuasion or social support can help improve an individual’s view of their ability to perform a particular action or activity. As such this can be achieved through the support a mentor or key worker might provide through verbal support for an individual whilst undertaking work experience, or conducting job-search activities. The final area relates much more closely to the individuals perception of their emotional state as this is likely to have an impact of how that individual perceives their skills and capabilities. Consequently, individuals lacking confidence, self-esteem, motivation, or experiencing anxiety are likely to have lower levels of self-efficacy, but are also likely to perceive that their ability to use their skills and capabilities maybe declining reinforcing the feelings of low confidence and/or anxiety. Consequently, improved confidence levels are likely to contribute to each of these areas enabling the self-efficacy of those individuals to be raised and therefore because of the support being provided move them closer to the labour market.

8.4.

Conclusion

This brief literature review has shown that soft skills and particularly self-efficacy are a key element of the generic or cross cutting skills employers are looking for. Such skills also facilitate individuals in their application of other practical or technical skills they may possess, but this is only possible where improved confidence levels can be seen to have improved the self-efficacy of the individuals involved. Consequently, what improved confidence most particularly demonstrates is the improved judgements and expectations of the project beneficiaries in their own capability to perform effectively as they move towards, and / or into employment.

Page | 79

Evaluation of the New Work Connections Project | Report 2: October 2012

9. Emerging findings and recommendations 9.1.

Introduction

This final section of the report draws together the emerging findings of the evaluation and makes a number of recommendations based on those findings. It is split into three broad headings: • • •

Expenditure, outputs and results to date; Management and delivery of the project; and Outcomes of the support provided.

9.2.

Expenditure, outputs and results to date

The evaluation has found that, for a number of reasons including delays in approving the re-profiled expenditure forecasts on the part of WEFO, NWC has progressed slower than forecast in terms of its expenditure; claimed expenditure to the end of May 2012 shows that under 30% of the approved budget has been utilised (£5,999,527 / £20,468,700). Taking into account the fact that claimed expenditure is likely to lag behind actual expenditure due to the needs to ensure that all the necessary audit evidence has been collected, this is still lower than would have been anticipated at this stage in the lifetime of the project. A strategy for addressing this need to be considered and there are two basic options; increase the rate of expenditure or reduce the forecast and hence the budget of the project. For a number of reasons (some of which have been outside the control of the project) NWC has also made limited progress to date in terms of a number of its targets for outputs and results. The number of participants engaged to date is 44% of the target for the lifetime of the project (1,983 / 4,555); this is as per the latest profile submitted to WEFO but there are variances amongst the four local authority areas. The ‘conversion rate’ (participants into results) to date is however lower than will be necessary to achieve the targets for results. For example, to achieve the target for the number of participants entering employment (896), a conversion rate of 20% is required (896 of 4,555); the current conversion rate is 4% (70 of 1,983). Whist we would anticipate that this rate will increase as participants spend more time within the project, the risk of not achieving the targets as set needs to be considered. The fact that the project is working with participants who need a significant amount of support over a significant period of time before achieving a positive outcome clearly needs to be taken into account. It is also important to recognise that the approach in each local authority area is different which makes direct comparison of measures such as cost per participant difficult and possibly somewhat misleading. But, the substantial differences between the forecast costs per beneficiary in the different areas needs to considered as part of any review with a view to ensuing that the forecast in each area is realistic and achievable within the lifetime of the project.

Page | 80

Evaluation of the New Work Connections Project | Report 2: October 2012

Recommendation 1. It is recommended that a review of forecast / profiled project expenditure, outputs and results is undertaken with a view to ensuring that the targets set are realistic and achievable within the lifetime of the project. This review should take into account the fact that economic conditions continue to be challenging. The focus of the project on those who are furthest away from the labour market also needs to be taken into account and reviewed. If the conclusion is that the current targets are not achievable, two potential actions should be considered (in consultation with WEFO): a) Reducing the targets so that they are achievable and realistic whilst continuing to work with the same client group; and/or b) Devising a strategy whereby the project will target specific participants with a view to achieving the targets that have been set.

9.3.

Management and delivery of the project

Page | 81

Evaluation of the New Work Connections Project | Report 2: October 2012

The fact that the project will inevitably experience a ‘winding down’ period as it moves towards the end of its lifetime unless there is a clear plan in place to continue the service beyond its current funding, needs to be considered during the coming months (including as part of the review of targets recommended above) as it has a number of important implications. For example, there is a risk that the staff who are so vital to the success of the project will move on before it comes to an end and that it will not be possible to replace those staff; this will impact on the delivery of the project and what can be achieved during the latter stages. NWC staff have also emphasised the fact that in many cases individuals being supported by NWC are being taken on a ‘journey’ towards work that will, for many, take a long time; longer that the lifetime of the NWC project in many cases. As noted frequently throughout this report, the individuals concerned are also often those furthest away from the labour market and change (more specifically, a withdrawal) in the support provided could lead to progress that has been made being lost therefore reducing the net and long term impact (and hence cost-benefit) of the project. Strategies for managing and reducing these risks need to be considered. The administrative burden on project staff was a constant theme in the ‘pause & review’ sessions and discussions with staff. This is despite the fact that it is clear that substantial effort has been made to try and introduce measures to minimise that burden or at least explain to staff why it is in place. The obvious recommendation is that the mechanisms that are in place therefore need to be reviewed and, indeed, that is a recommendation made below. It is however important to note and take into account the fact that, in many cases, this is the first time the staff concerned have been involved in the delivery of an European funded project. Accordingly, they are not familiar with such and administration / monitoring processes and this is one of the main reasons why they are uncomfortable with it. It is important to acknowledge that many of the changes to the administrative processes that have been introduced as the project has progressed are due to changes introduced by WEFO; for example, changes to the definitions of performance indicators. It is not, therefore, something that the project could control. Those changes have however been challenging from a project management perspective. Recommendations 2. We support the work that is already ongoing to develop ideas, options, proposals and plans for a successor to the NWC with a view to minimising the ‘transition period’ between the current and subsequent project / support mechanism. This process should continue to be prioritised. 3. NWC management should review the mechanisms in place to facilitate networking and sharing of experiences amongst NWC staff across both on a region basis and on a local level. Where appropriate, developments to the NWC service provided in the individual local authority areas should be encouraged with a view to developing a more consistent approach across the region. 4. Building on the work that has already been undertaken to date and the recommendation above, NWC management should consider the potential to establish a working group that brings together staff from across the project to review the project administrative process and, if possible, identify areas where further improvements could be made without compromising the need to meet audit requirements as set out by WEFO.

Page | 82

Evaluation of the New Work Connections Project | Report 2: October 2012

9.4.

Outcomes of the support provided

The interviews with participants found that they understand the focus within the project on getting people into work and there is evidence of a positive impact in that respect. However, the primary benefit of NWC apparent to interviewees was the ‘stability’ it brings to their lives and specifically the skills and opportunity to socialise and ‘get out of the house’. The confidence that participants develop as a result of their participation in the project was also a common theme within interviews. The themes that emerge can in fact be considered as having two dimensions: • •

A primary dimension of social development A secondary dimension of increasing employability

This is consistent with the emerging findings of the analysis of soft outcomes data; it found that 80% of the participants for whom data was available at the time of writing (255) had experienced positive outcomes on average across the seven dimensions of the Work Star: • • • • • •

Job-specific skills Aspirations and motivations Job-search skills Stability (this measures day-to-day stability of an individual’s life and takes into account factors like debt, alcohol or drug use and housing issues) Basic skills (e.g. literacy, English, numeracy) Social skills for work (working with others, self-confidence, timeliness, work attire)

From the participants perspective, it is clear that both social and employment ‘dimensions’ are important and positive outcomes of their participation in NWC. And the strong suggestion is that the employability dimension could not exist without the personal and social development dimension and that this dimension compliments, or even is essential for employability outcomes. In other words, it would not be possible to achieve employment outcomes without also achieving social development outcomes. That a literature review undertaken supports this view. Analysis of the 100 interviews with participants also further underlines the wide range of participants that NWC supports; ranging from those who are relatively close to the labour market to those who need a considerable amount of support before they could be considered to be ‘work ready’. This is further evidence of the fact that the project is working with some of those who are furthest away from the labour market and, hence, the most challenging in terms of achieving positive employment related outcomes, especially in the short term. Recommendation 5. NWC management should continue to take steps to ensure that soft outcomes (i.e. Work Star) data is collected from as many participants as possible so that the data can be fed into the evaluation process. 6. The evaluation should continue to undertake interviews with a sample of NWC participants that will explore both the social and employment outcomes of the support being provided. The evaluation will also continue to explore the link and relationship between ‘social’ and ‘employment’ outcomes.

Page | 83

Evaluation of the New Work Connections Project | Report 2: October 2012

Appendix 1: Local Authority level budget and targets Overall budget and targets Total Project Cost ESF funding Match funding

Budget £20,468,700 £10,747,678 £9,721,022

Source: NWC Regional Team

Results

Outputs

Performance indicators Total Participants (economically inactive and unemployed)

Original

Revised

5,248

4,555

1,034 2,875 788 2,668

896 2,553 907 4,056

20% 56% 20% 89%

290

72%

Employers assisted or financially supported Participants entering employment Participants gaining qualifications Participants entering further learning Participants gaining other positive outcomes Participants receiving support with caring responsibilities Employers adopting or improving equality and diversity strategies and monitoring systems

Results as a % of outputs

Source: NWC Regional Team

Anglesey Total Project Cost Match funding ESF funding

Funding £5,184,643 £2,591,661 £2,592,982

Source: NWC Regional Team

Results

Outputs

Performance indicators Total Participants (economically inactive and unemployed)

Target 1,229

Results as a % of outputs

Employers assisted or financially supported

248 700 243 927 0

20% 57% 18% 75% 0%

50%

Source: NWC Regional Team

Page | 84

Evaluation of the New Work Connections Project | Report 2: October 2012

Gwynedd Total Project Cost Match funding ESF funding

Funding £3,431,992 £1,775,083 £1,656,909

Source: NWC Regional Team

Results

Outputs

Performance indicators

Target

Total Participants (economically inactive and unemployed) Employers assisted or financially supported Participants entering employment Participants gaining qualifications Participants entering further learning Participants gaining other positive outcomes Participants receiving support with caring responsibilities Employers adopting or improving equality and diversity strategies and monitoring systems

1,226

Results as a % of outputs

19 236 686 254 1,209 36

19% 56% 21% 99% 3%

53%

Source: NWC Regional Team

Conwy Total Project Cost Match funding ESF funding

Funding £4,784,861 £2,447,835 £2,337,026

Source: NWC Regional Team

Results

Outputs

Performance indicators

Target Results as a % of outputs

Total Participants (economically inactive and unemployed)

480

Employers assisted or financially supported

98 167 160 300 24

20% 57% 18% 75% 5%

66%

Source: NWC Regional Team

Page | 85

Evaluation of the New Work Connections Project | Report 2: October 2012

Denbighshire Total Project Cost Match funding ESF funding

Funding £7,067,205 £2,906,442 £4,160,762

Source: NWC Regional Team

Results

Outputs

Target 1,620

Results as a % of outputs

26 314 1,000 250 1,620 230

19% 62% 15% 100% 14%

50%

Source: NWC Regional Team

Page | 86

wavehill t: e: w:

ymchwil gwerthuso arolygon research evaluation surveys

01545 571 711 info@wavehill.com www.wavehill.com

Head office (registered office): 21 Alban Square, Aberaeron, SA46 0DB Other offices: South Wales office: 28 The Balcony, Castle Arcade, Cardiff, CF10 1BY West England office: Unit 5.2, Paintworks, Arnos Vale, Bristol, BS4 3EH East England office: 29&30 Parman House, 30-36 Fife Road, Kingston, Surrey, KT1 1SY