Ex-post Evaluation of the LEAD Programme
Report prepared by Colin Wren and Jonathan Jones Economics, Business School, University of Newcastle upon Tyne Newcastle upon Tyne, UK, NE1 7RU
Ex-post Evaluation of the LEAD Programme Executive Summary The LEAD programme – Leading Enterprise and Development – seeks to develop the leadership and management abilities of smaller-sized enterprises in order to promote business development and growth. The focus of this evaluation is on the business effects of LEAD on the operations and outcomes of participant firms. A questionnaire survey of participants was administered in 2006, shortly after programme completion, from which 42 responses were obtained from the 65 LEAD participant firms. The principal findings of the ex-post evaluation are summarised as follows:
The LEAD programme has had strong effects on business outcomes, which have been driven by changes to the business operations of participants.
The major effects on operations are that it has allowed firms to develop a more strategic approach and improve the general management of the business.
Half the firms either have or plan to undertake other improvements, such as the development of a business plan or an improved service, and a third have appointed a business or operations manager.
All firms say LEAD played a role in these changes, with around 60% reporting that they depended “absolutely” or “to a great extent” on the programme.
About 90% of firms expect an increase in sales, employment, productivity or profits in the next 2 to 3 years, with no firm expecting none of these to increase.
The aspects of LEAD that were “particularly important” in promoting these outcomes were the master classes, coaching and peer interaction.
The firms reported strong increases in the volume of sales, but modest increases in the level of employment, leading to strong increases in productivity growth.
The sources of increased productivity growth (i.e. sales per employee) were organisational change and a better use of existing employees.
An independent analysis of gross value added per employee gives results that are consistent with the increase in productivity reported by the firms.
On the specific changes to business outcomes, the results are as follows: Change in sales turnover: •
About half of those firms reporting an increase in sales (actual or expected) said it depended substantially on the programme.
The median and modal annual sales increases due to LEAD were in the range £100,000 to £250,000. It compares with average annual sales turnover for the firms of around £1 million.
On average, the increase is about £200,000 per annum. On a pessimistic view that LEAD was completely ineffective in the non-respondent firms, the mean annual sales increase due to LEAD is £130,000 a year across all 65 firms.
Change in employment •
Most firms expected the net number of employees to increase by only 3 to 5 jobs, which compares with a mean employment level in firms of about 20 jobs.
Change in productivity •
32 of the 42 firms expected productivity (i.e. sales per employee) to increase over the medium-term, with two-thirds estimating it at more than £5,000 per annum.
Gross value added (GVA) per employee •
On average, there was a modest annual increase in GVA per employee, but the typical change is in the range £0 to £20,000. The mean figure is reduced by a small number of firms where there was a drop in sales, but static employment.
The conclusion of the evaluation is that the LEAD programme has had substantial effects on business outcomes (achieved or expected), and that these outcomes have been induced by changes to business operations, which are due to the programme. While some aspects of the LEAD programme appear more important in generating these changes, the overall conclusion is that the programme has been successful in achieving its objective of promoting business development and growth.
Ex-post Evaluation of the LEAD Programme
The LEAD programme – Leading Enterprise and Development – seeks to develop the leadership and management abilities of the owner-managers of smaller-sized enterprises in order to promote business development and growth. The programme commenced in October 2004 and concluded in spring of 2006. It operated across the northwest region of England, and was funded by the Northwest Development Agency. It was developed and run by the Institute for Entrepreneurship and Enterprise Development (IEED) at Lancaster University. The purpose of LEAD is to promote leadership through a range of learning methods (e.g. master classes, away days, action learning, workshops, shadows and consultancy) in order to drive business development and growth. The programme encourages participants to relate what they have learnt to their business, and it offers support through coaching, mentoring, consultancy and an electronic forum. Ultimately, the LEAD programme has sought to improve the ‘leadership’ capabilities of these ownermanagers in order to develop business operations and to promote growth. In total, around 67 businesses completed the LEAD programme in four sets of roughly equally sized cohorts. Cohorts 1 and 2 commenced in October / November 2004 and graduated about a year later, while Cohorts 3 and 4 started in May / June 2005 and completed in April 2006. The participants were extensively monitored at each stage on aspects of the programme, including feedback sheets and a post-course questionnaire. It led to the production of a LEAD Interim Narrative Report by the IEED in October 2005. This report was formative and qualitative in nature, finding that the programme had led to increased confidence in the leadership of participants, changes in business operations and likely increases in business sales. The IEED evaluation methodology also requires a summative assessment of LEAD to be carried out independently and after programme completion. This ex-post study was commissioned in March 2006 and carried out over the subsequent months, for which this represents the final report. The brief was to focus on the business effects of LEAD, i.e. operations and outcomes, including sales turnover,
employment, productivity and gross value added. For Cohorts 1 and 2 some effects may have been realised, but for others these are more likely to be prospective in nature. To help determine the actual and likely business effects of the LEAD programme the primary method of evaluation is a questionnaire survey of participant firms, and it is these results that are reported here. Attempts were made to match the information from this questionnaire with the monitoring and other data collected by IEED, but this proved to be of limited use due to the difficulties of matching of data, which were often collected on an anonymized basis and resulted in much smaller sample sizes. As such, this report concentrates on the results of the survey, but with the advantages that it is able to directly tackle the brief of examining the business effects of LEAD, and using a consistent sample of survey respondents.
Nevertheless, where possible, other data were matched to this,
including data collected by the IEED on Gross Value Added (GVA), which enables the change in GVA per employee to be examined.
Report Structure The structure of this report is as follows: Section 2: Questionnaire Survey, including the response rate and representativeness of the sample. Section 3: Participation in the LEAD Programme, including business plans and experience of LEAD programme. Section 4: Overview of Business Effects. Section 5: Business Outcomes, including sales, employment, productivity and GVA per employee. Section 6: Factors Underlying Programme Success. Section 7: Conclusions.
The LEAD Interim Narrative Report provides useful analysis and information on the nature of the participants and feedback on the LEAD programme components. It also has information on the effects of the programme on the individuals and businesses, including
employment and sales. However, to explore the business effects of LEAD it was clear that it was necessary to conduct a survey of participants. A short self-completion postal questionnaire was agreed with the IEED staff. This is given at the Annex, and reference is made to it throughout the report in the table notes. The LEAD Director, Ms Sue Peters, primed the LEAD participants in advance of the questionnaire, which was distributed to participants by post in early April 2006. For non-responding firms it was twice followedup by telephone at fortnightly intervals. In total, 42 useable responses were obtained. All the respondents had directly participated in the LEAD programme. Of the 67 LEAD participants, two individuals were excluded from the survey, as another person at these firms had also participated in LEAD, while two firms responded in such a way that their forms could not be included.1 It gives a response rate of around two-thirds (i.e. 42 / 65), which is very good. Table 1 shows the response rate is even across the four cohorts, except for Cohort 1. The reason for this could be because this cohort completed the LEAD programme a while ago (a 40% response rate is generally the norm), although it does not explain the much better response of Cohort 2. It could also be that the LEAD programme was relatively new and untested for these participants, and modifications were made to programme in the light of the interim evaluation.
Table 1: Survey Response Rate by Cohort Cohort Number of firms Number of responses Response rate
1 13 5 39%
2 16 11 69%
3 17 12 71%
4 19 14 74%
In the analysis we present the results for all survey respondents, while in Section 6 they are considered separately for Cohorts 1 / 2 and for Cohorts 3 / 4 in order to explore the factors underlying the success of the programme. The latter two cohorts may be more representative of LEAD when it is â€˜up-and-runningâ€™, for which there is a response rate of 72%, although of course there is relatively very little time between completion and the time of the questionnaire survey for these two cohorts. The representativeness of the sample is examined by comparing the employment levels of the respondents and non-respondents at the time of application. This is shown 1
One response was incomplete and the other was returned after the cut-off date. Both were for Cohort 1.
in Table 2 (it includes full-time and part-time employees). It shows the respondents are larger in size, so that none of the non-respondents has more than 25 employees, while 4 of the respondents have more than 50 employees.2 Ideally, the businesses participating in LEAD should have 4 - 20 employees, and Table 2 shows that this is satisfied, with both the respondents and non-respondents generally in the size range of 6-25 employees.
Table 2: Employment Size of Respondents Employment size group Respondents Non-respondents
1-5 15% 9%
6-10 24% 35%
11-25 43% 56%
26-50 9% 0%
50+ 9% 0%
Total 100% 100%
Finally, the interim evaluation shows that the vast majority of LEAD participants are in the service sector, and this is no different to our sample. Figure 1 shows the number of firms by activity for both the LEAD participants and respondent firms. The groupings are only roughly drawn, so that it is possible that a firm could fit under several headings, but it gives a good indication of the kinds of firm participating in LEAD and the survey. For example, the number of firms in manufacturing is small (these are mainly in engineering), but it is possible that food and drink or the digital industries could involve manufacturing,
Figure 1: Activity of LEAD and Survey Participants 15
16 14 12 10 8 6 4 2 0
13 8 6 3
ia nc na Fi
li ita ig D
3 3 1
ce s rv i se
ss Bu sin es
up po r
ng tu ri
an uf ac M
One business in contract cleaning reported employment of 500, including 450 part-time employees.
although the vast majority of firms are service based. The main activities are business support (e.g. training, recruitment, marketing, cleaning, etc) and consumer services (e.g. estate agents, car repairers, garden centres, security systems and so on). In addition, there are businesses in the digital and creative industries, financial and professional services and tourism. The food and drink grouping includes a number of farm-based businesses that may also service tourism (e.g. tea shops).
Participation in the LEAD Programme
The firms were questioned on their reason for participating in LEAD and their responses are summarised in Table 3. Most saw the programme as an opportunity to develop their leadership skills, with a sizeable number linking it in to the business performance. It suggests the objectives of the scheme were well understood. Of the 42 firms, about half (i.e. 19) said they had a definite ‘business proposition’ in mind before joining the scheme, of which the vast majority said it was either very (10) or moderately important (7) to them in joining LEAD. The business propositions included: improved management / business structure (6 responses); expansion of business (5); improved service or product (3); and improved management (2). Of those firms with a business proposition, all said that they planned to implement it having completed the programme (of which one said “during the programme”). This aspect is explored further in Section 6.
Table 3: Main Reason for Participation in LEAD Reason (Q. A2)
No. of responses
Develop / improve leadership skills Develop / improve leadership skills and grow company Develop / improve leadership skills and improve business Business growth Change to business operation Network / obtain a different view / curiosity / other
17 6 4 5 3 7
The participants were asked about the state of their business prior to joining LEAD. The responses to a question about which statement ‘best describes’ their business are shown in Table 4. Around 60 per cent said they were seeking to grow faster, but 20 per cent said they were in difficulty and another 10 per cent said that they weren’t growing at all. No
firm indicated that it was losing sales. Thus, of these, two-thirds expressed a desire to grow the business, and the other one third indicated a static or declining business.
Table 4: Position of Business Prior to Joining LEAD Statement that best describes business (Q. A3)
No. of responses
“We were seeking to grow faster” “We weren’t growing at all” “We were losing sales” “We were in difficulty” “Something else”
25 4 0 8 5
The LEAD programme is organised around a number of areas of support. The opinion on these was explored in the Interim Narrative Report, but participants were asked in our survey how useful each element was in developing their leadership abilities / capabilities, as it is useful in making a link to the business effects. This was according to a four-point scale, and the number of responses in each category is shown in Figure 2.
Figure 2: Usefulness of the LEAD Programme Components 45 40 35 30 25 20 15 10 5 0
Not very useful
Not useful at all
Note: Number of responses to Q. B1. Cases that are not relevant are not shown.
Figure 2 suggests that the elements of the LEAD programme that are most useful to the participants are the master classes, coaching and peer interaction. The first two of these are formal parts of the programme, while the interaction occurs as an inevitable part of 9
these and other elements, and is highly valued by participants.3 The away days are also well regarded, but the action learning and shadowing less so, although 30 of the survey respondents still found each of these to be either moderately or very useful.4 Shadowing is part of the business exchange process, but firms were questioned solely on this aspect, and we refer to this throughout the remainder of the report. Finally, the business support / consultancy, which was not undertaken by all businesses, was found to be very useful by only five participants. It occurs at the end of the LEAD programme and the benefits may not have been fully realised at the time of the survey. The firms were also questioned on two other aspects of the programme, but which are optional. The first is an electronic LEAD forum, which offers participants the chance to communicate with each other on-line. Twenty-five of the 41 respondents said that they took advantage of the electronic forum, and of these 60% thought that it was a useful part of the programme, although 24% thought it was not useful. The other optional aspect is the opportunity to work with a mentor. This is with a private or public sector leader who uses their experience and knowledge as a source of expertise. It spans the final phase of LEAD, and is on a long-term basis, so that again the benefits may not be fully realised.5 Twenty-five businesses (59%) took advantage of mentoring, which can be in more than one business area. The most popular areas were business planning (16 firms), marketing (7) and financial and management information (7). There were no cases in service / manufacturing design or systems, so that all were about business operations. 6 However, the impact of mentoring in helping the participant to implement change to the business seemed to be limited, with only 7 of 24 respondents saying that this had occurred or that they expected it to happen (Question B6).
Overview of Business Effects
After an introductory session, there are twelve master classes consisting of invited speakers, academics and other experts. Half the sessions covered leadership and the other half business theory and related issues. The coaching runs for four months offering three face-to-face and five telephone sessions with LEAD coaches, offering a broad range of industrial experience. 4 The first of these consists of both shadowing, where participants pair and visit each others’ business for 23 days to observe and consult, and exchanges, where they work in the partner’s business for a week and run a mini-consulting project. For the action learning, participants meet monthly for about six hours, involving 6-8 participants, to discuss issues of personal or mutual professional importance, agreeing actions. 5 It differs from coaching, where the coach operates in a facilitating role. 6 Responses in the ‘other’ category (question B5) were: exit strategy, networking, personal development, people management, strategic organisation and culture change.
Cohorts 1 and 2 completed LEAD towards the end of 2005, while the other two cohorts completed the programme only shortly before the time of the survey. Given the limited time period between the participant completion of LEAD and the date of the survey three steps were taken in order to tease out the likely business effects, as follows: •
A focus on the changes to business operations (i.e. intermediate outcomes) that are likely to generate improvements in business performance (i.e. final outcomes).
An examination of business changes occurring since the participant commenced in LEAD, as it is reasonable that these changes take root before completion.
A consideration of the actual and expected business effects over the 2 / 3 years after the time of the survey.
The ‘bottom-up’ questionnaire approach is designed to capture the business changes in participating firms, which are self-attributed by respondents to the LEAD programme through questions on ‘additionality’. No attempt is made to trace through the region or economy-wide effects of LEAD (i.e. impacts), allowing for indirect, multiplier and other effects, as these require a different approach, although the direct sales effects are scaled up to indicate the overall effect. The discussion begins by giving an overview of the LEAD programme business effects, before examining in greater detail in the next section the effects on some important aspects – sales turnover, employment, productivity and per capita GVA – which involves quantifying these different components. A cross-tabulation of results is later made to link the business effects to the characteristics of the firms and LEAD in order to indicate ways in which the programme works most effectively.
4.1 Business Operations First of all, firms were asked about the changes to their business operations since they commenced in the LEAD programme, either actual or planned over the next 2 to 3 years. The responses are given in Figure 3. It is possible that a firm may have undertaken some change and plan to do so further in future, so that the responses may sum to more than the number of respondents. The major changes to business operations already undertaken are in developing a more strategic approach or vision (86%) and an improvement in the general or financial management of the business (62%). Other actual changes include the development of a business plan (43%), the appointment of a business / operation manager 11
(36%), a marketing strategy (33%) and an improvement in service / product delivery or design (33%). Of those planning to undertake changes, around a half of firms are seeking to make further changes in respect of each of the above, but except for the appointment of a business / operation manager, where only a (further) half plan to do so.
Figure 3: Actual and Planned Changes to Business Operations 40 35 30 25 20 15 10 5 0
36 26 22
21 15 11
14 6 2
Develop strategic approach
Improved Develop New business management business plan / operations manager
Develop marketing strategy
Improved service/ product
Note: Number of responses to Qs. C1 and C4.
When asked if any of the above changes relate to the ‘business proposition’ that the firm had in mind before joining LEAD (see Section 3), around two-thirds of respondents said it did (in relation to each of the actual and planned changes). Further, when questioned about whether the changes described in Figure 3 were attributable to LEAD the responses were favourable. The responses on ‘additionality’ are given in Figure 4 in respect of both the actual and planned changes. In either case, no firm responded, “not at all”, but Figure 4 shows that the actual business changes are more likely to depend on LEAD than those that are planned. Of the changes carried out, about 20% depended “absolutely” on LEAD, 50% to a “great extent” and 30% to “some extent”. For the business changes planned, the respective figures are 16%, 40% and 44%.
Figure 4: ‘Additionality’ of Change to Business Operations 100%
Changes made Absolutely
To a great extent
To some extent
Note: Extent to which changes depended on participation in LEAD (Qs. C3 and C6). No firm responded, “not at all”.
4.2 Final Outcomes Firms were asked about the changes to their business performance. Again, a distinction was made between changes that had occurred since participation in LEAD commenced and those that were expected over the next 2 to 3 years. In both cases, and where relevant, the respondents were asked to take out the effect of general inflation, so that interest is in the real effects on business performance. The effects are shown in Figure 5. The results in Figure 5 are promising and suggest that the participant firms have or will experience strong improvements in business outcomes. Thus, over the next 2 to 3 years 85 to 95 per cent of firms expect increases in each of sales turnover, employment, productivity and profits, while no firm expects none of these to increase. Given these are final outcomes, the proportion of firms experiencing an actual change in these is much smaller, and generally in the range of about 40 to 60 per cent, although still encouraging. Of these the increase in sales turnover seems to have been realised the most. However, it says nothing about whether the effects were due to the LEAD programme or not, and this is explored below where the final outcomes are considered in turn.
Figure 5: Actual and Planned Changes to Business Performance 45 40 35 30 25
20 15 10 5 0
2 Sales turnover
None of these
Note: Number of responses to Qs. C7 and C8.
Firms were asked what aspects of the LEAD programme were â€œparticularly importantâ€? in promoting changes to the business, either actual or expected. The responses are shown in Figure 6, and they accord well with those reported above (Figure 2) on the usefulness of the LEAD programme components. It suggests the usefulness of the programme is judged in terms of its ability to promote business change. Figure 6 shows that the master classes are extremely well rated, with 40 of the 42 firms indicating that these were important to promote business change.
The coaching and peer interaction are also
important, while about fifty per cent rate the away days, shadowing and action learning as important. The figure shows the reduced importance of mentoring, and the limited role of the business support and the electronic forum, confirming the above results.
Figure 6: Aspects of LEAD in Promoting Business Change 45 30 40 28
ss es sin Bu
ng to M en
in rn le a io n
g ad ow Sh
ra te Pe
hi ng oa c
es la ss rc te M as
Note: Responses to Q. G1. Number of firms responding as important in each case.
The effects on business performance are now considered in greater detail in relation to each of the sales turnover, employment and productivity. We also consider the change in gross value added (GVA) per employee.
The number of firms reporting that they had experienced a real increase in sales turnover since they commenced participation in LEAD was 64%, with 93% saying they expected turnover to increase over the next 2 to 3 years (see Figure 5). It is possible that the actual sales increase may reflect changes made prior to joining LEAD, although these figures suggest a step-change in business performance. The nature of the sales turnover change and its relation to the LEAD programme is now explored. It is for those firms reporting an actual or planned change in turnover (i.e. 40 firms in total). First of all, an attempt was made to determine the increase in sales turnover that was due to the LEAD programme, i.e. ‘additionality’. Table 5 shows that of those firms experiencing / expecting an increase in sales turnover about a half (19 firms) thought that this depended substantially on the programme, of which two firms said that none of the increased sales would otherwise have occurred in the absence of LEAD. A further 8 firms thought that less than half the increased sales was due to participation, but 10 firms said that none of the increased sales was due to LEAD.
Table 5: ‘Additionality’ of LEAD on Sales Turnover Sales increase if not participated in LEAD (Q. D4)
“None of it would have occurred” “Less than 50% would have occurred” “More than 50% would have occurred” “Exactly the same”
2 (5%) 17 (46%) 8 (22%) 10 (27%)
This latter result may not be surprising given that 19 firms had a ‘business proposition’ in mind before joining LEAD, and that for the vast majority it was very or moderately
important to them in joining the programme.7 However, that said, around half (10) of the 19 firms with an increase in sales that depended substantially on LEAD also had no business proposition prior to joining. The distribution of the annual increases in sales due to LEAD for the 2 to 3 years after participation is given in Figure 7. These figures are self-reported by firms, and they are to some extent speculative, possibly representing goals rather than attainment. All the same, they are useful in at least indicating the aspirations of firms. In total there are 27 respondents (see Table 5), and Figure 7 shows that the median and modal sales increases are in the range £100,000 to £250,000. These are the annual increases in sales that the respondents attribute to LEAD participation over a 2 to 3-year horizon.
Figure 7: Annual Increase in Sales due to LEAD Up to £1000 0
£1001 - £10,000
£10,001 - £50,000
£50,001 - £100,000
£100,001 - £250,000
£250,001 - £500,000
£500.001 - £1m
Note: Number of responses to Q. D5 for firms indicating a sales increase due to LEAD in Q. D4.
Calculating the sales increases at the means of the ranges (e.g. £5500 for the range £1001 to £10,000 and so on), and setting ‘> £1m’ equal to £1.25 million, gives a mean increase in sales turnover for the 27 firms of around £310,000 per annum. If this is realised then it represents a substantial effect. For the 42 firms included in the survey it suggests a mean effect of around £200,000 per annum. Further, on the gloomy view that the programme was completely ineffective in the non-respondents then across the 65 firms it suggests a mean annual effect on sales of just under £130,000, which is still impressive. Overall, it indicates a total increase in sales (across the 65 firms) of about £8.5m per annum. 7
This is confirmed in section 6.
In total, 37 of the 42 firms said that they had either experienced or expected to experience an increase in employment over the next 2 to 3 years. Of these, Figure 5 shows that 22 businesses had experienced an increase in employment since commencing LEAD, while virtually all the 37 firms expected some further increase. When quizzed about the net increase in employment at their firms in 2 to 3 years time, the distribution of responses is given in Figure 8. Most firms expected their net employment to increase by relatively modest amounts, with the median and modal increase in the range 3 to 5 jobs. However, it will be recalled that the LEAD participant firms are rather small, with 80% of the firms surveyed having less than 25 employees when commencing the programme (Table 2).
Figure 8: Increase in Net Employment after 2 to 3 Years 1
26 -50 51-100
Note: Number of responses to Q. E1.
Again, the firms were probed on â€˜additionalityâ€™, but a similar pattern emerged to that for sales. The results are shown in Table 6. The firms were less likely to claim that none of the employment growth would otherwise have occurred, but otherwise the responses are much the same as before. The nature of the jobs was probed, and most jobs were at the technical or skilled level. On average, the respondents said that of the increased jobs, 48% were managerial / technical, 33% semi-skilled / unskilled, while 19% were at the managerial level. Further, on average, 90% of the increased jobs were full time and 56% were to be filled by males. 17
Table 6: ‘Additionality’ of LEAD on Employment Change Job increase if not participated in LEAD (Q. E2)
“None of it would have occurred” “Less than 50% would have occurred” “More than 50% would have occurred” “Exactly the same”
4 (11%) 17 (49%) 8 (23%) 6 (17%)
The number of firms reporting that they had experienced an increase in productivity (i.e. sales per employee) since they commenced participation in LEAD was 43%, with 83% saying they expected an increase over the next 2 to 3 years (Figure 5). Overall, 37 firms (88%) said that they had experienced or expected an increase in productivity. When asked about the annual increase in productivity, Figure 9 shows the firms tended to select towards the top end of the range that they were given. Of the 33 firms responding, 39% thought the actual / expected increase in productivity would exceed £10,000 per annum, while a further 27% put it in the range £5000 to £10,000 per annum. To some extent this is to be expected, as it is consistent with the large increase in sales revenue coupled with a relatively modest net increase in firm employment.
Figure 9: Annual Increase in Productivity 3
£1000 - £5000
£5000 - £10,000
Note: Number of responses to Q. F2.
Again, additionality’ was explored, and the results broadly corresponded to those found above. Overall, for the 34 relevant cases, 15% said the increase in productivity depended 18
“absolutely” on participation and a further 41% said it depended to “a great extent”. Only 3% (one firm) said it depended “not at all” on participation in LEAD. Finally, the source of the productivity growth was examined for the 37 firms. The responses are given in Figure 10. It suggests good efficiency gains, with 84% of firms indicating that the increased productivity arose from organisational change and 76% from the better use of existing employees.
Around 38% attribute the productivity
improvement to increased sales and / or to increased staff at the customer or production level, while an increased business scale (economies of scale), new managerial staff and better use of existing capital (including premises) also feature.
Overall, as well as
increasing the sales turnover, the firms have also improved their efficiency, of which around a half of firms said it depended substantially on the LEAD programme in some way.
Figure 10: Sources of Productivity Growth 45 30 15
ap i of c
ri an ag e at m
el al l
ca rs ge La r
er us to m
l le ve
s le sa d st af fa tc
na l tio rg an isa O
oy ee s
Note: Number of responses to Q. F1. Firms can choose more than one category.
Gross Value Added per Employee
An attempt was made to determine the gross value added (GVA) per employee for the survey firms, and to examine the annual change in this. Changes in GVA per employee reveal information on the efficiency of the firms and their competitiveness. However, the task poses difficulties in several respects. First, it is based on objective measurement, so it is possible to only examine the actual observed change in GVA. This is problematic
given that some of the firms have only just completed LEAD. Nevertheless, the above analysis suggests that substantial changes in operations and outcomes have occurred during the course of the programme. Second, GVA requires financial information on the firms, including sales turnover and profits, which many firms are reluctant to release. It also does not help that the survey was conducted at the time of the financial year-end for some firms, so that the ‘books’ were with the accountants. GVA per employee is defined as:8 GVA per employee = (sales turnover – employment costs – capital depreciation costs – net profit) / number of employees. The firms were asked to supply information on each of the above components for the latest financial year (Question G2). It was supplemented with similar data compiled by the IEED at two separate points of time. Broadly, the approach was to match the latest GVA data with that for the previous year. Other limited interpolation of the data was also undertaken, which generated 27 usable cases where GVA per employee was observed for two consecutive years. This approach is crude, as the latest data may relate to different years (i.e. 2005/06 or 2004/05), and it indicates the difficulty in carrying out this kind of work. All the same, the data were carefully checked and outliers investigated. The mean levels of GVA per employee and the change in GVA are given in Table 7. It shows a small mean annual change in GVA per employee of £2,215 across firms. It is encouraging, as it is positive, although it is not that much greater than the inflation rate. The table presents the standard deviations, which indicates significant variation in the data, which is worth further exploration.
Table 7: Mean Change in GVA per Employee Mean
Initial GVA / employee
Subsequent GVA / employee
Change in GVA / employee
Note: Calculations for 27 firms. The final row gives the mean change across firms.
The denominator is often measured in terms of full-time equivalents, but here it is measured simply by the number of employees. The error may not be too serious, as employment tends not to change dramatically over the course of a year, while most of the jobs are in fact full time (see Section 5.2). In practice, other factors seem much more important, and in particular the change in sales turnover.
Figure 11 plots the initial level of GVA per employee against the annual change in GVA per employee. It shows that a few firms have very high initial GVA per employee, but that it is generally less then £50,000. The change in GVA per employee is also generally positive and less than £20,000. There are a few outliers, where the change in GVA per employee is greater or less than £20,000, but these cases cancel, so that the mean figure in Table 7 is unaffected. However, Figure 11 shows that firms typically experience an increase in GVA per employee in the range £0 - £20,000, and the mean figure is pulled down by some negative cases. An examination of these negative cases shows the firms had a sharp drop in sales, with constant employment, causing the GVA per employee to fall. It is possible that the reduction in sales may be temporary or that employment will also fall over a longer time horizon, so that the negative figures may be transitory.
Figure 11: Initial Level and Change in GVA per Employee
Annual change in GVA / employee
£ 80,000 £ 60,000 £ 40,000 £ 20,000 £0 -£ 20,000
-£ 40,000 -£ 60,000 Initial GVA / employee
Factors Underlying Programme Success
We have seen that the LEAD programme is particularly useful in promoting sales growth, and according to respondents it also led to increases in net employment and productivity, which is consistent with the typical increase in GVA per employee. The aspects of the programme that are considered by firms as important in promoting business change have also been identified. It remains to consider the possible factors underlying the success of 21
LEAD, and in particular where the programme appears relatively effective. This is now considered for the different cohorts of firms and for those with a business proposition.
As regards the effect of the LEAD programme in different cohorts, Table 8 summarises the information for the business outcomes. For Cohorts 1 and 2 there are 16 firms in the survey responses, and for Cohorts 3 and 4 there are 26 observations (see Table 1). These numbers are small, which should be borne in mind. Cohorts 1 and 2 graduated towards the end of 2005, while Cohorts 3 and 4 completed about six months later.
Table 8: Business Outcomes by Cohort Cohorts 1 / 2
Cohorts 3 / 4
Mean firm size Employment Sales turnover
Sales turnover (Q. D5) Up to £50,000 £50,001 - £250,000 More than £250,000
11% (1) 56% (5) 33% (3)
22% (4) 50% (9) 28% (5)
19% (5) 52% (14) 29% (8)
Employment (Q. E1) 1–5 6 – 10 11 – 25 26+
50% (7) 21% (3) 7% (1) 21% (3)
61% (14) 21% (5) 9% (2) 9% (2)
57% (21) 22% (8) 8% (3) 14% (5)
Productivity (Q. F2) < £1000 £1000 - £5000 £5000 - £10,000 > £10,000
17% (2) 8% (1) 25% (3) 50% (6)
5% (1) 30% (6) 30% (6) 35% (7)
9% (3) 22% (7) 28% (9) 41% (13)
‘Additionality’ (none or < 50% would have occurred): Sales 47% (7) 55% (12) Employment 46% (6) 68% (15) Productivity 25% (3) 62% (13)
51% (19) 60% (21) 56% (16)
Note: * Excludes one firm with 500 employees, mostly part-time, and £13 million turnover. Number of cases associated with each percentage is given in parentheses.
The table shows that the first two cohorts are larger in size on average, both in terms of employment and sales. Table 8 shows the increased sales, employment and productivity for the cohorts, and then considers ‘additionality’.9 The sales are attributable directly to LEAD by the respondents in Table 8, but no such attribution is made for employment and productivity. Overall, there is relatively little difference between the two sets of cohorts. Cohorts 1 and 2 are associated with slightly larger effects with respect to all outcomes, but which may be due to their larger size. However, much clearer differences emerge in relation to ‘additionality,’ with higher proportions of firms in Cohorts 3 and 4 saying that the change in employment or productivity depended substantially on LEAD. As regards the change in sales that depended on LEAD Figure 12 shows that the difference between cohorts is not that great.
Figure 12: Sales Change due to LEAD by Cohort
Up to £1000 £1001 - £10,000 £10,001 - £50,000 £50,001 - £100,000 £100,001 - £250,000 £250,001 - £500,000 £500.001 - £1m > £1m
Cohorts 1 / 2 Cohorts 3 / 4 Note: Number of responses to Q. D5 for firms indicating a sales increase due to LEAD in Q. D4. Figure 7 gives the aggregate results for all four cohorts.
Finally, it is possible to compare how well the different aspects of the LEAD programme were valued by the two sets of cohorts. This is the question the firms were asked on what aspects of LEAD were “particularly important” in promoting business change (actual or expected). The results in Figure 13 again suggest relatively little difference between the 9
The first three of these correspond to Figures 7, 8 and 9 respectively.
cohorts, although peer interaction, shadowing and mentoring were relatively more valued by Cohorts 1 and 2 in terms of business changes, although the away days were much less so (31% compared to 65% for Cohorts 3 and 4).
100 80 60 40 20 m
ru fo ic on
ct r El e
s da y
Sh ad ow
g in ch
ss la rc te as M
Figure 13: LEAD Aspects Promoting Business Change by Cohort
Cohorts 1/2 Cohorts 3/4 Note: Responses to Q. G1. Figure shows percentage of firms responding positively in each case. Figure 6 gives the aggregate results for all four cohorts.
Another interesting aspect is whether those firms that had a definite business proposition in mind before joining LEAD faired any better or worse than the other firms. The results are summarised in Table 9. In total, 19 of the 42 firms responding to the survey said that they had a definite ‘business proposition’ in mind before joining LEAD, of which the vast majority said it was either very or moderately important to them.10 Table 9 suggests that the firms with a business proposition were different in character, as 79% of these firms were “seeking to grow faster” compared to 43% of firms otherwise. It may be suggestive of better effects for LEAD in these firms, but Table 9 shows that the LEAD
These is little difference between cohorts with 44% of Cohorts 1 and 2 having a business proposition,
compared to 46% for Cohorts 3 and 4.
programme was much more likely to be ‘additional’ in the firms that did not have such a proposition.
Table 9: Business Outcomes by Business Proposition Definite Business Proposition: Yes No Business position: “Seeking to grow faster” Other
79% (15) 21% (4)
43% (10) 57% (13)
‘Additionality’ (none or < 50% would have occurred): Sales 53% (9) 50% (10) Employment 53% (10) 69% (11) Productivity 47% (9) 67% (10)
60% (25) 40% (17) 51% (19) 60% (21) 56% (19)
Note: Number of cases associated with each percentage is given in parentheses.
Finally, the difference between the two kinds of firm can again be examined by looking at the change in sales turnover due to LEAD. It is shown in Figure 14. Overall, the firms with a business proposition show much larger increases in sales due to LEAD. Thus, while the programme is less likely to be ‘additional’ in these firms, where it does have an effect, it produces greater increases in sales attributable to the programme.
Figure 14: Sales Change due to LEAD by Business Proposition Up to £1000 £1001 - £10,000 £10,001 - £50,000 £50,001 - £100,000 £100,001 - £250,000 £250,001 - £500,000 £500.001 - £1m > £1m
Business proposition No proposition Note: Number of responses to Q. D5 for firms indicating a sales increase due to LEAD in Q. D4. Figure 7 gives the aggregate results for all firms.
The LEAD programme – Leading Enterprise and Development – seeks to develop the leadership and management abilities of smaller-sized enterprises in order to promote business development and growth. It operated in Northwest England from October 2004 and concluded in the spring of 2006, with the first two cohorts completing towards the end of 2005 and the other cohorts six months later. The focus of this evaluation is on the business effects of the LEAD programme on the operations and outcomes of participant firms, including sales, employment, productivity and gross value added per employee. A questionnaire survey of LEAD participants was administered in 2006, shortly after the final cohort completed the programme, from which a total of 42 useable responses were obtained from the 65 participant firms, giving a response rate of about two-thirds. Overall, the evaluation finds strong effects of the LEAD programme on business operations and outcomes. In relation to operations, LEAD allowed firms to develop a more strategic approach and improve the general management of the business. Half the firms either have or plan to undertake other improvements, such as the development of a business plan / marketing strategy or an improved product / service, while up to a third have appointed a new business or operations manager. All firms say LEAD played some role in these changes, with around sixty per cent reporting that they depended absolutely, or to a great extent on, participation in the programme. The changes in business operations have led to good effects on firm outcomes. Over the next 2 to 3 years, around 90 per cent of firms expect increases in either sales turnover, employment, productivity or profits, with no firm expecting none of these to increase, while the proportion of firms experiencing changes in these since commencing participation in the programme is generally in the range of 40 to 60 per cent. The aspects of LEAD that were “particularly important” in promoting these changes were the master classes, the coaching and peer interaction. About fifty per cent of participants rated the away days, shadowing (a part of the business exchange) and action learning as important, but mentoring to a lesser extent, as were the business support and the electronic forum.
On the specific business outcomes, about half of those firms reporting an increase in sales (actual or expected) said it depended substantially on the programme, leading to a very large increase in sales attributable to the programme. The median and modal annual sales increases due to LEAD were in the range £100,000 to £250,000, and on average it works out at about £200,000 per annum for the 42 firms included in the survey. On the pessimistic view that LEAD was completely ineffective in the non-respondent firms, the mean annual sales increase due to LEAD is about £130,000 across all firms. It compares with average annual sales turnover for the firms of around £1 million. On employment, most firms expected the net number of employees to increase by only modest amounts over the next few years, with the median and modal increase in the range 3 to 5 jobs. It compares with a mean employment level in firms of about 20 jobs. A similar pattern of ‘additionality’ applied as for the sales increase. Thirty-two of the 42 firms expected productivity (i.e. sales per employee) to increase, with two-thirds of these estimating it to be more than £5,000 per annum over the mid-term. This is expected, with the strong increase in sales, but only modest increase in employment. The main sources of the productivity growth were organisational change and a better use of existing employees, which is consistent with the purpose of LEAD and with the changes in business operations outlined above. The increase in productivity reported by firms is consistent with an independent analysis of gross value added (GVA) per employee using primary data. This finds a small average annual increase in GVA per employee of only about £2,200, which is about the inflation rate. However, the change in GVA per employee is typically in the range £0 to £20,000, with the mean reduced by some negative GVA cases, where firms experienced a sharp drop in revenue, but static employment, which may not have adjusted. Finally, the report examined areas where the LEAD programme may be relatively more successful. One concern is that the first two cohorts commenced six months earlier than the other firms, after which some changes were made to the programme. In fact, there was little difference between the earlier and later two cohorts, although later cohorts said the away days were more important, while the earlier ones thought more of the peer interaction, shadowing and mentoring. Another issue is that about the half the firms had a definite ‘business proposition’ in mind before joining LEAD (mainly to do with seeking growth). This produced weaker ‘additionality’ in these firms, but their greater growth meant that the sales increase due to LEAD was in fact much greater than for other firms.
Overall, the principal finding of this evaluation is that the LEAD programme has achieved substantial effects on business operations and outcomes, whether achieved or expected. In this view, we are, of course, following the well-established methodology in evaluation of relying on the views expressed by respondent participant firms. Naturally, it is difficult to independently verify the responses, although the independent analysis of gross value added does give us some confidence, and the results are internally consistent. It leads us to conclude that the LEAD programme has been successful in achieving its objective of promoting business development and growth, although some aspects of the programme seem to have been relatively more important in generating these changes.
Please tick relevant response and give explanation as required. Company number: _____________ A. THE INTERVIEWEE A1. Can I confirm that you are the person who participated in the LEAD programme, and that you were the sole company representative? ______Yes = 1 ______No = 0 Please explain: _______________________________________ A2. Briefly, what was your main reason for participating in the LEAD programme? Please explain: ____________________________________________________ _________________________________________________________________ A3. Which ONE of the following would you say best describes the position of your business at the time you applied to the LEAD programme? ______ We were seeking to grow faster = 1 ______ We weren’t growing at all = 2 ______ We were losing sales = 3 ______ We were in difficulty = 4 ______ Something else Please specify: _________________________________ A4. Did you have a definite ‘business proposition’ in mind before joining LEAD (i.e. a proposed change to your business that you would like to make)? ______ No = 0 Go to A7 ______ Yes = 1 Please explain: _______________________________________ _________________________________________________________________ A5. How important was this ‘business proposition’ to you in joining the programme? ______ Very important = 1 ______ Moderately important = 2 ______ Not very important = 3 ______ Unimportant = 4
A6. Have you, or do you intend to, implement this proposition after participation in LEAD? ______ Yes = 1 ______ No = 0 Please explain: ______________________________________ ________________________________________________________________ A7. The LEAD programme involves a commitment of your time. commitment had any detrimental effect on your business? ______ No = 0
______ Yes = 1 Please explain: ______________________________________ _________________________________________________________________
B. THE PROGRAMME B1. The LEAD programme is organised around areas of support. In relation to each of these how useful do you think they were in developing your leadership abilities / capabilities? Master classes: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Action learning: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Coaching: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Shadowing / Exchanges: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Peer interaction: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Away-days: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4
Business Support / Consultancy: ______ Very useful = 1 ______ Moderately useful = 2 ______ Not very useful = 3 ______ Not useful at all = 4 ______ Not undertaken / not relevant = 5 B2. Did you take advantage of the electronic LEAD forum? ______ Yes = 1 ______ No = 0 Go to B4 30
B3. Do you think the electronic LEAD forum was a useful part of the programme? ______ Yes = 1 ______ No = 0 Please explain: _______________________________________ ______ Donâ€™t know = 2 B4. The LEAD programme offers participants the opportunity to work with a mentor. Have you taken advantage of this? ______ Yes = 1 ______ No = 0 Go to Section C B5. In what business area(s) has / will the mentor give support? Tick all that apply: ______ Business planning = 1 ______ Marketing = 2 ______ Product or service quality and design = 3 ______ Financial and management information = 4 ______ Manufacturing and service systems = 5 ______ Other Please specify: ________________________________________ B6. Has this mentor helped you implement change to your business? ______ Yes = 1 ______ Not yet, but expect this to happen = 2 ______ No = 0 Go to Section C
C. THE BUSINESS C1. Have you made any of the following changes to your business operations since you commenced participation in the LEAD programme? Tick all that apply: ______ Development of a more strategic approach or vision = 1 ______ Development of a business plan = 2 ______ Development of a marketing strategy = 3 ______ Improvement in service / product delivery or design = 4 ______ Improvement in general or financial management = 5 ______ Appointment of a business / operations manager = 6 ______ Other Please specify: ________________________________________ _________________________________________________________________ ______No changes Go to C4 C2. Do any of the changes to operations identified in C1 reflect a â€˜business propositionâ€™ (referred to in A4) that was formed prior to joining the LEAD programme? ______ Yes = 1 ______ No = 0
C3. Overall, would you say the changes in C1 depended on your participation in LEAD? ______ Absolutely = 1 ______ To a great extent = 2 ______ To some extent = 3 ______ Not at all = 4 C4. Do you plan to undertake any of the following changes to your business operations over the next 2 to 3 years? Tick all that apply: ______ Development of a more strategic approach or vision = 1 ______ Development of a business plan = 2 ______ Development of a marketing strategy = 3 ______ Improvement in service / product delivery or design = 4 ______ Improvement in general or financial management = 5 ______ Appointment of a business / operations manager = 6 ______ Other Please specify: ________________________________________ _________________________________________________________________ ______No changes Go to C7 C5. Do any of the changes to operations identified in C4 reflect a â€˜business propositionâ€™ (referred to in A4) that was formed prior to joining the LEAD programme? ______ Yes = 1 ______ No = 0 C6. Overall, would you say the changes in C4 depended on your participation in LEAD? ______ Absolutely = 1 ______ To a great extent = 2 ______ To some extent = 3 ______ Not at all = 4 C7. Since commencing participation in LEAD have any of the following occurred to your business performance (after allowing for general inflation)? Tick all that apply: ______ Increase in sales turnover = 1 ______ Increase in overall employment (i.e. net employment) = 2 ______ Increase in productivity (sales per employee) = 3 ______ Increase in profits = 4 ______ None of the above = 5 ______ Something else Please specify: ________________________________ C8. Do you expect any of the following to occur in your business performance over the next 2 to 3 years (after allowing for general inflation)? Tick all that apply: ______ Increase in sales turnover = 1 ______ Increase in overall employment (i.e. net employment) = 2 ______ Increase in productivity (sales per employee) = 3 ______ Increase in profits = 4 ______ None of the above = 5 ______ Something else Please specify: ________________________________ 32
D. SALES TURNOVER If you have experienced or expect an increase in SALES TURNOVER over the next 2 to 3 years (C7 and C8) then please complete this section. Otherwise go to Section E. D1. What is the source of the increased sales? Tick all that apply: ______ New plant and / or premises, including possible relocation = 1 ______ New machinery and / or equipment = 2 ______ Employment of new staff at management level = 3 ______ Employment of new staff at production / customer level = 4 ______ Better use of existing employees = 5 ______ Other Please specify: ________________________________________ _________________________________________________________________ D2. Do you think the increase in sales will be maintained into the foreseeable future? ______ Yes = 1 ______ No = 0 ______ Don’t know = 2 D3. Would you say the increase in sales will increase your market share? ______ Yes = 1 ______ No = 0 ______ Don’t know = 2 D4. If you had not participated in LEAD, do you think the increase in sales would have been? ______ Exactly the same = 1 Go to Section E ______ More than 50% would have occurred = 2 ______ Less than 50% would have occurred = 3 ______ None of it would have occurred = 4 D5. On average, what would you say is the annual increase in sales due to the LEAD programme in the 2 to 3 years after participation? ______ Up to £1000 =1 ______ £1001 - £10,000 =2 ______ £10,001 - £50,000 =3 ______ £50,001 - £100,000 =4 ______ £100,001 - £250,000 =5 ______ £250,001 - £500,000 =6 ______ £500,001 - £1 million =7 ______ greater than £1 million =8
E. EMPLOYMENT If you have experienced or expect an increase in EMPLOYMENT over the next 2 to 3 years (C7 and C8) then please complete this section. Otherwise go to Section F. E1. In the 2 to 3 years after participation in LEAD what is the overall increase (i.e. net) in the number of employees that you expect your business to employ by the end of this time? ______ 1 =1 ______ 11 – 25 =5 ______ 2 =2 ______ 26 – 50 =6 ______ 3 – 5 =3 ______ 51 – 100 =7 ______ 6 – 10 =4 ______ 101 + =8 E2. If you had not participated in LEAD, do you think the overall job increase would be? ______ Exactly the same = 1 Go to Section F ______ More than 50% would have occurred = 2 ______ Less than 50% would have occurred = 3 ______ None of it would have occurred = 4 E3. Of the increase in jobs approximately what percentage is: Managerial Please specify %: ____________________________________ Technical / skilled Please specify %: _______________________________ Semi-skilled / unskilled Please specify %: ___________________________ E4. Of the increase, approximately what percentage are full-time jobs? ____________ E5. Of the increase, approximately what percentage is male? ________________
F. PRODUCTIVITY If you have experienced or expect an increase in PRODUCTIVITY over the next 2 to 3 years (C7 and C8) then please complete this section. Otherwise go to Section G. F1. What are the main reasons for the increase in productivity, i.e. sales per employee? Tick all that apply: ______ Implementation of organisational change = 1 ______ Increased sales but with no increase in business scale = 2 ______ A larger business scale with more capital = 3 ______ Better use of existing capital, including new premises = 4 ______ Hiring of new staff at management level = 5 ______ Hiring of new staff at production / customer level = 6 ______ Better use of existing employees = 7 ______ Other Please specify: ________________________________________ _________________________________________________________________ 34
F2. Approximately, what is the overall increase in sales turnover per employee (i.e. the increase in sales turnover divided by the number of employees)? ______ Up to £1,000 per annum = 1 ______ Between £1,000 and £5,000 per annum = 2 ______ Between £5,000 and £10,000 per annum = 3 ______ More than £10,000 per annum = 4 F3. To what extent do you think the increase in productivity is due to your participation in the LEAD programme? ______ Absolutely = 1 ______ To a great extent = 2 ______ To some extent = 3 ______ Not at all = 4
G. OVERVIEW G1. Overall, what aspects of LEAD do you think were particularly important in promoting changes to your business (either actual or expected)? Tick all that apply: ______ Master classes = 1 ______ Action learning = 2 ______ Coaching = 3 ______ Shadowing / Exchanges = 4 ______ Peer interaction = 5 ______ Away-days = 6 ______ Business Support / Consultancy = 7 ______ Electronic LEAD forum = 8 ______ Mentoring = 9 ______ Other Please specify: ________________________________________ _________________________________________________________________ G2. Policymakers are interested in the gross value added (GVA). For the latest financial year can you please supply the following information that will help us to calculate GVA: Financial year: ______________________ Sales Turnover (£): ________________________ Employment costs (£): _________________________ Capital depreciation (£): ___________________________ Net Profit (£): __________________________ Number of employees: _______________________ If non-response please give reason: ___________________________________ Thank you for your time and cooperation, which is greatly appreciated.