Costello Medical Independent SROI investigation 2024

NOISE SOLUTION

SROI Model: Executive Summary and Methodology

APRIL 2024

Costello Medical UK | US | Singapore | China www.costellomedical.com

0

SROI MODEL METHODOLOGY

Contents List of Figures ................................................................................................................ 2 List of Tables ................................................................................................................. 2 Executive Summary........................................................................................................ 3 Introduction .......................................................................................................................................... 3 Methods ................................................................................................................................................ 3 Results .................................................................................................................................................. 3 Key Takeaways ...................................................................................................................................... 3

Model Methodology ........................................................................................................ 6 Social Return on Investment (SROI) ................................................................................ 6 Calculating SROI ............................................................................................................ 6 Objectives ..................................................................................................................... 7 Scope and Stakeholders ................................................................................................. 8 Time Horizon ......................................................................................................................................... 8 Stakeholders ......................................................................................................................................... 8

Population Inputs ........................................................................................................... 9 Investments ................................................................................................................ 10 Activities and Outcomes ............................................................................................... 11 Impact Map ......................................................................................................................................... 11 Evaluating and Monetising Outcomes .................................................................................................... 13 Impact on Participants’ Education ......................................................................................................... 14 Re-entry into education, employment or training ................................................................................ 14 Reduced behavioural issues/school conduct problems ......................................................................... 15 Reduced/avoided school exclusion ..................................................................................................... 16 Reduced need for additional support in education and related services ................................................ 17 Impact on Participants’ Health .............................................................................................................. 19 Increased social capital/reduced isolation........................................................................................... 19 Diverted from acute mental health admission ..................................................................................... 20 Disengagement from mental health services ...................................................................................... 21 Impact on Participants’ Family .............................................................................................................. 22 Reduced stress and burden for family members/guardians.................................................................. 22 Increased labour productivity from guardians ..................................................................................... 24 Adjustment ......................................................................................................................................... 25 Attribution ........................................................................................................................................... 25 Drop-Off ............................................................................................................................................. 25

Sensitivity Analyses ...................................................................................................... 27 Assumptions and Limitations ......................................................................................... 28

Copyright © Costello Medical Consulting Ltd

|

1

SROI MODEL METHODOLOGY

General Limitations of SROI Methodology .............................................................................................. 28 Limitations of the Noise Solution SROI Model ........................................................................................ 28

References .................................................................................................................. 29

List of Figures Figure 1. Summary of SROI analysis methodology ...................................................................................... 4 Figure 2. Per participant costs associated with each outcome included in the SROI analysis, with and without Noise Solution, across two years ................................................................................................................ 5 Figure 3. Impact map .............................................................................................................................. 12

List of Tables Table 1. Seven principles of the SROI method1 ........................................................................................... 6 Table 2. Summary of outcomes valued in the SROI analysis ...................................................................... 13 Table 3. Estimated proportion of Noise Solution participants with NEET status ............................................ 14 Table 4. Age of Noise Solution participants ............................................................................................... 14 Table 5. Financial proxy for NEET status ................................................................................................... 14 Table 6. Estimated behavioural problems in non-NEET participants in school .............................................. 15 Table 7. Financial proxy for managing school behavioural problems ........................................................... 15 Table 8. Estimated school exclusions of participants .................................................................................. 16 Table 9. Financial proxy for school exclusions, per exclusion ...................................................................... 16 Table 10. Education and related service use, as reported in literature ........................................................ 17 Table 11. Estimated use of support in education and related services by participants .................................. 18 Table 12. Financial proxy for additional support in education and related services ....................................... 18 Table 13. Estimated loneliness of participants ........................................................................................... 19 Table 14. Financial proxy for loneliness .................................................................................................... 20 Table 15. Estimated in-patient mental health admission ............................................................................ 20 Table 16. Financial proxy for in-patient mental health admission ................................................................ 21 Table 17. Estimated engagement of participants with mental health services .............................................. 21 Table 18. Financial proxy for community mental health services................................................................. 22 Table 19. Estimated stress and burden for siblings .................................................................................... 22 Table 20. Estimated stress and burden for guardians ................................................................................ 23 Table 21. Financial proxy for stress and burden for family members/guardians ........................................... 23 Table 22. Estimated loss of productivity of guardians per participant .......................................................... 24 Table 23. Financial proxy for labour productivity ....................................................................................... 24 Table 24. Annual drop-off for each outcome ............................................................................................. 26

Copyright © Costello Medical Consulting Ltd

|

2

SROI MODEL METHODOLOGY

Executive Summary Introduction Noise Solution is an innovative organisation harnessing the power of music alongside technology to boost and build relationships around those that they work with, while positively impacting upon the well‑being of participants. Their 1:1 music mentoring programme gives individuals facing a variety of challenging circumstances time to work on and direct a creative project that uses a combination of music technology and instruments. The whole process is securely documented by participants using video, photos and audio on a cloud-based platform, creating a private ‘digital story’ of success that develops with each weekly session. Each participant can share their success with friends, family and professional key workers in a way that looks and feels like a social media feed, inviting their network to comment on the digital story. In this way each feed mirrors success and creates a cycle of positive affirmation from people whose opinions matter to the young person. You can find out more about this here.

The programme’s approach is designed to focus on instilling feelings of autonomy, competence and relatedness; evidenced within Self Determination Theory as the basic psychological needs vital for well-being to flourish. Through measuring well-being using a National Health Service validated scale (Short Warwick‑Edinburgh Mental Well-being Scale [SWEMWBS]), it was found that 61% of participants showed a meaningful increase in well-being through attendance of Noise Solution’s programme.

Furthermore, qualitative analyses suggested that the programme was often transformational for participants, with positive changes including re‑engagement with education or key workers, reduced isolation and increased hopes and aspirations for the future.

This study aimed to bring together insights gathered from participants engaged with the Noise Solution programme to quantify the financial value that the programme brings to society. A social return on investment (SROI) analysis was conducted, which indicates the social and economic value (£) generated per £1 of investment. An SROI greater than 1 indicates that more value is generated by the programme than that which was initially invested.

Methods The SROI analysis methodology is summarised in Figure 1 and presented in the Model Methodology section.

Results The investment per participant attending the programme was valued at £2,075.00. This was estimated to generate £26,071.87 of social and economic value, meaning that for every £1 invested into the Noise Solution programme, there is a return of £12.56 in social and economic value. This indicates that resources are used effectively to result in transformative impact for participants in the programme.

The costs per participant associated with each outcome in the SROI analysis, before and after participation in the programme, are presented in Figure 2. The majority of outcomes considered in the analysis were experienced by the participants of the programme themselves and fell into two broad categories of education and health. Outcomes experienced by participants’ families were also considered. Other individuals engaged with Noise Solution, such as supporting professionals and musicians involved in the programme, were not considered in the analysis due to difficulty robustly quantifying the associated outcomes.

Key Takeaways The analysis indicates that the Noise Solution programme uses resources effectively to achieve transformative impact for participants. The SROI analysis captured outcomes that could be robustly modelled and applied factors to ensure they reflected the real-world impact of Noise Solution as closely as possible. However, the exclusion of certain outcomes and people who benefit from Noise Solution means that these results are likely to be an underestimate, and the impact of the Noise Solution programme is anticipated to extend beyond that captured here.

3

SROI MODEL METHODOLOGY

Figure 1. Summary of SROI analysis methodology

Impact Map

Indicators

Financial Proxies

Adjustment

SROI Calculation

An impact map captured the resources invested into the Noise Solution programme, the activities that took place, and the changes (outcomes) experienced by participants engaged with the programme.

These included participants, their families, professionals, musicians and the Noise Solution team.

These changes were gathered through interviews and feedback.

For each outcome, it was evaluated whether there were robust data that evidenced the change taking place (an indicator).

Noise Solution's internal records were then used to identify the proportion of people who experienced each outcome. In some cases, online sources and assumptions were also used.

A monetary value (financial proxy) was assigned to each outcome. As cost data were not available from Noise Solution's records, they were obtained from online sources.

To ensure that the analysis reflected the real-world impact of the programme, adjustment factors were applied to the outcomes. These included deadweight, which considers the proportion of outcomes that would have happened without the programme, and attribution, which considers the impact of other services on the outcomes.

The data collected were used to quantify the social and economic benefit experienced by participants engaged with the Noise Solution programme. To calculate the SROI ratio, this was compared to the investment required to run the programme.

Abbreviations: SROI, social return on investment.

4

SROI MODEL METHODOLOGY

Figure 2. Per participant costs associated with each outcome included in the SROI analysis, with and without Noise Solution, across two years £100,000

↓£19,980.93 £10,000 ↓£1,976.99

↓£1,336.44

↓£690.68 ↓£634.33

£1,000

↓£969.32 ↓£208.41

↓£256.31 ↓£18.46

£100

£10

£1 Re-entry into Reduced Reduced/avoided Increased social Diverted from Disengagement Reduced need for Reduced stress Increased labour education, behavioural school exclusion capital/reduced acute mental from mental additional and burden for productivity from employment or issues/school isolation health admission health services support in family members guardians training conduct problems education and and guardians related services Without Noise Solution

With Noise Solution

Abbreviations: SROI, social return on investment.

Copyright © Costello Medical Consulting Ltd

|

5

SROI MODEL METHODOLOGY

Model Methodology Social Return on Investment (SROI) An SROI analysis is an outcomes-based framework that enables organisations to understand, quantify and monetise the social, environmental and economic value created through their services. To assign a financial value to social and environmental outcomes that lack a well-defined market value, financial proxies, derived from published literature, are used. The final output of an SROI analysis is an SROI ratio, indicating the value of an organisation’s services (expressed in cost units, £) delivered per £1 investment.

There are seven principles behind the SROI analysis method, as set out by the SROI Network. 1 These are described in Table 1.

Table 1. Seven principles of the SROI method1 Principle

Description

Involvement of stakeholders

The consultation of well-defined stakeholder groups throughout the analysis ensures that the value of the organisation’s outcomes, and the way in which they are measured, are informed by those directly affected.

Understand what changes

Robust analyses include an evaluation of how both positive and negative change may be created by an organisation, including those which are unintentional.

Value the things that matter

Financial proxies are used to assign a value to outcomes generated by the organisation, for outcomes that do not have a well-defined market value or financial worth.

Only include what is material

Critical selection of the inputs and outcomes to be included in the model is crucial to give a true and fair representation of the value generated by an organisation.

Do not over-claim

A correct evaluation of outcomes involves recognition of the other factors which may contribute to the outcome, beyond the organisation’s services.

Be transparent

An honest account should be provided of when the SROI analysis may be considered accurate. The methodology and results of the analysis should be reported to and discussed with stakeholders.

Verify results

Independent validation by appropriate parties may help to remove subjectivity.

Abbreviations: SROI: social return on investment.

Calculating SROI The SROI ratio is calculated as a ratio of the net benefits generated through an organisation’s activities and the net investment required to run these activities, and represents the value of their services over a defined period of time. The simple equation applied to calculate the SROI is as follows: SROI =

Total benefit (net) Total investment (net)

Any SROI ratio with a value greater than one indicates that more value is being generated than invested.

However, there is currently no agreement for what constitutes a “good” SROI ratio, as methodological differences prevent robust comparison of SROI analyses conducted by different parties. In particular, the SROI ratio estimates yielded by these analyses may vary based on the type of outcomes being included and excluded, the considered time horizon, and the level of conservatism applied while making subjective assumptions.

Copyright © Costello Medical Consulting Ltd

|

6

SROI MODEL METHODOLOGY

Objectives The objective of this SROI analysis was to identify the value generated by Noise Solution services and activities. This report reflects the methodology of the updated SROI analysis, which builds on the original SROI analysis carried out in 2019 by incorporating an updated impact map and alternative inputs for quantifying outcomes. Details of the original SROI model can be found in a separate report. It is hoped that the identified evidence can be used, in conjunction with qualitative evaluations, to support future funding of Noise Solution.

The SROI ratio produced indicates the value generated by Noise Solution per £1 of external investment. The focus of this ratio is placed on social value, as opposed to purely financial. In addition, whilst the conversion of social value to monetary units is analytically useful and helps to convey social value in a widely accepted and tangible format, it was not possible to monetise all the anticipated social outcomes of Noise Solution’s work. Furthermore, whilst the most appropriate model inputs were used wherever possible, some assumptions were required due to limitations in the available data.

The SROI ratio presented in this report therefore provides a conservative and provisional estimate, rather than a comprehensive outlook, of Noise Solution’s impact. The ratio is designed to complement an evaluative assessment of the social value generated by Noise Solution, as well as to identify gaps in Noise Solution’s internal evaluative data.

This report describes the methodology of the updated SROI analysis conducted and details all financial proxies used to monetise the socioeconomic outcomes produced by Noise Solution’s work. The seven principles underpinning the SROI framework, described above, have been adhered to throughout the analysis and this report:1 ⬧

Involvement of stakeholders: The impact map of outcomes generated by Noise Solution were primarily informed by discussions with participants, immediate family/guardians and mentors in the original analysis, with support from other stakeholders within Noise Solution. Where possible, data used in the model were informed by responses from users of Noise Solution services

⬧

Understand what changes: Key outcomes captured by the model were determined, where possible, by looking at relevant changes in users of Noise Solution services. Expert opinions from Noise Solution stakeholders were also used to inform the outcomes

⬧

Value the things that matter: Informed by published financial proxies, the SROI model considers not only the benefits and costs to Noise Solution service users, but also to the wider population through governmental-related costs

⬧

Only include what is material: It is crucial to accurately establish impact in SROI analyses, by estimating how much of an outcome would have happened anyway and what proportion of an outcome experienced by a stakeholder can be isolated as stemming from Noise Solution activities. 1 Model results were therefore adjusted for deadweight, attribution and temporal drop-off. Please see the section on Adjustment for further details on these three adjustment factors

⬧

Do not over-claim: Where estimates and assumptions were made as part of this SROI analysis, these were conservative and clearly stated

⬧

Be transparent: All sources used to inform both outcomes and financial proxies were recorded and are detailed in this report. All decisions (and the accompanying rationale) likely to affect the outcome of the analysis are also detailed in the SROI model

⬧

Verify results: The SROI framework is subjective, and as a result multiple sources were used to justify the inclusion and exclusion of all outcomes and costs. Where possible, discussions with Noise Solution were held to verify outcomes and their association with Noise Solution services

Copyright © Costello Medical Consulting Ltd

|

7

SROI MODEL METHODOLOGY

Scope and Stakeholders Time Horizon The SROI focuses on participant outcomes that are generated within two years of completing the programme.

Longer-term outcomes that occur beyond two years of interacting with Noise Solution are not explored in the analysis. This approach is rationalised by the fact that many model outcomes depend on participation in Noise Solution services (e.g. reduced behavioural problems, increased social capital). Complex interactions between internal and external factors mean that estimates of impact made over longer time horizons are unlikely to be accurate. In addition, a single time horizon across the model facilitates interpretation, compared with a model using varying time horizons across inputs. A duration of two years therefore balanced these requirements to maximise both accuracy and interpretability of estimates.

It should be noted that some outputs are estimated to deliver a different impact in the first and subsequent years after Noise Solution participation. For example, the impact of Noise Solution on productivity loss of guardians includes a partial impact in the year of Noise Solution and a subsequent full impact the following year, due to the time allocated by guardians to attend sessions with participants. The total impact is therefore quantified as an average of the two years.

The model also includes the functionality to account for ‘drop-off’ in value from Noise Solution services throughout the year (see section on Adjustment; see the Model User Guide for practical considerations). For example, it would be reasonable to assume that the impact of the programme on a participant would be lower after 12 months of completion than after one month. This functionality allows for more granular and conservative estimates of the social value produced by Noise Solution.

Stakeholders Several stakeholder groups are considered in the SROI model. Discussions with Simon Glenister, Director of Noise Solution, and Damien Ribbans, Operations Director, assisted in directing the scope of the analyses.

Stakeholder groups include: ⬧

Programme participants

⬧

Immediate family members (e.g. guardians and siblings)

⬧

Other related services: private and government-funded health/social/educational services (including employees)

Musicians tutoring Noise Solution participants and Noise Solution were also considered in the impact mapping stage (see Impact Map); however, it was decided that they were not relevant for inclusion in the SROI model.

This was due to a lack of outcome materiality, with some outcomes relating to these stakeholder groups being difficult to quantify/monetise and other outcomes difficult to accurately attribute to Noise Solution; for example, the supporting of the local community and economy by hiring musician tutors. It was therefore decided that their inclusion may overclaim the benefits of Noise Solution’s programme.

In the original SROI analyses, interviews with one person from each of the key stakeholder groups were conducted. The interviews aimed to identify and confirm the key outcomes of interest for the SROI model, as well as identify outcomes to consider for future data collection and any relevant stakeholders who were not originally considered in the model. In the updated analyses, the original research was complemented with qualitative analyses of feedback received from Noise Solution participants, conducted by Noise Solution employees.

Copyright © Costello Medical Consulting Ltd

|

8

SROI MODEL METHODOLOGY

Population Inputs Noise Solution participants were defined as individuals who had completed the programme. Data on 374 previous participants of Noise Solution were obtained and used in the model to inform age demographic inputs, including median ages and the proportion of participants under 18. Well-being data from previous participants were also incorporated into the model; during the Noise Solution programme, questionnaires reflecting the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) are completed by participants to measure their well-being. A significant improvement in well-being is defined by an increase of at least one in the SWEMWBS score, as per Noise Solution’s suggestion and the previously reported minimum important difference.2 Further analyses of the SWEMWBS data can be found in a separate report.

Population inputs can continue to be updated by Noise Solution to increase representation of the participant population.

Copyright © Costello Medical Consulting Ltd

|

9

SROI MODEL METHODOLOGY

Investments The investment per participant for Noise Solution has been calculated as the cost of participating in the 10-week programme (£2,075). This was deemed an appropriate valuation of the investment, as Noise Solution is costed on a cost-recovery basis, and therefore covers all necessary investment costs incurred.

10

SROI MODEL METHODOLOGY

Activities and Outcomes Impact Map An updated impact map for the programme was developed collaboratively by Noise Solution and Costello Medical. The impact map details the relationship between the resources (inputs) that are used by Noise Solution to deliver their activities, and the direct and tangible results of these activities (outputs). Outcomes are the consequences of the activities on relevant stakeholder groups. Relevant outcomes are listed on the impact map, separated into categories based on the timepoint at which they are expected to be achieved: ⬧

Immediate: Achieved immediately up until three months of engaging with Noise Solution, and when maintained, may result in medium-term or sustained outcomes. Outcomes in this category include developing new skills and improvements in self-esteem and self-confidence

⬧

Short-term: Achieved within three to six months of engaging with Noise Solution, and when maintained, may result in a sustained outcome. Outcomes falling into this category include increased educational attainment and willingness to try new activities and tasks

⬧

Medium-term outcomes: Achieved in six to twelve months, and possibly sustained for a longer period.

Outcomes in this category include increased labour productivity from guardians and reduced need for additional support in other areas through increased resilience and independence

⬧

Long-term outcomes: Achieved from 12 months, for example reduced financial burden on families and reduced isolation

This relationship is called a theory of change and aims to summarise the impact Noise Solution has on society. The finalised impact map is presented in Figure 3.

Copyright © Costello Medical Consulting Ltd

|

11

SROI MODEL METHODOLOGY

Figure 3. Impact map

Abbreviations: FTE: full-time equivalent; NEET: not in education, employment or training; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.

Copyright © Costello Medical Consulting Ltd

|

12

SROI MODEL METHODOLOGY

Evaluating and Monetising Outcomes The outcomes presented in the impact maps were subsequently assessed to determine whether they could be modelled robustly. Attempts were made to capture and quantify the vast majority of these outcomes; however, this was not possible in all cases for a number of reasons: ⬧

Some shorter-term outcomes overlapped with longer term outcomes, such as ‘Increased labour productivity of families’ and ‘Reduced financial burden’. In this case, modelling both outcomes would have led to double counting in the value generated

⬧

Some long-term outcomes affected broad and ill-defined groups of stakeholders, such as ‘Increased awareness of self’. Monetising these outcomes would introduce considerable uncertainty to the findings

⬧

Despite attempts to monetise all outcomes, suitable financial proxies could not be found for some, such as ‘Improved sense of hope’

The following sub-sections detail the outcomes that have been included in the SROI model. For each outcome considered, the parameters and the respective data sources are described, including the indicators and financial proxies used in the model. An indicator is used to determine whether change has taken place. A proxy is a value that is deemed to be close to the desired indicator, for which exact data might be unavailable.1 A summary of the outcomes valued in the SROI analysis is provided in Table 2.

For each outcome, multiple estimates and assumptions have been made. These have been described below.

To decrease the uncertainty in the model, inputs could be replaced by data collected by Noise Solution in the future. Inputs for which this is applicable have been highlighted throughout the input tables using orange shading.

Table 2. Summary of outcomes valued in the SROI analysis Outcome Re-entry into education, employment or training Impact on participant’s education

Reduced behavioural issues/school conduct problems Reduced/avoided school exclusions Reduced need for additional support in education and related services Increased social capital/reduced isolation

Impact on participant’s health

Diverted from acute mental health admission Disengagement from mental health services

Impact on participant’s family

Reduced stress and burden for family members/guardians Increased labour productivity from guardians (increased/returned to work)

Abbreviations: SROI: social return on investment.

For all financial proxies, the costs have been adjusted for inflation using the consumer price index (CPI). The calculation for adjusting the costs has been provided below: Inflated cost =

CPI (current year) x financial proxy CPI (year of financial proxy)

Costs were inflated to the 2023 price due to the 2024 CPI having not yet been published at the time of the SROI analysis. All costs provided in this report are those from original sources and are unadjusted for inflation; please check the model for inflated costs.

Copyright © Costello Medical Consulting Ltd

|

13

SROI MODEL METHODOLOGY

Impact on Participants’ Education Re-entry into education, employment or training “He has made great progress since working with Noise solution, he is now enrolled onto a full-time Music course. The common assessment framework I had open for him has now closed as things at home have greatly improved. His work with Youth Justice has been completed and there have been no more incidents with the police. It has been great to see his journey and the music he has produced.

Thank you for moving him forward with his life!” – Referring Professional

As participants show re-engagement with education, training or employment, there is an expected reduced burden on government social services that would otherwise be spent to support these youth. The proposed indicator for this outcome was the proportion of Noise Solution participants who are not in education, employment or training (NEET) prior to Noise Solution, and those who have shed their NEET status following Noise Solution. It was assumed that individuals with NEET status would not be in any form of education, employment or training for the entirety of a given year. Data on truancy of Noise Solution participants are currently not collected. Therefore, estimates from Noise Solution were used; these are presented in Table 3.

Table 3. Estimated proportion of Noise Solution participants with NEET status Input

Value

Source/Justification

Proportion of all participants with NEET status, before Noise Solution

40.0%

Noise Solution estimate for proportion of participants with NEET status

Proportion of NEET status holders shedding their NEET status, after Noise Solution

30.0%

Noise Solution estimate for proportion of NEET participants shedding their NEET status

Abbreviations: NEET: not in education, employment or training.

The financial proxy used for this outcome was the weekly Jobseekers Allowance (JSA) for individuals aged over 16. Allowances vary for individuals aged 16–24 and over 25, therefore the proportion of participants in each age category was calculated using a Noise Solution dataset of past participants (Table 4) and different costs assigned to each category (Table 5).3 The proportions of participants moving between age groups during the time horizon were not considered. It was assumed that individuals under 16 years old would not be receiving JSA if they are NEET, therefore they were not accounted for in this outcome. Other costs included in the model, including guardian labour productivity, were assumed to already capture the associated costs of individuals aged under 16 having NEET status.

Table 4. Age of Noise Solution participants Input

Value

Proportion of all Noise Solution participants aged 16–24 years

22.9%

Proportion of all Noise Solution participants aged 25+ years

5.3%

Source/Justification

Noise Solution dataset

Table 5. Financial proxy for NEET status Financial Proxy

Value

Job Seeker’s weekly allowance for 16– 24 years

£67.20

Job Seeker’s weekly allowance for 25+ years

£84.80

Source/Justification

Department for Work and Pensions3

Abbreviations: NEET: not in education, employment or training.

Copyright © Costello Medical Consulting Ltd

|

14

SROI MODEL METHODOLOGY

Reduced behavioural issues/school conduct problems As participants develop sense of competency in a new area, and increased engagement in pro-social activities and behaviours in the short term, there is expected to be a reduction in school conduct problems, such as detentions. These outcomes from the impact map have been calculated by estimating the proportion of Noise Solution participants that would receive behavioural complaints that disrupt classroom teaching both pre- and post-participation in Noise Solution. As this outcome is specific to school behaviour, it is applied to schoolaged participants (under 18) with non-NEET status only.

In the absence of any data identified in the literature or collected by Noise Solution, the reduction in school conduct problems has been based on assumptions. It has been assumed that before Noise Solution, 80% of school-aged, non-NEET participants receive behavioural complaints at school. Post Noise Solution, it is assumed that participants who show a significant improvement in well-being have a 65% reduction in receiving behavioural complaints, whilst all other non-NEET participants receive the same number of complaints. These inputs are summarised in Table 6. Participants who shed NEET status and return to school are also considered following Noise Solution. It is assumed that these individuals return to school with the same reduction in behavioural problems as individuals who were in school.

Table 6. Estimated behavioural problems in non-NEET participants in school Input

Value

Source/Justification

Proportion of school-aged participants with behavioural problems, before Noise Solution

80.0%

Noise Solution estimate for number of school-aged non-NEET participants

61.0%

Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues

65.0%

Noise Solution estimate for reduction of behavioural complaints in school-aged, nonNEET participants who have a significant improvement in well-being

Proportion of school-aged participants with previous behavioural problems, who showed reduced behavioural problems post Noise Solution Proportion of reduction in behavioural problems due to Noise Solution

Abbreviations: NEET: not in education, employment or training; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.

The financial proxy used to calculate the value of this outcome was the cost of a school-based social and emotional learning programme, as reported in the PSSRU 2021, in addition to the annual cost of teaching time salary that is lost to management of classroom behavioural issues per disruptive child. 4-6 These values are summarised in Table 7.

Table 7. Financial proxy for managing school behavioural problems Financial Proxy Cost of a school-based social and emotional learning programme

Annual cost of teaching time (salary) that is lost to management of classroom behavioural issues per disruptive child

Value

Source/Justification

£181.00

PSSRU 20214 Cost per child per year

£157.23

The Time and Cost of Classroom Behavior Management (Gormley, 2021)5 Assuming 3 seconds of disruption per occurrence, teachers spend 10.95 hours responding to a single referred child’s behaviour each year. Based on average England (excluding London/fringe) teaching salary of £28,0006 (assumed 37.5 hours/week)

Abbreviations: PSSRU: Personal and Social Services Research Unit.

Copyright © Costello Medical Consulting Ltd

|

15

SROI MODEL METHODOLOGY

Reduced/avoided school exclusion “Without Noise Solution, the student would have become further disengaged from learning and ultimately society, as he felt undervalued and did not see himself as a learner or someone who could be taught by teachers. Noise Solution is absolutely brilliant! Phenomenal way to engage students in learning. Disengaged become engaged.” – Referring Professional

Outcomes relating to young people re-engaging with education, training, employment or voluntary placements are included throughout the impact map. As participants are expected to gain confidence, competency and achievement in the short-term, and hence re-engagement in the medium-term, there is an expected re-engagement (either in mainstream schooling, or other means) of a proportion of those participants who were previously excluded from schooling. This outcome has been calculated as the proportion of participants who have been excluded before Noise Solution, and the proportion of participants who remain excluded after Noise Solution, and has only been applied to school-aged participants. These inputs have been estimated by Noise Solution and are presented in Table 8. It has been estimated that of the ~45% of participants that are expelled/suspended from school, there is a 1:1 split between exclusions and suspensions. This estimate has been used due to a lack of appropriate input data, however should be updated to better reflect the proportion of Noise Solution participants who are excluded from school based on data captured at Noise Solution referral.

Table 8. Estimated school exclusions of participants Input Proportion of school-aged participants who have been excluded, before Noise Solution Proportion of school-aged participants who remain excluded, after Noise Solution

Value

Source/Justification

22.5%

Noise Solution estimate – ~45% of participants have been expelled/suspended from school (1:1 split between exclusions and suspensions)

13.5%

Estimate from Noise Solution – ~40% of participants with prior exclusions re-engage with education, employment or voluntary placements

To estimate associated costs, a report published by New Philanthropy Capital that estimates the total cost of school exclusions in the UK was reviewed.7 Specifically, costs to the education system and social services were extracted. Other costs included in the report that were not considered in the SROI model were impact on earnings, cost to the NHS and cost of higher crime. These were not included as they were either captured elsewhere in the model (e.g. mental health treatment costs), or likely to come to bear outside the duration scope of the model (e.g. physical health costs). The cost of managing the process of an exclusion was applied in the model as a one-off cost in the first year and not considered in subsequent years. A summary of the costs associated with school exclusions is provided in Table 9.

Table 9. Financial proxy for school exclusions, per exclusion Financial Proxy

Value

Cost of managing the process of an exclusion (one-off cost)

£831.00

Net annual cost of alternative educational provisions

£7,181.00

Annual cost of increased use of social services

£1,169.00

Copyright © Costello Medical Consulting Ltd

Source/Justification

Misspent Youth Report: Cost of Truancy and Exclusion, 20077

|

16

SROI MODEL METHODOLOGY

Reduced need for additional support in education and related services “If one of our incredible vulnerable children is not engaging with education or needs support through a difficult time … then Noise Solution is the first port of call. Regardless as to whether they have music as a background. … I can't thank Noise Solution enough for the work they do with our most vulnerable children and young people.” – Referring Professional

Another outcome identified in the impact map is the reduced need for additional support in education and other related services. This outcome has been calculated by estimating the use of educational support services pre- and post-participation in Noise Solution. As this outcome is specific to schools, it is only calculated for school-aged, non-NEET participants.

Three educational services were captured in the model; educational psychologist, special needs support and classroom assistance, based on sources cited in literature.8, 9 The proportion of participants in contact with an educational psychologist, and/or with special educational needs (SEN), is based on the Mental Health of Children and Young People 2017 survey, stratified by type of behavioural disorder (see Table 10).8 In the absence of estimates for classroom assistants, this was assumed equal to the proportion of children accessing SEN support. Of all Noise Solution participants, it was assumed that 80% had either an emotional, conduct or hyperactive disorder, distributed equally. The remaining 20% were assumed to not have a disorder. It is assumed that the prevalence of disorders is irrespective of participant age. Mean use of each service was derived from Romeo 2006 (assumptions of frequency [that the time recurred weekly] were made due to a lack of their being reported), a paper estimating the cost of severe antisocial behaviour in children, which collected service use data for children who had been referred to mental health services in the UK (Table 10).9 Following participation in Noise Solution’s programme, it was assumed that participants who had a meaningful improvement in well-being, measured through SWEMWBS, reduced their use of each service by 65% (estimate provided by Noise Solution). Those without a meaningful improvement in SWEMWBS continued with the usage of services as before. The impact of individuals who shed NEET status are also considered postNoise Solution, and are assumed to follow the same distributions in terms of disorders and service use as individuals who did not have NEET status at the start of the programme. A summary of the inputs derived from literature are presented in Table 10. The resulting estimated proportion of Noise Solution participants receiving support before and following Noise Solutions is presented in Table 11.

Table 10. Education and related service use, as reported in literature Input

Value

Source/Justification

Proportion of all Noise Solution participants with disorder

Emotional disorder: 26.7% Conduct disorder: 26.7% Hyperactivity: 26.7% No disorder: 20.0%

Noise Solution estimate

Proportion of children who sought educational psychology services

Emotional disorder: 20.1% Conduct disorder: 31.9% Hyperactivity: 32.1% No disorder: 1.9%

Mental Health of Children and Young People in England, 20178

Proportion of children with SEN

Emotional disorder: 26.8% Conduct disorder: 42.4% Hyperactivity: 62.9% No disorder: 6.1%

Mental Health of Children and Young People in England, 20178

Proportion of children who sought classroom assistance

Emotional disorder: 26.8% Conduct disorder: 42.4% Hyperactivity: 62.9% No disorder: 6.1%

Assumed equal to SEN

2.00

Romeo 20069 Based on mean use of 2 visits (one-to-one service) in 12 months

Educational psychologist annual use (hours)

Copyright © Costello Medical Consulting Ltd

|

17

SROI MODEL METHODOLOGY

Special needs support annual use (hours)

8.70

Adapted from Romeo 20069 Assumes a mean use of 40 minutes per week across a period of 3 months

Classroom assistance annual use (hours)

43.48

Adapted from Romeo 20069 Assumes a mean use of 2 hours per week across a period of 5 months

Abbreviations: SEN: special educational needs.

Table 11. Estimated use of support in education and related services by participants Input

Value

Proportion of school-aged participants receiving educational psychologist support, before Noise Solution

11.7%

Proportion of school-aged participants receiving special education needs support, before Noise Solution

18.7%

Proportion of school-aged participants receiving classroom assistance, before Noise Solution

18.7%

Proportion of school-aged participants receiving educational psychologist support, after Noise Solution

8.5%

Proportion of school-aged participants receiving special education needs support, after Noise Solution

13.0%

Proportion of school-aged participants receiving classroom assistance, after Noise Solution

11.7%

Source/Justification

Combination of the proportion of individuals who receive selected support according to behavioural disorder, as reported in Table 10

Reduction in support services is applied at the cohort level, so that of participants who have a significant improvement in wellbeing, 65% cease to use support services.

This is equivalent to each of these participants reducing use of services by 65% each, if service use has a 1:1 ratio with disorder intensity.

Participants who shed NEET status have the same reduction in use of education support and related services

Abbreviations: NEET: not in education, employment or training.

Costs for each service were derived from the PSSRU and GOV.UK.4, 10 The cost of an educational psychologist was assumed equal to a paediatric occupational therapy visit. Special needs support and classroom assistance hourly rates were calculated based on the mean wage of a teaching assistant in the UK in 2023 (£23,000), assuming 195 working days and 6.5 hour days.11 Table 12. Financial proxy for additional support in education and related services Financial Proxy Educational psychologist

Classroom assistance and SEN

Value

Source/Justification

£160.00

PSSRU, 20214 Paediatric occupational therapy, cost per hour

£18.15

Cost per hour based on average annual salary of SEN teaching assistant (£23,000)11 with 195 working days per year, and 6.5 hours per day

Abbreviations: PSSRU: Personal and Social Services Research Unit; SEN: special educational needs.

Copyright © Costello Medical Consulting Ltd

|

18

SROI MODEL METHODOLOGY

Impact on Participants’ Health Increased social capital/reduced isolation “The young person I am working with loved the sessions. He had not been in any form of education or learning/social activity for around a year ... The sessions gave him the confidence to interact with other people and the knowledge that he could achieve something given the right environment.”

– Referring Professional

A further outcome identified in the impact map is the increased social capital as a result of Noise Solution participation and thereby reduced isolation. The value of this outcome was approximated by calculating the proportion of individuals experiencing loneliness based on data published by the ONS. 12 This assumes that all participants who join Noise Solution have the same experience of loneliness as someone with a probable mental health disorder. The proportion of individuals experiencing loneliness was reported in individuals aged 11–16 and 17–22, as presented in Table 13. The proportion of individuals experiencing loneliness before Noise Solution was calculated assuming all participants aged 16 and under have the same likelihood of loneliness as one another, and that all participants aged 17 and over have the same likelihood of loneliness as one another. The proportion of participants who no longer experience loneliness following Noise Solution has been estimated by Noise Solution based on discussions with participants who took part in the programme.

Table 13. Estimated loneliness of participants Input

Value

Source/Justification

16.6%

ONS, 2023; Proportion of individuals experiencing loneliness by mental health of child, 2023; aged 11–1612 Assumes that all participants in Noise Solution have the same experience of loneliness as someone with a mental health disorder and participants aged 16 and under have the same likelihood of experiencing loneliness

Proportion of individuals aged 17 and over often or always experiencing loneliness, before Noise Solution

29.5%

ONS, 2023; Proportion of individuals experiencing loneliness by mental health of child, 2023; aged 17–2212 Assumes that all participants in Noise Solution have the same experience of loneliness as someone with a mental health disorder and participants aged 17 and over have the same likelihood of experiencing loneliness

Total proportion of participants experiencing loneliness, before Noise Solution

27.8%

A weighted average of the above inputs based on the age distribution of Noise Solution participants

Proportion of participants experiencing loneliness who are no longer experiencing loneliness, after Noise Solution

30.0%

Noise Solution estimate based on communication with families

Proportion of individuals aged 16 and under often or always experiencing loneliness, before Noise Solution

Abbreviations: ONS: Office for National Statistics.

The financial proxy associated with this outcome was based on an analysis by the Department for Digital, Culture, Media and Sport which estimated the costs associated with loneliness stemming from the impact on subjective well-being, health and productivity.13 For individuals aged 17 and over, the total cost considered the impact of health, productivity and well-being for individuals with moderate to severe loneliness. The report did not consider the cost for individuals under 16, therefore it was assumed that the cost of mild

Copyright © Costello Medical Consulting Ltd

|

19

SROI MODEL METHODOLOGY

loneliness, which does not consider productivity loss, would be an appropriate proxy for individuals 16 and under. The costs have been summarised in Table 14.

Table 14. Financial proxy for loneliness Financial Proxy Cost of moderate to severe loneliness in individuals aged 17 and over

Cost of moderate to severe loneliness in individuals aged 16 and under

Value

Source/Justification

£9,976.00

Loneliness monetisation report, 202013 Assumes loneliness impacts subjective wellbeing, health and productivity

£6,429.00

Loneliness monetisation report, 202013 Assumes cost of moderate to severe loneliness in individuals aged 16 and under is proportionate to mild loneliness in individuals aged 17 and over, to account for the lack of impact on productivity

Diverted from acute mental health admission “Two of the acts booked for last Saturday's gig were made of Noise Solution graduates. Their second gigs. The one girl gave a long, emotional speech on stage about how it saved her life and she wouldn't be here today without it.” – Referring Professional

In-patient admission to mental health services was identified as an outcome impacted by Noise Solution. The proportion of patients with previous in-patient mental health admission prior to Noise Solution was estimated

by Noise Solution, as was the proportion diverted from in-patient admission following the intervention. These proportions were assumed to apply equally to participants irrespective of age.

Table 15. Estimated in-patient mental health admission Input

Value

Proportion of all participants with previous in-patient mental health admission, before Noise Solution

30.0%

Proportion of all participants with in-patient mental health admission, after Noise Solution

5.0%

Source/Justification

Noise Solution estimate

This outcome has been monetised by valuing the cost of in-patient admission for children, which is under the responsibility of CAHMS, and for adults. For children, a report in 2017 reported the average cost of CAMHS in‑patient admission per person (£61,000.00).14 For adults, the average cost per person was estimated using the average cost in-patient admission per day (£446.00), the average length of in-patient admission (49 days) and the costs following discharge from acute medical units (£2,431.00).15-17 The costs have been summarised in Table 16.

Copyright © Costello Medical Consulting Ltd

|

20

SROI MODEL METHODOLOGY

Table 16. Financial proxy for in-patient mental health admission Financial Proxy Cost of average CAMHS in-patient admission

Cost of average adult in-patient admission

Value

Source/Justification

£61,000.00

Children's Commissioner: Children's Mental Healthcare in England14 Average cost of CAMHS in-patient admission from 2017

£24,285.00

East London NHS Foundation Trust, 2020; daily cost of in-patient mental health admission (£446.00)16 The strategy unit, 2019; average length of mental health in-patient admission (49 days)17 PSSRU, 2022; post-discharge costs (£2,431.00)15

Abbreviations: CAMHS: Child and Adolescent Mental Health Service; NHS: National Health Service; PSSRU: Personal and Social Services Research Unit.

Disengagement from mental health services In addition to in-patient mental health admissions, it is also expected that individuals who attend Noise Solution will disengage from community mental health services as a result of improved well-being. In the absence of any data identified in the literature or collected by Noise Solution, it was estimated that 35% of participants use community mental health services prior to Noise Solution. Following Noise Solution, it is estimated that this reduces to 15%. These inputs have been summarised in Table 17.

Table 17. Estimated engagement of participants with mental health services Value

Source/Justification

Proportion of all participants using community mental health services, before Noise Solution

Input

35.0%

Noise Solution estimate

Proportion of all participants using community mental health services, after Noise Solution

15.0%

Noise Solution estimate

As with in-patient mental health admissions, different providers are used for children and adult services. For children, the outcome was monetised by valuing the cost of CAHMS community support per child per year, as reported by the NHS.14, 18 For adults, the cost was estimated using the average cost per attendance of adults requiring mental health support to day care, provided by local authority social services (£58.00) or the private and voluntary sector (£41.00), with the assumption that the adult would attend one or the other, with equal likelihood of each, for the average length of recovery (three sessions). The cost of delivering behavioural activation, a treatment for depression provided in a group setting, was also considered, with the assumption that the adult would attend a complete programme (12 sessions) once a year (£228.00) in addition to local authority/voluntary services.15 These costs have been summarised in Table 18.

Copyright © Costello Medical Consulting Ltd

|

21

SROI MODEL METHODOLOGY

Table 18. Financial proxy for community mental health services Financial Proxy Cost of CAMHS community mental health services, per year

Cost of adult community mental health visits, per year

Value

Source/Justification

£2,518.00

NHS Benchmarking Network: Benchmarking Mental Health18 Average cost of a year of CAMHS community mental health services from 2017

£376.50

PSSRU, 202215 Cost per attendance to day care, provided by social services (£58.00) or voluntary or private sector (£41.00); assumed attendance of three sessions.

Cost per attendance to behavioural activation programme (12 sessions), once a year (£228.00)

Abbreviations: CAMHS: Child and Adolescent Mental Health Service; NHS: National Health Service; PSSRU: Personal and Social Services Research Unit.

Impact on Participants’ Family Reduced stress and burden for family members/guardians “An awful lot of tutors in the past haven’t been able to cope with him because of his disabilities and have given up on him. He used to be a big, big handful! To watch him now is a joy. I've given him so much praise.” – Family of Participant

An outcome identified in the impact map is the reduction in stress and reduced burden experienced by family members, resulting from improvements in participants’ behavioural problems. This outcome estimates the proportion of guardians and siblings experiencing increased stress/caregiver burden, before and after Noise Solution, and applies the cost of a counselling intervention, to represent the value with and without Noise Solution. It is assumed that guardians and siblings are affected by the participant’s behaviour, regardless of the participant’s age.

Separate calculations were made for guardians and siblings, each of which are described in turn. The proportion of siblings affected by the participant’s behaviour pre Noise Solution was based on reported data from the Mental Health of Children and Adolescents 1999 study. 19 Specifically, the answers to ‘child’s problems cause difficulties with other family members’ for children with emotional, conduct and hyperactive disorders was considered. The proportion of siblings affected ranged from 19–35% for individuals with behavioural disorders. Given that not all Noise Solution participants are anticipated to have behavioural disorders, a conservative estimate was used whereby 20% of all siblings are disrupted by the participant’s behaviour. After Noise Solution, it was anticipated that individuals who have a significant improvement in well-being, measured through SWEMWBS, would have a reduction in behavioural problems that would remove the stress and burden experienced by siblings. The average number of siblings per participant was calculated using data reported by the ONS.20 The inputs used to value the reduced stress and burden for siblings are described in Table 19.

Table 19. Estimated stress and burden for siblings Input Proportion of siblings affected by participant's behaviour, before Noise Solution

Copyright © Costello Medical Consulting Ltd

Value

Source/Justification

20.0%

Conservative estimate based on proportion of siblings affected by child’s behaviour according to different behavioural disorders; reported in Mental Health of Children and Adolescents, 199919

|

22

SROI MODEL METHODOLOGY

Proportion of siblings affected by participant's behaviour, after Noise Solution

Average number of siblings per participant

7.8%

Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues All siblings of participants who do not show a significant improvement in well-being remain affected, if already in an affected state

0.75

ONS, 2017; Families with dependent children by number of children20 Equal to the average number of children per family (1.75) minus one (the participant)

Abbreviations: ONS: Office for National Statistics; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.

For guardians, it was assumed that prior to Noise Solution, all guardians showed increased stress resultant of the child’s behaviour, in the absence of relevant data. Stress and caregiver burden post-Noise Solution was calculated in the same way as for siblings, with a proportional reduction based on Noise Solution participants showing a meaningful improvement in well-being. The average number of guardians per participant was calculated based on the proportion of families with one or two guardians using data published from the ONS.21 Table 20 summarises the described inputs used to calculate the proportion of guardians who are impacted by the participant’s behaviour, pre- and post-Noise Solution.

Table 20. Estimated stress and burden for guardians Input

Value

Source/Justification

100.0%

Assumption

Proportion of guardians affected by child’s behaviour, after Noise Solution

39.0%

Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues The model assumes that all guardians of participants who do not show a significant improvement in well-being maintain affected, if already in an affected state

Average number of guardians per participant

1.85

ONS, 2023; Calculated from the proportion of families with one or two guardians21

Proportion of guardians affected by participant's behaviour, before Noise Solution

Abbreviations: ONS: Office for National Statistics; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.

The financial proxy associated with this outcome was the average cost per counselling intervention, as reported in the PSSRU 2021.4 Specifically, the cost applied was for interventions ranging from 9 to 12 counselling sessions (i.e. short- to medium-term) over 12 hours of intervention, at a value of £1,217 per counselling intervention (i.e. per family member affected). It should be noted that this cost is applied as a representative approximation of the ‘value’ of reduced stress and caregiver burden, rather than the actual cost that would be expected to be incurred by family members/guardians experiencing increased stress. The cost has been summarised in Table 21.

Table 21. Financial proxy for stress and burden for family members/guardians Financial Proxy Cost per counselling intervention

Value

Source/Justification

£1,217.00

PSSRU, 20214 Short to medium term intervention, ranging from 9 to 12 counselling sessions (12 hours per case)

Abbreviations: PSSRU: Personal and Social Services Research Unit.

Copyright © Costello Medical Consulting Ltd

|

23

SROI MODEL METHODOLOGY

Increased labour productivity from guardians As participants show increased engagement in pro-social activities and behaviour in the short-term, there is anticipated increased labour productivity from guardians due to increased/return to work. The value of this outcome was approximated by calculating the average number of days absent from work pre, during and post Noise Solution, and applying a pro-rata average wage for each day absent, across both guardians. This outcome is assumed only to impact participants who live with their guardians and/or need adult supervision.

Therefore, the outcome was only considered for participants aged 18 and under.

In the absence of data from Noise Solution, it was assumed that pre Noise Solution, one guardian per participant was absent one day per month, resulting in an annual estimate of 12 days. For the year including and immediately after the 10-week duration of Noise Solution, in which participants attend 10 two-hour sessions, it was assumed that a guardian attended the first five sessions with participants and that this time represented lost productivity in addition to the 12 day background rate. This additional productivity loss was pro-rata adjusted to reflect the proportion of sessions that participants failed to attend (15.8%), assuming that non-attendance was equally likely between guardian-attended and guardian-non-attended sessions. In the case of non-attendance, it is assumed that there is no additional loss of productivity for the guardian above the base rate of productivity loss, i.e. the guardian would have been absent from work on the given day irrespective of a planned Noise Solution session and that the attendance time is absorbed in the 12 day background rate.

For both the year of the programme, and the years following that, it was assumed that the guardians of participants who show a significant improvement in well-being no longer have any absent days from work related to caring for the participant, while guardians of other participants remain at pre-Noise Solution levels.

These inputs are summarised in Table 22. Productivity loss relating to care of the participant was applied in addition to productivity loss due to attendance of sessions with participants in the year of the programme. In subsequent years, only productivity loss relating to the care of the participant is considered, as guardians no longer attend the course with the participant.

Table 22. Estimated loss of productivity of guardians per participant Input

Value

Source/Justification

Annual numbers of days absent from work, caring for participant, before Noise Solution

12.00

Noise Solution estimate of one guardian absent from work one day per month

6.02

Assumes guardians attend half of Noise Solution sessions and that non-attended sessions do not result in loss of productivity.

Assumes that guardians of participants who show a significant improvement in wellbeing no longer have any absent days from work, while other guardians of participants remain at pre-Noise Solution levels

4.68

Assumes that guardians of participants who show a significant improvement in wellbeing no longer have any absent days from work, while other guardians of participants remain at pre-Noise Solution levels

Average numbers of days absent from work, caring for participant, during Noise Solution

Annual numbers of days absent from work, caring for participant, after Noise Solution

This outcome has been monetised by valuing the loss of productivity resulting from guardians being absent from work. The financial proxy for this outcome is the national average weekly wage in the UK (Table 23).22 Table 23. Financial proxy for labour productivity Financial Proxy National average weekly wage

Value

Source/Justification

£682.00

ONS, 202322 Median weekly earning of full-time employees in the UK

Abbreviations: ONS: Office for National Statistics.

Copyright © Costello Medical Consulting Ltd

|

24

SROI MODEL METHODOLOGY

Adjustment To avoid overestimating the impact of Noise Solution, the following adjustments were applied in the SROI model: ⬧

Deadweight: Deadweight involves the recognition that some stakeholders might experience outcomes of interest, even if Noise Solution did not intervene. Deadweight was built into the final calculations for each outcome modelled, by considering the number of stakeholders who would have experienced the outcome due to other influences, in a situation where Noise Solution had not provided the service

⬧

Attribution: Attribution involves the recognition that outcomes achieved may not entirely be due to Noise Solution’s services alone. Service users, for example, may be engaged with several organisations, providing overlapping support. Through use of a percentage, attribution estimates the proportion of impact created entirely by Noise Solution, after deducting the contribution of other services. Only this is accounted for when calculating the final value of the service in question. Please see Attribution for further information

⬧

Drop-off: Drop-off involves the recognition that outcomes may not last indefinitely. The impact of an outcome may decrease over time, or benefits may be more likely to derive from other factors. Drop-off was accounted for by applying monthly percentages to each outcome; please see Drop-Off for further information on how drop-off was calculated, and for drop-off factors

⬧

Discounting: Discounting involves the recognition that outcomes that occur in the future are valued less than those incurred immediately. In the base case, a 3.5% discount rate is assumed across all variables for year two onwards, based on the HM Treasury Green Book for discounting in the public sector.23 The model includes the flexibility for the user to vary this to 5% (more conservative) or 0% (less conservative) for all of the outcomes, or alternatively, the user can adjust it for each outcome individually

Attribution The default values for attribution included in the model were informed through discussion with stakeholders.

From the interviews conducted in the original SROI analysis, it was understood that many participants engaging with Noise Solution simultaneously receive additional support (such as counselling or therapy).

However, Noise Solution is understood to act as an enabler to the effectiveness of these alternative interventions, and therefore remains a key component to the overall success of the participant. Furthermore, the majority of participants who are referred to Noise Solution have already tried alternative interventions that have not delivered the satisfactory results. As a result, it is estimated that the proportion of outcomes observed attributable to other interventions is expected to be relatively low.

In the absence of data, 25% of each outcome has been attributed to another intervention, and therefore not included in the final base case SROI calculation. The model includes the flexibility for the user to vary this to 50% (more conservative) or 0% (less conservative) for all of the outcomes, or alternatively, the user can adjust it for each outcome individually.

Drop-Off All outcomes included in the model are expected to last two years. Estimates for drop-off after one year have been based on long-term data reported in the literature where possible. For outcomes where no data have been identified for estimating drop-off, it has assumed to be 50%.1 The model includes the flexibility for the user to vary this to 50% or 75% for all inputs (more conservative), or 0% (less conservative). Alternatively, the user can adjust this input for each outcome individually. In addition, the model also includes the flexibility to calculate the SROI assuming each outcome lasts for one year only. If this option is selected, drop-off is no longer relevant, and therefore not applied in the model. The drop-off values used in the base case are described in Table 24.

Copyright © Costello Medical Consulting Ltd

|

25

SROI MODEL METHODOLOGY

Table 24. Annual drop-off for each outcome Outcome

Annual Drop-Off

Source

Re-entry into education, employment or training

48%

Drop-off based on Impetus 2017 where 52% of NEET people remained NEET the following year.24 In the absence of data, this outcome has been conservatively assumed to only last two years.

Reduced behavioural issues/school conduct problems

50%

Reduced/avoided school exclusions

50%

Reduced need for additional support in education and related services

50%

Increased social capital/reduced isolation

50%

Diverted from acute mental health admission

50%

Disengagement from mental health services

50%

Increased labour productivity from guardians (increased/returned to work)

50%

Reduced stress and burden for family members/guardians

50%

In absence of alternative data, dropoff has been aligned with a SROI publication of a UK school-based wellbeing programme, which assumes 50% each year.1

Abbreviations: NEET: not in education, employment or training; SROI: social return on investment.

Copyright © Costello Medical Consulting Ltd

|

26

SROI MODEL METHODOLOGY

Sensitivity Analyses A one-way sensitivity analysis was conducted on the base case results to identify the inputs which the SROI was most sensitive to. Sensitivity analysis was conducted through varying the inputs used to calculate each outcome one at a time, with an upper and lower value. The impact on the SROI ratio was subsequently recorded, in order to identify the top ten variables that had the greatest impact on the SROI. The results of the sensitivity analysis are presented in a tornado plot in the model.

Copyright © Costello Medical Consulting Ltd

|

27

SROI MODEL METHODOLOGY

Assumptions and Limitations General Limitations of SROI Methodology Every economic model is a simplification of events happening in the real world, and thus is subject to assumptions and limitations. However, while assumptions and limitations are an inherent part of economic modelling, the model itself can still produce useful and reliable results if limitations are addressed, or results are interpreted within the constraints of those limitations.

SROI analyses are subject to several inherent limitations. First, SROI methodology attempts to quantify and monetise the non-tangible impact of an organisation. The process of assigning financial values to nonfinancial outcomes is inevitably associated with a degree of uncertainty.1 Therefore, it is important to be cautious when drawing detailed conclusions from the analysis. Moreover, the lack of a universally accepted approach to SROI modelling means that few meaningful conclusions can be drawn from the comparison of different organisations’ final SROI ratios. Nonetheless, when used in the right context, SROI analysis is a valuable tool to highlight the breadth and depth of impact of the activities of Noise Solution on its stakeholders in a granular and semi-quantitative way.

Limitations of the Noise Solution SROI Model There was a lack of sufficient evaluative data to fully capture the impact of the Noise Solution programme. In most cases, exact inputs were not available and input-specific assumptions had to be made. Where assumptions have been made for model input values, there were informed by comparable figures, published evidence and discussions with Noise Solution. Therefore, any model input that relies on assumptions is presumed to be close to the true value and any impact on model results is anticipated to be minor.

Furthermore, conservative values were used to minimise the risk of inflating the SROI results, and assumptions were validated by Noise Solution. Noise Solution are currently working to update the information captured about their participants in referrals and their customer relationship management platform, in order to better understand the journey of the individuals that join their programme. Once these data becomes available, the SROI model can better reflect the population.

The lack of sufficient data also meant that certain outcomes and stakeholder groups had to be excluded from the analysis. For example, a significant improvement in well-being has been observed from participation in the programme. However, the paucity of available quantitative data and a lack of a robust way to incorporate this in the SROI model led to exclusion of this outcome from the analysis. Consequently, these limitations mean that the final SROI values are likely to be an underestimate due to the use of conservative assumptions and the exclusion of certain outcomes.

Other limitations of the model include its limited ability to capture the value that beneficiaries of the programme were able to pass on. For example, participants of Noise Solution may have been able to share their experience with peers facing a similar situation and therefore introduce them to the programme. It is expected that impact generation occurred beyond that captured in the model, and this impact is likely to continue to grow beyond the model time horizon as knowledge and expertise continues to spread. However, quantification of the value of these relationships comes with inherent uncertainty and would also require an element of forecasting, which was beyond the scope of this analysis.

Copyright © Costello Medical Consulting Ltd

|

28

SROI MODEL METHODOLOGY

References 1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

Nicholls J, Lawlor E, Neitzert E, et al. A Guide to Social Return on Investment. Available at: https://socialvalueuk.org/resources/a-guide-to-social-return-on-investment-2012/ Last accessed: Feruary 2024: The SROI Network, 2012.

Shah N, Cader M, Andrews WP, et al. Responsiveness of the Short Warwick Edinburgh Mental WellBeing Scale (SWEMWBS): evaluation a clinical sample. Health and Quality of Life Outcomes 2018;16:239.

Department for Work and Pensions. Benefit and pension rates 2023 to 2024. Retrieved from: https://www.gov.uk/government/publications/benefit-and-pension-rates-2023-to-2024/benefit-andpension-rates-2023-to-2024. Last accessed: February 2024, 2023.

Personal Social Services Research Unit (PSSRU). Unit costs of health and social care 2021. Retrieved from: https://www.pssru.ac.uk/project-pages/unit-costs/unit-costs-of-health-and-social-care-2021/.

Last accessed: February 2024, 2021.

Gormley MJ. The Time and Cost of Classroom Behavior Management. Presented at the National Association of School Psychologists Annual Convention. 2021.

Gov.uk. National pay scales for eligible teaching and education leadership occupation codes.

Retrieved from: https://www.gov.uk/government/publications/national-pay-scales-for-eligibleteaching-and-education-jobs/national-pay-scales-for-eligible-teaching-and-education-leadershipoccupation-codes. Last accessed: February 2024, 2023.

Brookes M, Goodall E, Heady L. Misspent Youth - The Costs of Truancy and Exclusion. Available at: https://www.thinknpc.org/wp-content/uploads/2018/07/Misspent-youth-report.pdf Last accessed: Feruary 2024. New Philanthropy Capital, 2007.

Collinson D. The mental health of children and young people in England. Retrieved from: https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-andyoung-people-in-england/2017/2017. Last accessed: February 2024. In: National Centre of Social Research, Office for National Statistics, eds, 2017.

Romeo R, Knapp M, Scott S. Economic cost of severe antisocial behaviour in children - and who pays it. British Journal of Psychiatry 2006;188:547–553.

Education Endowment Foundation. Teaching and learning toolkit. Available at: https://educationendowmentfoundation.org.uk/education-evidence/teaching-learning-toolkit Last accessed: February 2024, 2018.

National Careers Service. Special educational needs (SEN) teaching assistant. Available at: https://nationalcareers.service.gov.uk/job-profiles/special-educational-needs-(sen)-teaching-assistant.

Last accessed: February 2024, 2023.

Office for National Statistics. Mental Health of Children and Young People in England, 2023: Wave 4 follow up to the 2017 survey. Retrieved from: https://digital.nhs.uk/data-andinformation/publications/statistical/mental-health-of-children-and-young-people-in-england/2023wave-4-follow-up/part-1-mental-health. Last accessed: February 2024, 2023.

Department for Digital, Culture, Media & Sport. Loneliness monetisation report. Available at: https://assets.publishing.service.gov.uk/media/602fcb91d3bf7f72154fabc3/Loneliness_monetisation_r eport_V2.pdf Last accessed: February 2024, 2020.

Children's Commissioner. Briefing: Children’s Mental Healthcare in England. Available at: https://www.childrenscommissioner.gov.uk/resource/briefing-childrens-mental-healthcare-in-england/ Last accessed: February 2024, 2017.

Personal Social Services Research Unit (PSSRU). Unit costs of health and social care 2022. Retrieved from: https://www.pssru.ac.uk/project-pages/unit-costs/. Last accessed: February 2024, 2022.

East London NHS Foundation Trust. FOI DA3620. Available at: https://www.elft.nhs.uk/sites/default/files/2022-01/anon_response_-_foi_da3620.pdf Last accessed: February 2024, 2021.

The Strategy Unit. Exploring Mental Health Inpatient Capacity across Sustainability and Transformation Partnerships in England. Available at: https://www.strategyunitwm.nhs.uk/sites/default/files/201911/Exploring%20Mental%20Health%20Inpatient%20Capacity%20accross%20Sustainability%20and %20Transformation%20Partnerships%20in%20England%20-%20191030_1.pdf Last accessed: Feruary 2024, 2019.

Copyright © Costello Medical Consulting Ltd

|

29

SROI MODEL METHODOLOGY

18.

19.

20.

21.

22.

23.

24.

National Health Service (NHS). Benchmarking Mental Health. Available at: https://static1.squarespace.com/static/58d8d0ffe4fcb5ad94cde63e/t/58ecf71de58c62adea37fa27/149 1924766551/BenchmarkingMentalHealthCard2017FINAL.pdf Last accessed: Feruary 2024, 2017.

Meltzer H, Gatward R. The mental health of children and adolescents in Great Britain. Available at: https://www.dawba.info/abstracts/B-CAMHS99_original_survey_report.pdf Last accessed: February 2024. In: Office for National Statistics, ed, 1999.

Office for National Statistics. Families with dependent children by number of children, UK, 1996 to 2017. Retrieved from: https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/families/adhocs/0 08855familieswithdependentchildrenbynumberofchildrenuk1996to2017. Last accessed: February 2024, 2018.

Office for National Statistics. Families and households: 2022. Retrieved from: https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/families/bulletins/ familiesandhouseholds/previousReleases. Last accessed: February 2024, 2022.

Office for National Statistics. Employee earnings in the UK: 2023. Retrieved from: https://www.beta.ons.gov.uk/employmentandlabourmarket/peopleinwork/earningsandworkinghours/b ulletins/annualsurveyofhoursandearnings/2023#:~:text=Median%20weekly%20earnings%20for%20f ulltime%20employees%20was%20%C2%A3682,highest%20growth%20since%20comparable%20recor ds%20began%20in%201997. Last accessed: February 2024, 2023.

HM Treasury. The Green Book: Central Government Guidance on Appraisal and Evaluation. Retrieved from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1 063330/Green_Book_2022.pdf Last accessed: February 2024, 2018.

Impetus Private Equity Foundation. Youth Jobs Index. Available at: https://impetusorg.files.svdcdn.com/production/assets/publications/Report/Youth-Jobs-Index-II-report-final.pdf Last accessed: February 2024, 2017.

Copyright © Costello Medical Consulting Ltd

|

30