Analyses of 2024 SWEMWBS scores by Costello Medical

NOISE SOLUTION

Analyses of SWEMWBS Scores from Noise Solution

11 APRIL 2024 Costello Medical UK | US | Singapore | China www.costellomedical.com

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Contents List of Figures................................................................................................................3 List of Tables.................................................................................................................3 Glossary.........................................................................................................................4 Context..........................................................................................................................5 Objective.......................................................................................................................6 Methods.........................................................................................................................6 Data Sources.........................................................................................................................................6 Analysis Approach..................................................................................................................................6 Software............................................................................................................................................6 Data Cleaning and Transformation.......................................................................................................6 Data Summaries.................................................................................................................................8 Analyses.............................................................................................................................................8

Results.........................................................................................................................10 Summary Statistics...............................................................................................................................10 Demographics...................................................................................................................................10 Analyses..............................................................................................................................................10 Change in SWEMWBS scores.............................................................................................................12 Meaningful Changes in SWEMWBS Scores..........................................................................................14 Transitions in Well-Being Bands.........................................................................................................14 Fujiwara Social Value Change................................................................................................................16

Discussion....................................................................................................................17 Limitations and Cautions.......................................................................................................................18

References...................................................................................................................19 Appendix......................................................................................................................20 Methodology........................................................................................................................................20 Statistcal Results Summary...................................................................................................................21 Additional Results.................................................................................................................................23

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

List of Figures Figure 1. Individual changes in SWEMWBS score after participation in the Noise Solution programme, ranked by magnitude of changea..........................................................................................................................13 Figure 2. SWEMWBS scores at the start and end of Noise Solution participation, by gender a,b.......................13 Figure 3. Transitions between high, medium and low well-being categories a,...............................................15 Figure 4. SWEMWBS scores as a function of age, at the start and end of the Noise Solution programme a.....23 Figure 5. Change in SWEMWBS score as a factor of participant age a...........................................................23 Figure 6. Change in SWEMWBS scorea following participation in the Noise Solution programme as a factor of SWEMWBS score at the start of the programme........................................................................................24 Figure 7. The number of participants in each well-being band at the start and end of the Noise Solution programme, by age and gendera, b............................................................................................................25 Figure 8. Transitions in Fujiwara (2017) SWEMWBS value categories 8.........................................................26

List of Tables Table 1. Research questions.......................................................................................................................6 Table 2. SWEMWBS well-being band definitions..........................................................................................7 Table 3. Fujiwara (2017)8 categories with associated SWEMWBS scores and social values..............................8 Table 4. Summary of analyses performed...................................................................................................9 Table 5. Ages of Noise Solution Participants, by gender.............................................................................10 Table 6. Summary of analyses results.......................................................................................................11 Table 7. SWEMWBS scores at the start and end of the Noise Solution programme, and change in SWEMWBS scores, by gender....................................................................................................................................12 Table 8. Proportions of participants showing positive, negative and no meaningful change in SWEMWBS scores, and their respective changes in score............................................................................................14 Table 9. Changes in value for participants, by increase type and gender.....................................................16 Table 10. R package used for research questions 1–6................................................................................20 Table 11. Results of analyses...................................................................................................................21 Table 12. Transitions between high, medium and low well-being SWEMWBS bands a....................................24

Copyright © Costello Medical Consulting Ltd

|

Glossary Technical terms used throughout the report are defined here. Please also refer to footnotes in individual tables and figures to support interpretation.

Term

Definition

Alpha

Pre-specified statistical significance threshold

Centring

Adjusting data values by subtracting the mean (average) from each value, shifting the average to zero. This technique simplifies analysis by aligning the data around a common starting point

Intercept

The value of the dependent variable when all other variables are set to zero

Linear model

A statistical model that presents the relationship between an outcome (or outcomes) of interest (dependent variable[s], such as change in SWEMWBS score) and other factors that might be associated with the outcome of interest (independent variables, such as age and gender)

Link function

A tool used in some statistical models to ensure that the model’s predictions stay within a range reflective of the real-world. Different link functions, such as logit or identity, are used depending on the type of data/their values

Log10

Data may be transformed using this logarithm to make them more suitable for analysis using linear models. To convert from Log10(value) to the absolute value, calculate as absolute value = 10Log (value) 10

Logistic model

A statistical model that estimates the likelihood of a binary outcome (such as yes/no) by fitting data to a curve instead of a straight line (as in linear models)

Mean

The most commonly used type of average-of-all-values in a dataset, calculated as the sum of all the datapoints divided by the number of datapoints. This type of average is affected by extreme values

Median

The middlemost value in a dataset, or half-way between the two middlemost values when there is an even number of values or entries in a dataset. This type of average is not biased by extreme values or outliers and so is often used when these are present in a dataset

Mixed model

A statistical modelling approach (applied in some instances here to both linear and logistic models) that includes both fixed factors (effects which are consistent across individuals or groups) and random factors (effects which vary across individuals or groups, representing natural variability or differences not explained by the fixed effects)

Multinomial model

A statistical model that predicts outcomes where there are more than two possible categories, such as SWEMWBS meaningful change category (e.g. low, medium and high)

N

The number of datapoints in a given group. For instance, the overall N of participants in the SWEMWBS analysis dataset was 374

n

The number of datapoints in a subgroup, such as the number of male or female participants in the analysis

P value

Represents the probability that a pattern in the data (i.e. the observed effect of a given variable) is due to chance. For instance, whether differences in SWEMWBS scores between genders are likely to be ‘real’ or due to random chance. Higher p values mean a real effect is less likely; a value <0.05 is considered statistically significant, whereby we interpret the observed effect as real

Paired t test

A statistical test used to determine if there is a significant difference between the means of two related groups, such as participants’ SWEBWBS scores before and after Noise Solution participation

Quartile 1 and Quartile 3

The values under which 25% and 75% of datapoints are found, respectively, when they are arranged in increasing order. These values provide a measure of variation that is not biased by extreme values or outliers and is usually presented alongside the median

Standard Deviation (SD)

A measure of how much data are spread out around the mean, alongside which it is usually reported

Variable

Anything that can take on different values, such as age, gender or starting SWEMWBS score (independent variable), on which a measured outcome of interest (dependent variable, such as change in SWEMWBS score) might be dependent upon

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Context Noise Solution’s programme aims to instil feelings of autonomy, competence and relatedness; evidenced in Self Determination Theory as the basic psychological needs for well-being to flourish. 1 Their 1:1 music mentoring programmes gives individuals facing a variety of challenging circumstances time to work on and direct a creative project that uses a combination of music technology and instruments. The participant can share their success with friends, family and professional keyworkers in a way that looks and feels like a social media feed, inviting their network to comment on the digital story. In this way, the programme creates a cycle of positive affirmation from people whose opinions matter to the young person.

The result of programme participation is typically improved participant well-being, quantitatively measured using a National Health Service validated scale (Short Warwick-Edinburgh Mental Well-being Scale [SWEMWBS]), a questionnaire which participants are encouraged to self-complete at the start and end of the programme. The SWEMWBS uses seven statements regarding thoughts and feelings, which focus primarily on functioning, and are positively worded with five response categories ranging from ‘none of the time’ to ‘all of the time’.2, 3 Using these scales, children and young people are asked to describe their lived experiences over the past two weeks. Importantly, improved well-being has been empirically shown to improve education, engagement, social and health (both mental and physical) outcomes, ultimately impacting individual quality of life and reducing the financial burden on local and governmental services. 4-6 A minimum important change threshold of ±1 point in SWEMWBS score has been reported and may be considered to constitute a meaningful change (positive or negative),3 whilst scores ±1 standard deviation either side of the UK national average may be considered to represent categories of ‘high’ and ‘low’ well-being, respectively. 2, 7 Over time, Noise Solution have accrued a sizeable SWEMWBS dataset. This dataset was used in the social return on investment (SROI) model, developed by Costello Medical for Noise Solution, to inform various model estimates (further details are available in a separate report). Furthermore, based on the work of Fujiwara et al. who developed a method for estimating the social value created by interventions which improve people’s mental health and subsequent well-being (as measured via SWEMWBS), it is possible to ascribe an estimated financial value to the improvements in well-being demonstrated by Noise Solution participants. 8 These latter estimates, however, were not considered appropriate to integrate within the SROI model, since the methods of Fujiwara et al. produce an indirect estimate of ‘how much additional money or income would be required to have the same impact on well-being as a change in the SWEMWBS score’ and not an actual estimate of cost saving based on the downstream effects of such a change. Instead, these values can be calculated separately to provide an alternative financial indicator of the beneficial effects of Noise Solution’s programme on well-being.

Given the richness of Noise Solution’s available dataset, there existed an opportunity to pursue further analyses, including statistical interrogation of the observed effects of Noise Solution’s programme and application of the Fujiwara method to estimate an alternative financial indicator for changes in subjective well-being. Therefore, the purpose of this report is to present the results and underlying methodology of a suite of statistical analyses of Noise Solution’s available SWEMWBS dataset.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Objective The objective of the analyses was to answer the research questions outlined in Table 1.

Table 1. Research questions Question 1

Are male and female participants of similar average age?

2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

3

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

4

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

5

How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

6

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme?

7

What is the social value change associated with change in SWEMWBS scores between the start and the end of the programme?

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Methods Data Sources The core dataset comprised data collected from 374 individual Noise Solution participants who completed the programme. Age and gender (male; female; non-binary; other; prefer not to say) were collected from participants, who could withhold information if so desired. Participants also completed the standard SWEMWBS questionnaire,7 from which pre- and post-Noise Solution programme participation SWEMWBS scores were derived.

Analysis Approach Software All data manipulation, visualisation and analysis was conducted in R, version 4.3.0. 9

Data Cleaning and Transformation A small number (n=11) of participants included in the dataset had attended the Noise Solution programme multiple times (maximum rounds: 3); for each of these participants, only the first round of participation was retained and subsequent rounds were excluded. This removed dependence between points and any effect of multiple rounds of participation on scores. One participant was additionally removed as the recorded age (4 years) was confirmed by Noise Solution to be erroneous and the validity of other data for this individual was therefore uncertain.

A small number of participants declined to provide information on gender (n=1), or stated ‘other’ (n=3) or ‘non-binary’ (n=1). These groups were too small to treat separately in analyses and all such responses were therefore replaced with ‘not applicable’ in order to group them alongside those participants who provided no information on gender.

For all statistical analyses, where age and SWEMWBS scores were used as predictors, these variables were centred. Doing so ensured that the intercept for models was at the mean value, not at zero. This was

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

considered more meaningful, because neither age nor SWEMWBS scores would ever equal zero in this dataset and thus centring prevented the model from predicting zero values.

Based on the previously reported minimum important difference, a change in SWEMWBS score (Δ SWEMWBS score) of at least one point increase or decrease between the start and end of the Noise Solution programme was considered a meaningful change.3 Participants were thus categorised according to the meaningful change category into which they fell: 

Up: a meaningful increase in score; ΔSWEMWBS score ≥1



Down: a meaningful decrease in score; ΔSWEMWBS score ≤1



None: no meaningful change in score; 1> ΔSWEMWBS score <1

Participants were also mapped to well-being bands at the start and end of the Noise Solution programme, based on their respective SWEMWBS scores. Following the methods of Fat et al. (2016), these bands represent functionally important levels of well-being and were defined using the national average statistics for SWEMWBS scores (mean: 23.5; SD: 3.9).2, 7 These data were used so that Noise Solution participants could be put into the context of broader society. Bands were defined as follows in Table 2.

Table 2. SWEMWBS well-being band definitions Definitiona

Band Low Medium High

SWEMWBS Score Threshold

SWEMWBS score < (Mean – SD)

≤19.5

(Mean – SD) > SWEMWBS score < (Mean + SD)

>19.5–<27.5

SWEMWBS Score > (Mean + SD)

≥27.5

2, 7

Footnote: Mean and SD refer to the national mean (23.5) and national SD (3.9).

Abbreviation: SD, standard deviation; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

SWEMWBS scores were mapped to social value changes, following the methods of Fujiwara (2017), who used linear modelling to attribute social value to categories formed from discrete ranges of SWEMWBS scores (see Table 3).8 Participants’ start and end SWEMWBS scores were rounded down (to be conservative) to the nearest whole number, allocated to the Fujiwara categories and attributed the appropriate social value; changes in value were then calculated as the starting value (pre Noise Solution) subtracted from the ending value (post Noise Solution).

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Table 3. Fujiwara (2017)8 categories with associated SWEMWBS scores and social values Category

SWEMWBS Score

Social Value (£)

1

7–14

0

2

15–16

9,639

3

17–18

12,255

4

19–20

17,561

5

21–22

21,049

6

23–24

22,944

7

25–26

24,225

8

27–28

24,877

9

29–30

25,480

10

31–32

25,856

11

33–34

26,175

12

35

26,793

Abbreviation: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Data Summaries Summary statistics were calculated from the dataset; missing values were excluded from calculations of averages and associated variation.

Analyses The analyses conducted on the dataset are outlined in Table 4. A paired t-test was used to compare ages between male and female participants. For the remaining analyses, more complex models were constructed to incorporate estimates for the effects of multiple variables at once, and to accommodate different response variable statistical distributions. Model types and link functions were specified based on the expected statistical distribution of the response variable. Across all research questions, an alpha (significance threshold) of 0.05 was used; consequently, any p value below this value was considered to represent a ‘statistically significant’ observed effect. Further details of the analyses’ methodology are available in the Appendix.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Table 4. Summary of analyses performed Research Question

Dependent Variable

Variables

Model Fitted/Test Performed

1

Are male and female participants of similar average age?

Agea

Gender

Paired t-test

2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

SWEMWBS Scores (start and end of programme)

Gender, age,a start versus end of Noise Solution programme

Linear mixed model (identity link)

3

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

Change in SWEMWBS Scores (from start to end of programme)

Gender, age,a starting SWEMBWBS score

Linear model (identity link)

4

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

SWEMWBS meaningful change category (up, down or none)

Gender, age,a starting SWEMWBS score

Multinomial model (logit link)

5

How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

SWEMWBS wellbeing band (low or non-low; from start to end of programme)

Gender, age,a start versus end of Noise Solution programme

Logistic mixed model (logit link)

6

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme?

SWEMWBS wellbeing band after programme for participants in the low band at the start

Gender, agea

Multinomial model (logit link)

Footnote: Log10 participant age was used, as this ensured that the distribution is suitable for linear modelling.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

Results Summary Statistics Demographics The distribution of Noise Solution participants by age and gender is presented in Table 5. The majority (67.1% of all participants) of participants were male. The median age of all participants was 14 years, ranging between 8 and 60 years old. Although the median age of females was higher than males, this was not statistically significant (p=0.13). The vast majority were legally children, being less than 18 years old (85.6% of all participants), and 71.7% of all participants were less than 16 years old.

Table 5. Ages of Noise Solution Participants, by gender Age (Years)

Proportion (%)

Gender

N

Proportion of N (%)

Median

Minimum

Maximum

Quartile 1

Quartile 3

<18b

<16

All

374

100.0

14

8

60

13

16

85.6

71.7

Female

85

22.7

15

9

60

13

17

82.3

62.0

Male

251

67.1

14

8

53

12

16

86.1

73.0

38

10.2

14

11

25

13

15

89.5

84.2

a

Other

Footnote: a This category includes participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1); sample size was too small to produce meaningful analyses for these groups; b i.e. legally considered children and of school age (% adult can be calculated as 100 – %<18 years old).

Analyses A summary of the analyses results are presented in Table 6. Further details of the results of each research question are provided thereafter, and detailed results of the statistical tests are provided in the Appendix.

Copyright © Costello Medical Consulting Ltd

|

Table 6. Summary of analyses results Research Question 1

Were male and female participants of similar average age?

2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

3

4

5

6

Age

No evidence for a difference in age between sexes (p=0.13)

Start versus end of programme

Significant increase in SWEMWBS scores after programme compared to the start of the programme (p<0.0001)

Gender

Females have significantly lower SWEMWBS scores than males at the start and end of the programme (p<0.001)

Age; start versus end of programme

Older participants have significantly lower SWEMWBS scores than younger participants at the start of the programme, but not at the end of the programme (p=0.01)

Gender

No evidence for gender affecting change in SWEMWBS score between the start and end of the programme, i.e. males and females increased by a similar amount, on average (p=0.08)

Age

Older participants have significantly larger positive differences in SWEMWBS scores between the start and end of the programme than younger participants (p=0.02)

Starting score

Participants with higher-than-average starting SWEMWBS scores are likely to have smaller differences, or even reductions, in SWEMWBS scores between the start and end of the programme compared with participants with average or lower-than-average starting SWEMWBS scores (p<0.0001)

Gender

No evidence for gender affecting meaningful change category between the start and end of the programme (negative change: p=0.42; no meaningful change: p=0.80)

Age

No evidence for age affecting meaningful change category between the start and end of the programme (negative change: p=0.36; no meaningful change: p=0.06)

Starting score

The higher the starting SWEMWBS scores of participants, the more likely they are to have no meaningful change (p<0.001), or a negative meaningful change (p<0.0001) compared with a positive meaningful change between the start and end of the programme

Start versus end of programme

Participants are significantly more likely to be in the low well-being band than the non-low well-being bands at the start of the programme compared with the end of the programme (p<0.001)

Gender

Females are significantly more likely than males to be in the low well-being band compared with the non-low well-being band (p<0.001)

Age; start versus end of programme

The older a participant, the more likely they are to be in the low well-being band than the non-low well-being band at the start of the programme, but this relationship is not present at the end of the programme (p<0.0001)

Gender

No evidence for gender affecting transitions from the low well-being band to the medium (p=0.52) or high (p=0.14) well-being bands between the start and end of the programme

Age

No evidence for age affecting transitions from the low well-being band to the medium (p=0.26) or high (p=0.22) well-being bands between the start and end of the programme

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

Resulta

Variable

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the Noise Solution programme?

Footnote: a Green signifies the variable having a statistically significant effect on the result; yellow signifies a lack of statistical significance.

Abbreviations: SE, standard; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

Change in SWEMWBS scores The average change in SWEMWBS score following Noise Solution participation was an increase of 2.5 points (SD: 4.32); this varied between 1.83 and 2.80 across genders (Table 7).The individual changes in SWEMWBS score are presented in Figure 1, which demonstrate that whilst the majority of individuals experienced an increase in SWEMWBS score, some participants experienced no change or a decrease in score. The overall change in SWEMWBS score after Noise Solution participation across all participants was shown to be statistically significant (p<0.0001; Figure 2).

At both the start and end of the programme, average participant SWEMWBS scores were lower than the national average score of 23.5.7 Females had significantly lower SWEMWBS scores, on average, at both the start and end of the programme, than males (p<0.001; Figure 2). However, the changes in SWEMWBS scores between the start and end of the programme were similar for males and females. Those in the ‘other’ gender category had the lowest SWEWMBS scores but represented a mixed group with a small sample size, limiting interpretability of this finding.

Table 7. SWEMWBS scores at the start and end of the Noise Solution programme, and change in SWEMWBS scores, by gender Gender All Female Male Othera

SWEMWBS Score

Start or End of Programme

Mean

SD

Start

20.34

3.93

End

22.83

4.41

Start

18.76

4.13

End

21.56

4.15

Start

20.95

3.86

End

23.44

4.55

Start

19.85

2.84

End

21.68

3.05

a

Change in SWEMWBS Score Mean

SD

2.50

4.32

2.80

4.50

2.49

4.42

1.83

3.05

Footnote: This category includes participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1); sample size was too small to produce meaningful analyses for these groups.

Abbreviations: SD: standard deviation; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 1. Individual changes in SWEMWBS score after participation in the Noise Solution programme, ranked by magnitude of changea

Footnote: a The broken orange lines indicate thresholds for meaningful changes in SWEMWBS scores, where a change of at least 1 SWEMWBS point (increase or decrease) was considered meaningful.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Figure 2. SWEMWBS scores at the start and end of Noise Solution participation, by gender a,b

Footnote: a the central, horizontal line in the box plot indicates the median value, the box horizontals indicate the first and third quartiles, and the whiskers indicate the range (excluding outliers). Outliers (closed circles on figure) are defined as values greater than 1.5 times the extent of the third quartile, or less than 1.5 times the extent of the first quartile; b participants for whom gender was not known, as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1), were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Older participants had significantly lower scores at the start of the programme compared to younger participants (see Appendix, Figure 4). However, at the end of the programme, there was no correlation between age and SWEMWBS score, meaning that older participants had a larger positive increase in score than younger participants (see Appendix, Figure 5).

Participants with a higher SWEMWBS score at the start of the programme had a smaller increase in their score following the Noise Solution programme compared with participants with lower SWEMWBS scores. For individuals who experienced a decrease in SWEMWBS score following the Noise Solution programme, participants with a higher initial SWEMWB scores had a larger decrease in score compared with participants with a lower initial SWEWMBS score (see Appendix, Figure 6).

Meaningful Changes in SWEMWBS Scores The majority (61.0%) of participants showed a meaningful increase in SWEMWBS score, whilst 17.6% experienced a meaningful decrease and 21.4% showed no meaningful change (Table 8). Individuals with higher-than-average starting SWEMWBS scores had a statistically significant increase in likelihood of seeing a negative meaningful change (p<0.0001) or no meaningful change (p<0.001) compared with a positive meaningful change. Thus, participants with lower starting SWEMWBS scores were more likely to experience meaningful increase in SWEBWBS score after participation in the Noise Solution programme. Neither gender nor age was a significant predictor of meaningful change category.

Table 8. Proportions of participants showing positive, negative and no meaningful change in SWEMWBS scores, and their respective changes in score Meaningful Change Categorya

Value

Positive

Negative

None

Proportion of population (%)

61.0

17.6

21.4

Number of individuals

228

66

80

Mean SWEMWBS score change

4.96

−3.06

0.07

Standard deviation of SWEMWBS score change

3.42

2.42

0.58

Mean SWEMWBS starting score

19.24

23.11

21.17

Median SWEMWBS starting score

19.25

23.21

20.73

Footnote: a A change of at least 1 SWEMWBS point was considered meaningful.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Transitions in Well-Being Bands The transitions in well-being bands between the start and end of the Noise Solution programme have been presented visually in and numerically in the Appendix (see Table 12). At the start of the Noise Solution programme, the majority of the population were in the low and medium well-being SWEMWBS bands (43.42% and 53.74%, respectively). Following the programme, 59.36% of participants did not change well-being band and 34.76% experienced an improvement in their band, resulting in a general shift towards higher well-being bands and away from the low band. However, some participants did decrease band (5.88%), including 0.27% moving from high to low and 4.81% from medium to low. Overall, participants were significantly less likely to be in the low well-being band at the end of the programme compared to the start (p<0.001).

Well-being bands were also compared between genders and ages (see Appendix, Figure 7). Females were significantly more likely than males to be in the low well-being band compared with other bands (p<0.001).

Older participants were also more likely to be in the low well-being band compared to other ages (p<0.0001), however this was only found at the start of the programme. Among those participants who started in the low well-being band, age and gender did not affect the band which they were in at the end of the programme.

Figure 3. Transitions between high, medium and low well-being categories a,

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Footnote: a Box size is proportional to the number of participants in each category before and after the Noise Solution programme, and arrow width is proportional to the number of participants experiencing a given transition (see Table 12); b methodology based on Fat et al. (2016).2 Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Fujiwara Social Value Change Across all participants, the average Fujiwara SWEMWBS value change per participant was +£3,478.48 (Table 9), corresponding to transitions to nearby categories (see Appendix, Figure 8). However, some participants experienced extreme value changes with increases as great as +£25,856.00 and declines of as much as – £21,049.00, corresponding to extreme positive or negative transitions. The average value change was also found to be higher for females (+£4,645.51) than males (+£3,146.61).

Table 9. Changes in value for participants, by increase type and gender Increase Type

Value Change (£)a Gender

3

Maximu m

Minimum

2,912.0 0

0.00

7,316.00

25,856.00

−21,049.0 0

6,878.6 7

3,488.0 0

0.00

10,689.0 0

24,877.00

−12,255.0 0

3,146.6 1

5,900.9 3

2,536.0 0

0.00

5,383.00

25,856.00

−21,049.0 0

Otherc

3,060.1 1

4,815.8 2

2,896.0 0

0.00

5,306.00

15,238.00

−5,306.00

All

6,391.3 5

4,898.8 7

5,306.0 0

2,912.0 0

9,639.00

25,856.00

0.00

Female

7,699.0 3

4,985.4 0

8,358.0 0

3,254.0 0

11,229.7 5

24,877.00

0.00

Male

5,761.7 1

4,950.9 5

3,838.5 0

2,536.0 0

7,922.00

25,856.00

0.00

Otherc

6,118.9 1

3,917.6 8

5,306.0 0

3,488.0 0

8,576.00

15,238.00

0.00

SD

Median

All

3,478.4 8

6,057.8 0

Female

4,645.5 1

Male

All

Positive score change onlyb

Quartiles 1

Mean

Footnote: a Defined as the amount of additional money or income required to have the same impact on well-being as the corresponding change in the SWEMWBS category, following methods by Fujiwara 2017;8 b defined as all participants showing a change in SWEMWBS score >0; c this category includes participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1); sample size was too small to produce meaningful analyses for these groups.

Copyright © Costello Medical Consulting Ltd

|

Discussion SWEMWBS data collected by Noise Solution from a total of 374 participants who had participated in their programme were analysed to investigate possible patterns and trends with regards to the effectiveness of the programme. Overall, the majority of participants identified as male (67% of all participants versus 22% identifying as female), however the age of participants was balanced between male and female, with an average age of approximately 14 years. 85.6% of all participants were under 18 years of age when they first started participating. At the start of the programme, mean SWEMWBS scores were typically markedly lower than the national average (20.34 versus a national average of 23.5), 2, 7 with males having significantly higher SWEMWBS scores than females (average: 20.95 versus 18.76, respectively; p<0.001). In population-level reports of well-being, there does not seem to be the same discrepancy between males and females, 10 suggesting that females attending the Noise Solution programme differ from the wider population in typically having lower starting well-being than males who attend the programme. Starting SWEMWBS scores were also typically lower in older participants.

Overall, participants had significantly higher SWEMWBS scores at the end of the programme than at the start, such that they were only slightly below the national average (overall average end score: 22.83; mean change: 2.50; p<0.0001). Of all participants, 61.0% experienced a meaningful improvement of at least one SWEMWBS point, whilst 21.4% did not experience a meaningful change; only a small number of participants showed meaningful decreases in SWEMWBS score after participation (17.6%). Akin to the difference seen at the start of the programme, at the end of the Noise Solution programme, males still had significantly higher SWEMWBS scores than females (average: 23.44 versus 21.56, respectively; p<0.001), however the average level of improvement was similar between genders (mean score change in males: +2.50; females: +2.80).

Older participants experienced significantly greater increases in SWEMWBS score than younger participants (p=0.02), offsetting their lower starting values, such that there was no significant effect of age on SWEMWBS score at the end of the programme.

The magnitude and direction of change in SWEMWBS score was significantly linked to starting score (p<0.0001), such that those with higher starting scores tended to have smaller positive, and occasionally negative, changes in SWEMWBS by the end of the programme compared with individuals with lower starting scores. Applying bands of well-being (‘high’, ‘medium’ or ‘low’ based on participants having SWEMWBS scores greater than, within, or lower than one standard deviation of the national average), overall, participants tended to increase (34.8%) or maintain (59.4%) their pre-Noise Solution well-being band, with only a small proportion experiencing a negative band change (5.9%). This led to an overall substantial increase in the proportion of participants in the ‘high’ well-being band by the end of the programme, and a reduction of those in the ‘low’ band, whilst the proportion in the ‘medium’ band had a slight increase. There was no effect of age or gender on the ability of participants to transition away from the low well-being band.

When applying the Fujiwara well-being financial proxy to the dataset, on average, the Noise Solution programme provided substantial net value gain (+£3,478.48), and this was higher for females (+£4,645.51) than males (+£3,146.61), however some participants experienced a negative value change. Participants typically transitioned to nearby well-being value categories, however some individuals experienced more extreme changes. Consistent with the findings for well-being bands, the greatest increases in value categories were most often seen in those who started in the lower categories, whilst those who started in the highest categories more often did not transition or transitioned in a slightly lower category.

The present analyses demonstrate a marked and statistically significant overall effect of the Noise Solution programme on participant well-being, as measured by the SWEMWBS. Furthermore, application of the Fujiwara well-being financial proxy complements the findings of the SROI analysis, demonstrating an alternative, but similarly favourable, monetary-equivalent value of the programme. Future investigation may be warranted to try and understand why older participants seem to respond better to the programme.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Limitations and Cautions The data analysed in this suite of analyses were not from a randomised sample and no control group or condition exists. Thus, all results presented within this report must be interpreted with caution and without extrapolation to wider populations. Most importantly, it should be reminded that observed relationships (associations) may not be causal in nature and may instead be attributed to unmeasured factors outside of the Noise Solution programme. Specifically, it is important to note that only limited demographic data were available for participants; other key factors known to affect SWEMWBS scores and associated major categories, such as ethnicity and diet (amongst many other possible factors) 2 were not available and hence could not be included in the analyses. This may affect the results surrounding the variables we do have, as effects may be dependent on unavailable characteristics, which could mask true effects, or create false effects.

Furthermore, caution must be applied in the interpretation of findings. For instance, reported trends are average trends; there is substantial variation around such data, so they are not deterministic (e.g. a participant with a low starting score is more likely to see a larger increase in end score, but this cannot be said with certainty). Without a matched control group or condition, the overall effect of Noise Solution cannot be separated from any background processes. For example, some participants showed reduced SWEMWBS scores at the end of the programme, but it is impossible to know whether this was related to Noise Solution participation (i.e. the programme was detrimental to the individual), whether participation failed to stop a reduction (i.e. the programme had no effect on the individual) or mitigated some but not all reduction (i.e. the programme had a positive effect, but this was not enough to completely reverse the effects of external challenges). Finally, it must be recalled that participants who did not complete the programme, and thus did not have a post-Noise Solution SWEMWBS score, were not included in the dataset, which may bias results towards demonstrating positive changes.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

References 1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.

Ng Fat L; Mindell J B, Stewart-Brown Evaluating and establishing national norms for the short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) using the Health Survey for England.

Quality of Life Research 2016;26:1129-1144.

Shah N, Cader M, Andrews WP, et al. Responsiveness of the Short Warwick Edinburgh Mental WellBeing Scale (SWEMWBS): evaluation a clinical sample. Health and Quality of Life Outcomes 2018;16:239.

Boehm JK, Kubzansky LD. The heart's content: the association between positive psychological wellbeing and cardiovascular health.

Glenister S. A study of stakeholder perceptions of Noise Solution's practices: measuring impact on the well-being of youth facing challenging circumstances. Faculty of Education: University of Cambridge, 2017.

Gutman LM, Vorhaus J. The Impact of Pupil Behaviour and Wellbeing on Educational Outcomes, 2012.

Warwick Medical School.

Fujiwara ea. HACT, MentalHealth_and_LifeSatisfaction_web.pdf, 2017.

R_Core_Team. R: A Language and Environment for Statistical Computing_. R Foundation for Statistical Computing, Vienna, Austria. <https://www.R-project.org/>. 2023.

Office for National Statistics. Personal well-being in the UK, quarterly: April 2011 to September 2021., 2022.

Hartig. F. DHARMa: Residual Diagnostics for Hierarchical (Multi-Level / Mixed) Regression Models. R package version 0.4.6., 2022.

Fox J WSARCtAR, Third edition. Sage, Thousand Oaks CA., 2019.

Douglas Bates MM, Ben Bolker, Steve Walker. . Fitting Linear Mixed-Effects Models Using lme4.

Journal of Statistical Software 2015;67:1-48.

Yee TW. Vector Generalized Linear and Additive Models: With an Implementation in R. New York, USA: Springer, 2015.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Appendix Methodology In order to conduct the paired t-test to address research question 1, data were first investigated for normality to confirm the test was valid. Normality is a method of evaluating how data are distributed around a central point (the average); normally distributed data have most values spread around the centre, with fewer values occurring further away from that centre, forming a ‘bell curve’ when plotted on a graph. Simulated residuals of other models were also inspected with respect to their assumptions of normal distribution and homogeneity of variance using the Dharma package.11 In order to meet the requirements of a paired t-test, age, which was right-skewed (meaning many participants were younger than the average and only a few were older), was subject to a Log10 transformation prior to conducting analyses, to allow the data be handled appropriately in the analysis.

Model selection followed the process of backwards elimination. Initial models included all pairwise interactions between included predictors; three-way interactions were not included as they were not underpinned by an expected mechanism and resulted in over-parameterisation of models. The Car::Anova function in R was then used to test for significance of terms using type 3 sums of squares. 12 Non-significant interaction terms were removed sequentially in descending order of p value. Non-significant main variables were not removed from the model. Where an interaction term was significant but one of its constituent main effects was not significant, the interaction was still retained for investigation as it was likely caused by a crossover or similar effect. P values from regression analysis, rather than the analysis of variance (ANOVA) used to select models, were used to assess significance of terms in the final models as they are relevant at a category level for discrete variables, which was of greater interest. For the multinomial regressions, which cannot support missing values in independent variables, only individuals with complete data for relevant characteristics were included in these analyses.

The R package::function used for research questions 1–6 are described in Table 10.

Table 10. R package used for research questions 1–6 Research Question

R Package::Function

1

Are male and female participants of similar average age?

Stats::t.test9

2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

Lme4::lmm13

3

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

Stats::lm9

4

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

VGAM::vglm14

5

How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

Lme4::lmm13

6

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme?

VGAM::vglm14

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

Statistcal Results Summary Table 11 provides results of the statistical analyses completed for research questions 1–6.

Table 11. Results of analyses Research Question

Variablea,b

1

Are male and female participants of similar average age?

Log10(Age)

2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

3

4

Interceptc Gender[Male]

f

Reference Level

Estimate Type

Estimate (SE)

Test statistic (df)

P Valued

NA

NA

0.026 (0.017)

t = 1.52 (126.95)

0.13

NA

19.02 (0.42)

NA

NA

Female

1.90 (0.47)

t = 4.09 (320)

<0.001

State[End]

Start

2.50 (0.24)

t = 10.6 (321)

<0.0001

Log10(Age)

Mean age

−5.95 (1.83)

t = −3.26 (518.65)

0.001

Log10(Age) × State[End]

Mean age; Start

6.79 (1.86)

t = 3.66 (321)

0.01

Interceptc

NA

1.87 (0.45)

NA

NA

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

Copyright © Costello Medical Consulting Ltd

Gender[Male]

f

0.9 (0.52)

t = 1.74 (319)

0.08

Log10(Age)

Mean age

4.23 (1.75)

t = 2.41 (319)

0.02

Starting Score

Mean start score

−0.43 (0.06)

t = −7.60 (319)

<0.0001

Interceptc

NA

Down: 0.28 (1.41)

NA

None: 0.37 (1.35)

NA

Gender[Male]f

Female; meaningful increase

Down: 0.72 (1.50)

z = −0.82 (NA)

0.42

None: 0.92 (1.42)

z = −0.25 (NA)

0.80

Mean Log10(Age); meaningful increase

Down: 0.27 (4.16)

z = −0.92 (NA)

0.36

None: 0.08 (3.82)

z = −1.90 (NA)

0.06

Log10(Age)

|

Female

Response (change in SWEMWBS scores)

Odds ratio

NA

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Estimate (SE)

Test statistic (df)

P Valued

Mean starting score; meaningful increase

Down: 1.32 (1.05)

z = 5.71 (NA)

<0.0001

None: 1.16 (1.04)

z = 3.48 (NA)

<0.001

Interceptc

NA

1.56 (1.41)

z = 1.29 (NA)

NA

Gender[Male]f

Female

0.25 (1.51)

z = –3.35 (NA)

<0.001

Variablea,b

Research Question

Reference Level

Starting Score

5

6

How does age and gender affect the well-being band of participants, and does the wellbeing band that participants occupy at the start of the programme differ from the band that they occupied at the end?

Estimate Type

State[End]

Start

0.21 (1.32)

z = –5.67 (NA)

<0.001

Log10(Age)

Mean age

409.59 (5.69)

z = 3.46 (NA)

<0.0001

0.003 (7.98)

z = –2.86 (NA)

<0.01

High: 0.04 (2.82)

NAe

NA

Medium: 0.99 (1.35)

z = -0.03 (NA)

High: 4.99 (3.00)

z = 1.47 (NA)

0.14

Medium: 1.27 (1.45)

z = 0.64 (NA)

0.52

High: 18.13 (10.59)

z = 1.23 (NA)

Medium: 4.81 (4.08)

z = 1.12 (NA)

Odds ratio Log10(Age) × State[End]

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the Noise Solution programme?

Intercept

b

Gender[Male]

Log10(Age)

Mean age; Start

NA

f

Female

Mean Log10(Age)

Odds ratio

NA

0.22 0.26

Footnote: a Both age and starting score are centred around the mean; b Contents in square brackets describe subgroups; c the intercept is the value of the estimate when all variables are at their reference level; d this represents the p value from the regression analysis, not the ANOVA used to select models; statistically significant p values are in bold – p values for categorical variables show the likelihood that the variable is not different from the reference, whilst values for continuous variables show the likelihood that the slope (i.e. the strength of association) of the relationship is 0. If a significant interaction is present, this should be used for interpretation over the main effects; e a z-value is not reported here due to detection of Hauk-Donner effect, which, due to low data availability, artificially inflates the z-value; f participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1) were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

Abbreviations: SE, standard error; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|

Additional Results Figure 4. SWEMWBS scores as a function of age, at the start and end of the Noise Solution programme a

Footnote: a The teal lines represent the age component of the linear model. Log10 participant age is shown, as this ensures that the distribution is suitable for linear modelling; to convert to absolute age, from Log10(age), calculate as age = 10 Log10(age), where Log10(age) is a value on the bottom (x) axis. Closed circles represent individual participant datapoints.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Figure 5. Change in SWEMWBS score as a factor of participant agea

Footnote: a The teal line represents the age component of the linear model and the orange dashed lines show the thresholds for meaningful change (score change at least 1 point). Log10 Participant age is shown, as this ensures that the distribution is suitable for linear modelling; to convert to absolute age, from Log10(age), calculate as age = 10Log10(age), where Log10(age) is a value on the bottom (x) axis.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale. Closed circles represent individual participant datapoints.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 6. Change in SWEMWBS scorea following participation in the Noise Solution programme as a factor of SWEMWBS score at the start of the programme

Footnote: a The teal line represents the age component of the linear model and the orange dashed lines show the thresholds for meaningful change (score change at least 1 point).

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Table 12. Transitions between high, medium and low well-being SWEMWBS bands a Start Band

End Band

Number of Participants

Percentage of Population (%)

High

7

1.87

High

Medium

3

0.80

Medium

Low

Low

1

0.27

High

34

9.09

Medium

149

39.84

Low

18

4.81

High

11

2.94

Medium

85

22.73

66

17.65

Change: Up

Low

130

34.76

Change: Down

22

5.88

No Change

222

59.36

a

Footnote: SWEMWBS has a mean of 23.5 and a standard deviation of 3.9 in UK general population samples and the cut-point for the high wellbeing band was set at 27.5 (mean + 1 SD) and for the low band at 19.5 (mean – 1 SD), with the medium band being between these points, following methodology of Fat et al. (2016) and using mean and SD values from the Warwick Medical School website. 2, 7

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 7. The number of participants in each well-being band at the start and end of the Noise Solution programme, by age and gendera, b

Footnote: a To convert to absolute age, from Log10(age), calculate as age=10Log10(age), where Log10(age) is a value on the bottom (x) axis; b participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1) were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

Copyright © Costello Medical Consulting Ltd

|

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 8. Transitions in Fujiwara (2017) SWEMWBS value categories 8

Footnote: Grid headings show starting category, as defined by Fujiwara (2017)8 (no participants started in categories 10 or 11), the size of the purple point indicates the proportion of all participants in that starting category, which is also denoted by the value (n) underneath the point. The thickness of the teal line indicates the percentage of participants who started in that category and transitioned to each Fujiwara category, also denoted by the value at the end of each line.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Copyright © Costello Medical Consulting Ltd

|