Initial Reports from a Large Scale Evaluation of Outcome Measures across residential care services f

Initial Reports from a Large Scale Evaluation of Outcome Measures across residential care services for people with a Learning Disability and Challenging Behaviours • •

Dr Andrew Hider, Clinical Director, Iris Care Group Molly Bishop, Assistant Psychologist, Ocean Community Services 1

OCS Outcome Measures

iriscaregroup.co.uk

•

The ‘Practice Gap’ means that organisations often prioritise administrative governance over practice / clinical governance. But knowing with reasonable confidence about practice, culture, compliance and staff and service user experience is vital.

•

Lack of central direction re: what is the right combination of • SROMs (Service Reported Outcome Measures) • SREMs (Service Reported Experience Measures) • PROMs (Patient Reported Outcome Measures) • PREMs (Patient Reported Experience Measures)

• • •

Project began in 2018 – as part of restraint reduction and governance work to improve oversight of key safety and quality data.

Huge interruption caused by Covid lockdowns Resumed focus late 2021. early 202

•

Data collected over 11 LD residential homes

•

Outcomes to be collected monthly/quarterly/yearly depending on measures. All homes have a schedule to follow.

• Here we outline our initial findings when analysing this data set.

2

Positive Behaviour Support (PBS) Plans

iriscaregroup.co.uk

• PBS plans can easily become ornamental • A measure of implementation is staff team engagement in active review of the plan (with the service user) • Standard formats for monitoring across the organisation.

• The more you check the more you get.

• Care always benefits when staff are actively involved in plan review and feedback.

• Since January 2022 there has been a steady increase in the percentage of PBS’s that have received positive monitoring feedback from the staff team during monthly MDT’s.

• In July 2023 was at 70% • Aim is 100% by end 2023.

3

Building Better Lives (BBL)

iriscaregroup.co.uk

• BBL is a framework created by Iris Care Group grounded in active support and goal setting, which support individuals to identify skills and activities they wish to participate in. BBL goals are monitored to track progress as individuals develop skills.

• Increase over time in goals that are reviewed (i.e. changed) • Not able to make direct equation between frequency of review and quality (as some goals may be long lasting…)

• However, it’s likely that the more frequent reviews are the more that people are achieving goals and building skills.

• Important to overview how dynamic care is (and can be) for each individual.

• Skills gaps are often key drivers of challenging behaviour, environments must be designed to support with skills using active support principles.

4

Building Better Lives (BBL)

iriscaregroup.co.uk

• This chart represents the average percentage of BBL goals that clients improved on each quarter.

• On average, clients improved on 36% of their BBL goals every 3 months.

• This may increase with more frequent goal reviews (more appropriate goals and/or better documentation…).

• Cohort-based evaluation is critical (it may be very good that improvement on a third of goals was seen depending on cohort) • It may indicate the goals have been set too high.

• Information supports evaluation 5

PREM – Service Experience Questionnaire

iriscaregroup.co.uk

• The Service Experience Questionnaire was developed by Iris Care Group. This asks clients about the care they receive, if they feel supported, and if they feel hopeful about their future.

• Completed by the people we support, with assistance from staff.

• It has a total score of 68, and a hope score out of 5.

• Here we can see the average total score, and hope score, by home.

(Review period Jul-Sept 2023) • Generally positive results.

• Tool needs norming Note: Blank scores are due to non completion. This is engagement information!

6

PROM – MANS-LD

iriscaregroup.co.uk

• The Maslow Assessment of Needs Scale - Learning Disabilities (MANS-LD) is a quality of live outcome measure developed by Skirrow and Perry (2014).

• This was completed by people we support, with assistance from staff.

• The MANS-LD is scored out of 95.

• Here we can see the average score across each residential home (Review period Jul-Dec 2023).

(Skirrow and Perry, 2014) https://www.researchgate.net/publication/269634979_MANS-LD_Easy_Read_Retrospective

7

Quality Indicator of Rehabilitative Care –

iriscaregroup.co.uk

Supported Accommodation (QulRC - SA) • The QulRC-SA is a tool used to assess the quality of care provided by specialist support accommodation services. This measure was developed by UCL as part of the QuEST (Quality and Effectiveness of Supported Tenancies for people with mental health problems) study.

•

It provides ratings across 7 care domains: • Living Environment • Therapeutic Environment • Treatments and Interventions • Self-Management and Autonomy • Social Interface • Human Rights • Recovery-based Practice.

• This outcome measure is completed by the Home Manager.

• It is a mental health tool but fits the needs of learning disability services.

• Supports self-evaluation around a range of areas of good quality social care practice. https://quirc.eu/quirc-sa/

8

Quality Indicator of Rehabilitative Care –

iriscaregroup.co.uk

Supported Accommodation (QulRC - SA)

•

• •

•

• https://quirc.eu/quirc-sa/

This represents the rating of each domain across residential homes.

Social Interface scored the lowest across most homes Living Environment scored the highest across most homes Indicates the extent to which focus on social inclusion is important.

Internal and external drivers for degrees of social inclusion 9

Group Home Culture Scale (GHCS) •

• •

•

The GHCS was developed by La Trobe University (2019) to assess teams views on the staff culture occurring within the residential home.

The tool is completed anonymously by all members of the staff team.

It evaluates colleagues feedback domains of supporting residents wellbeing, social distance from residents, valuing residents, collaboration with the organisation, alignment of staff with organisational values, factional behaviour, and effective team leadership.

This heat map represents scores on each subscale (scored 0-5) across each residential home. All scores are above the norm cut-offs so the heatmap is comparative.

https://doi.org/10.1111/jar.12693

iriscaregroup.co.uk

10

PBS Audit

iriscaregroup.co.uk

• In March 2022, the Court of Protection submitted further guidance on developing high quality and informative PBS plans. Using these current guidelines we developed an auditing system to evaluate if our current PBS plans meet all elements of this guidance.

• This graph looks at the percentage of criteria met in our current PBS plans, by residential home.

• Here we can see most homes are meeting over 80% of criteria outlined in the guidance.

• The guidance is highly exacting and not included in any existing NICE guidance or good practice guides.

11

PBS Audit – Critical Safety Items

iriscaregroup.co.uk

• Within the PBS audit there are 22 safety items that relate to physical safety, use of medication, deprivation of liberty or other statutory requirement.

We have highlighted these as the most important criteria to meet.

• Heatmaps can be helpful to quickly visualise compliance • Each row in this heatmap represents compliance with the audit requirements.

• The 2 areas for improvement were ‘involving the client in the development of their plan’ and ‘creating a debrief timeline’.

• Deviations from 100% are rectified immediately • Lower scores give an indicator of potential need for additional training/MDT focus on particular areas in the plan.

12

Level of Concern

iriscaregroup.co.uk

• Level of Concern was developed as a means to triage and track clinical risk.

• The ratings are based on an operationalised scale including behavioural, physical health, and other relevant items • Important for service leaders in particular to be able to be aware of current status so that additional support / resources and risk management can be overseen.

• These are updated monthly in MDTs using this criteria …

13

Level of Concern

iriscaregroup.co.uk

Level of Concern

• This heat map represents the documented levels of concern for all community clients between January 2023 to August 2023.

14

2024 Update

iriscaregroup.co.uk

Data from 2023 indicated some prioritiy of improing compliance with monthly ‘positive monitoring’ Additional training and support interventions put in place for staff teams and managers, emphasising importance of staff and service user feedback re: use of plan and staff and service user evaluation of plan effectiveness / helpfulness.

15

Conclusions

iriscaregroup.co.uk

• Transparent sharing of quality related data across Wales is important if we are to ensure meaningful cross comparison of CB services and the continuous improvements we all want to see.

• Challenge : Some data is threshold-based (normed), whereas other data requires judgement to interpret (e.g. how should we interpret ‘success’ or ‘failure’ in active support / skill building?)

• Giving data back to users and staff is a core principle of all quality improvement / person centred care.

• Challenge : Reducing care to numbers/categorical evaluations brings intrinsic tensions.

• Advice is to focus on this kind of data oversight after you have embedded the 6 core strategies re: restraint reduction – unless there is a focus on improved environments and staff experience, positive measures can be neglected / hard to get people to engage with.

• Measures of culture are critically important – the GHCS is not perfect but it is important that people have a private space to talk abut the culture of the care environment. A Wales standardised approach would be helpful • Being clear about the need for compliance is also important. It is helpful if as well as internal (governance) drivers for this there are true evidence-based, credible external drivers (i.e. contractual compliance, national guidance, regulatory guidance) that is granular and specific (i.e. not just ‘is there a PBS plan, is the plan signed? Etc • AI Large Language Models (LLMs) will revolutionise the use of and availability of outcome data across health and social care.

16

Thank you andrew.hider@iriscaregroup.co.uk molly.bachelor@iriscaregroup.co.uk

Unit 1, Castleton Court, Fortran Road, St Mellons, Cardiff CF3 0LT t: 03300 567 888

iriscaregroup.co.uk