A11.3 Best Practice in Performance Measurement_Plante and De Wit

Best Practice in Performance Measurement of Fall Prevention Training: Toronto Public Health's Step Ahead Program for Health Care Providers and Caregivers to Older Adults Toronto Public Health Susan Plante and Margaret de Wit Public Health/Primary Health Care Conference June 4-5, 2014 1 of 20

Foundations for this Work

Mandates

• OPHS • Accountability Agreements • City Priorities

Partnerships

• LHINs; PHUs • Research Organizations • Service Providers/Agencies

Best Practice Guidelines

• RNAO - BPG • International • Performance Measurement

2 of 20

Step Ahead Program Goals • Improve health outcomes of older people and reduce burden of falls on the healthcare system in partnership with all healthcare providers serving older adults in Toronto

Objectives • Facilitate adoption of best practice fall prevention strategies for healthcare agencies • Improve access to fall prevention services for older adults

3 of 20

Program Target and Reach • Targeted to agencies and staff in Toronto serving healthcare needs of elderly, and to other personal caregivers • 400 agencies currently being reached through marketing, outreach, other strategies • 60 agencies participated in the training in the last 10 months (timeframe of this study)

4 of 20

Step Ahead Module 1: Fall Prevention Education

A Step Ahead to Fall Prevention in Older Adults Training Program for Health and Allied Health Care Providers

5 of 20

Step Ahead Module 2: Exercises (HSEP – CCAA)

6 of 20

Use of Best Practice Performance Measures •

Funders and other stakeholders now requiring tangible evidence of program efficacy, the capacity to produce a desired result

•

Consequently, a growing emphasis on measures of quality and impact in addition to activity (e.g., counts of activities or numbers served)

•

Performance measurement – focus on outcomes and results about efficiency and effectiveness of a program

7 of 20

Methods •

Various theories used to frame questions about acquisition, intention and practice of skills/knowledge

Between July, 2013 and April, 2014 • Self-administered questionnaire following completion of training by 140 participants

•

Electronic or telephone survey of 47 consenting participants at 3 months post training

8 of 20

Methods (continued) Key measures Initial post-training survey:  (Activity) How many agencies/people have we served?  (Quality) 1. Did you acquire knowledge/skills?

2. Do you intend to use/share them?

3 months post-training survey:  (Impact) Have you used/shared your knowledge/skills with clients?

9 of 20

Methods (continued) Additional measures Initial post-training survey: ďƒ˜ (Quality) How satisfied were you with the workshop/training?

3 months post-training survey: ďƒ˜ (Impact) With how many clients have you used the exercises?

10 of 20

Survey Completion Rates

11 of 20

Results - Quality Knowledge Acquisition (initial survey) • 97% reported knowledge increase as a result of the training, with 68% stating substantial knowledge increase • 98% reported new skills learned

12 of 20

Results – Quality (continued) Intention (initial survey) • 97% reported intention to share workshop information with clients • 95% reported intention to use training resources with clients

13 of 20

Results – Impact Practice (3 months post training) • 77% with the opportunity (“yes”/”no”) reported knowledge practice • 41% overall reported “no use” of knowledge or “no opportunity to use”

14 of 20

Results – Impact (continued) Practice (3 months post training) • 68% with the opportunity (“yes”/”no”) used HSEP • 55% overall reported “no use” or “no opportunity to use” HSEP exercises with clients

15 of 20

Results – Additional Measures Quality Satisfaction (initial survey) 96% satisfied/very satisfied

Impact Clients served (3 months post training) Over 200 trained by 21 caregivers

16 of 20

Key Findings and Conclusions • Survey measures of quality (knowledge/skill, intention) and impact (practice following training) appear useful for collecting meaningful performance data • Step Ahead provides participants with high quality practical knowledge and training • Many participants are also integrating Step Ahead knowledge and training into practice

17 of 20

Challenges and Limitations Challenges (3 months post training) • Among those implementing HESP, 21% reported challenges

Limitations • Survey instruments collected information about self-reported knowledge acquisition, intentions and practice, not actual acquisition, intentions and practice • Self-selecting nature of participants studied does not facilitate generalization of results

18 of 20

Next Steps •

Explore possible challenges with implementation and reasons for non-use, despite initial intentions  better identify in-scope agencies and appropriate target audiences to maximize relevance and uptake of training  assess organizational readiness to support the use of Step Ahead

•

Continue with performance measurement and expand program reach to include all healthcare providers of older adults

19 of 20

THANK YOU Questions?

20 of 20