IMPACT MEASURING SYSTEM GUIDE
This document has been created to assist you in developing an effective impact measurement system for the programmes delivered to young people This approach was supported by the steps outlined in the BOSS project toolkit and supports Beyond Horizons partners in implementing a structured approach to evaluating the outcomes of their initiatives
By following the below approach, it will support you in understanding the true impact of your work, identifying areas for improvement, and demonstrating the value of your efforts to stakeholders
1. Define the Scope of the Project:
What is your objective?
What is the timescale of the project?
What are the resources you have to undertake the project?
Who do you need to showcase the impacts to – participants, funders, parents?
2. Identify Stakeholders:
Who are your participants (Age range, demographics, experience, issues, abilities / disabilities)
Who else needs to be involved (funders, teachers, youth leaders, parents)
3. Engage with Stakeholders:
Find out what the participants expect to gain from the project – what are they looking for
Find out what the funders expect to gain from the project
Find out what other stakeholders expect you to deliver through the project
4. Select Relevant Outcomes:
What are the Outcomes you are trying to achieve. The BOSS project toolkit highlights a range of outcomes that could be achieved through outdoor sports projects:
Improved physical health
Improved mental well-being
Improved active citizenship
Improved education and lifelong learning
Improved environmental awareness and responsibility
Reduced crime and anti-social behaviour
Other benefits
5. Identify Indicators and Tools:
Identify the indicators that you will use and what tools will be required to provide the evidence on outcomes The toolkit and the Question bank on the Beyond Horizons website provide great ideas of ways to gather the data. Please also see Appendix 1 at the bottom of this document for more suggestions on indicators
The BOSS toolkit provides a list of indicators that can be used to measure the impact of outdoor sports activities
There are standardised indicators such as: whether the participants achieve World Health Organisation (WHO) minimum levels of physical activity for health benefits 1 using standardised questionnaires for mental well being 2 using the nature connectedness index 3.
When working with young people ensure that any surveys used are as short as possible and in very simple language.
What indicators will be used for the programme?
6. Data Collection Plan:
Using a mix of formal and informal methods provides a balanced understanding of data The best method depends on:
Purpose (evaluation vs improvement)
Resources (time, budget, expertise)
Audience (age, access, language barriers)
Collecting data effectively requires careful planning, as it often takes longer than expected. Incentivising surveys, such as offering prize draws, can boost participation.
Formal Data Collection Methods:
Structured processes ensure reliability and comparability:
Surveys & Questionnaires – Efficient, anonymous, and easy to analyse. Interviews – Provide in-depth insights and adaptability
Focus Groups – Encourage discussion and identify trends.
Observations – Capture real-time behaviour and engagement.
Existing Data Analysis – Cost-effective and useful for trend identification
Informal Data Collection Methods
Less structured approaches offer real-time insights:
Informal Conversations – Quick, honest feedback through casual discussions Some of the best conversations can happen over a campfire
Suggestion Boxes – Anonymous input highlighting recurring issues
Anecdotal Reports – Personal testimonials providing rich insights.
Within the Beyond Horizons toolkit section there is an guide and excel template to help with developing surveys with question examples.
How will data be collected?
7. Data Analysis Approach:
Once data is collected, it needs to be analysed to measure the impact of the programme.
Within the Beyond Horizons toolkit section there is an excel template to help with analysing data. Some survey tools provide this for you, but developing a bespoke one for your needs is generally better
Data may be quantitative in nature or qualitative. Significantly more time is required for analysing qualitative data
8. Reporting:
Develop a framework for reporting the findings of the impacts and outcomes This could include written reports but the use of infographics, videos, images and diagrams can be much more powerful.
Remember to tell great stories of your adventures and moments when young people shone or came to life.
Indicators
Selecting the best indicators depends on the specific objectives and outcomes of the outdoor sports programmes for young people However, here are some potential indicators that the partners could consider:
Participation Rates: Measure the number of young people participating in outdoor sports programmes over time. This can indicate the reach and engagement of the programs.
Physical Fitness Levels: Assess changes in participants' physical fitness, such as improvements in cardiovascular endurance, strength, flexibility, and agility.
Mental Well-being: Use surveys or assessments to measure changes in participants' mental well-being, including reductions in stress, anxiety, and depression, and improvements in self-esteem and confidence.
Social Skills Development: Evaluate the development of social skills among participants, such as teamwork, communication, leadership, and conflict resolution abilities.
Environmental Awareness: Assess changes in participants' knowledge, attitudes, and behaviours related to environmental conservation and sustainability.
Academic Performance: Track participants' academic performance over time to determine if participation in outdoor sports programs has a positive impact on educational outcomes.
Community Engagement: Measure the extent to which participants are involved in community activities and initiatives related to outdoor sports and recreation.
Long-term Engagement: Monitor the extent to which participants continue to engage in outdoor sports and recreation activities beyond the duration of the program.
Safety and Risk Management: Assess the effectiveness of safety measures and risk management protocols within the outdoor sports programs to ensure the well-being of participants.
Feedback and Satisfaction: Collect feedback from participants, parents, and program staff to gauge satisfaction levels and identify areas for improvement.