MY HOME STARS, INC.
IMPACT ASSESSMENT 2023
“Inspiring Excellence & Lighting the Star Within Every Child”
Table of Contents
About Us
03
Assessment Methodology
04
Qualitative Survey Tool
05
Impact on Teaching
06
Impact on Learning
07
Trend Analysis
08
Visualizing Our Impact
09
Key Themes
10
Acknowledgements
11
Appendix
12
2
ABOUT US MY HOME STARS My Home Stars is a registered 501(C)3 nonprofit (tax exempt #92-3026694) organization with the mission of making education more accessible in a fun and engaging way to underprivileged children in Ugandan refugee camps. Majority of our work involves developing educational board games that break down the Ugandan school curriculum into modularized packages that make learning more engaging through play. Through fostering child-centered learning approaches that encourage active student participation and classroom engagement, we strive to transform classrooms into spaces that allow both children and educators to thrive and reach for the stars
OUR MISSION
Our Impact by Numbers
9
We innovate fun & engaging academic tools that help children & educators thrive in challenging learning spaces.
Schools Served
OUR VISION
200
We hope to inspire children to take the lead in their academic journeys & build a brighter future for themselves & their communities.
Teachers Trained
5 Staz Packages Distributed
3,000 Students Served
8,000 3
Assessment Methodology Two different surveys with five open-ended questions each were distributed to randomly chosen students and teachers from at 4/5 of our partner schools: Planning for Tomorrow, Ngurwe, Coburwas, and Kinakyeitaka primary schools in the Kyangwali Refugee Camp. After gathering data, our analysis team adopted qualitative coding approaches which involved dividing up the various questions and created coding schemes to explore the perspectives and experiences of both students and teachers who have engaged with the 5 Staz game. In addition, we collected historical PrimaryLeaving Examinations (PLE) test scores across each school to analyze trends of academic performance since the implementation of the 5 Staz in 2020. See the Appendix Section I for coding interpretation, example responses, and historical PLE scores.
THE NEED FOR IMPACT ASSESSMENT The 5 Staz game seeks to meet the academic needs of the participating students and provide their teachers with useful tools. When we explore the impact of the game, we can better understand if the games are helping to meet the needs of both groups.
The impact assessment helps us to evaluate the game and the user experience among teachers and students to further identify areas of improvement for our program.
Process Overview Collect Written Feedback Via Surveys
Gather Historic PLE Exam Data
Qualitative Coding & Data Analysis
Evaluation & Findings 4
Presented to
Qualitative Survey Tool Given that the goal for the assessment was to better understand the impact of the 5 Staz game on the students’ and teachers’ experiences, our team designed a set of five questions tailored to both students and teachers. These questions were designed with an intent to capture students’ and teachers’ free expressions of their thoughts and experiences associated with using the 5 Staz game throughout the school year. The key elements that we sought to investigate are the game’s impact on learning and teaching, its impact on relationship building amongst learners, as well as shifts in attitudes about education and learning. Refer to Appendix Section IV for some examples of code interpretation and responses.
Survey Questions for Teachers 1. How has the 5 Staz game impacted your teaching experience in your classrooms? 2. Has the 5 Staz impacted students' behavior in class? If yes, how? 3. Do you think the 5 Staz is a good resource? If yes, what excites you about using the 5 Staz educational game? If no, why? 4. What have you liked most about the game? What would you change about the game? 5. What feedback do you have for My Home Stars team? What does the team need to improve?
Survey Questions for Students 1. What do you enjoy most about the 5 Staz educational game? 2. Do you ever use the game to prepare for exams? Why? 3. How do you use the game to study for exams? 4. Does the game help you make friends at school? How does the game help you make friends? 5. Please tell us one thing you would like to change about the 5 Staz game.
5
Impact on Teaching The chart on the right depicts the emergent themes across the five survey questions for teachers. The majority of them agreed that the 5 Staz games eased teaching and improved students’ collaborations and relationships.
Emergent Theme
Overall, teachers held very positive perceptions of the game and shared that it helped with student learning, community building, and student confidence. See Appendix Section II for tables showing each survey question and their corresponding top responses for the teachers.
Percentage
Eased and improved teaching
65
Improved student collaboration and relationships
65
Aligned well with the primary curriculum and terms
45
Made learning fun and interesting for students
35
The game is very interesting. It improves thinking and encourages competition among learners. It has encouraged learners to create friendships . . . and also [improved] their performance in class.
Violet Regina Teacher at Planning for Tomorrow Primary School
The 5 Staz has profoundly impacted students’ behavior. They are learning to manage their emotions as they are challenged during the play. I think it’s also a great reason to improve their cognitive and emotional well-being.
Okello Louis Teacher at Ngurwe Primary School
It has simplified my teaching topics and has also exposed learners to different ways to ask and approach exam questions. It has also eased my work of teaching, as children some times do their own research ahead of me teaching the content.
Esengo John Baptist Teacher at Planning for Tomorrow Primary School
6
Impact on Learning The student data was sorted and coded to further analyze the results. The emergent themes from the student survey questions are summarized in the table below. All students believe that the 5 Staz game helped them make friends. The majority of them were able to revise the cards in the game to help with exam preparations. See Appendix Section III for tables showing each survey question and their corresponding top responses for the students.
“It sometimes helps us to make friends in school and with pupils from other schools during competitions.” Erithie Muhawe Benjamin Student Planning for Tomorrow
Emergent Theme
Percentage
Game helped students to make friends
95
Students revised the cards to help with exam preparation
85
Students enjoy the 5 Staz's Q&A format
Students love playing the 5 Staz
45
45
“We make friends through discussing and sharing with one another during the game.” Muknyarugelelo Emmanuel Student at Coburwas
“I enjoy asking and answering questions on the cards and spelling the words.” Jibu Sadrack Student at Kinakyeitaka
7
Trend Analysis - PLE Scores As part of the impact assessment, the team was able to compile historical Primary Leaving Examination (PLE) test results from 4 of the primary schools: Kinakyeitaka, Coburwas, Ngurwe, and Planning for Tomorrow Primary Schools. Administered by the Uganda National Examinations Board (UNEB), the PLE is a mandatory exam that certifies the completion of primary education for the Primary 7 students that want to pursue secondary education. 2019
2020
2022
60
Number of Students with PLE Scores
50
40
30
20
10
0
Kinakyeitake
Coburwas
Ngurwe
Planning for Tomorrow
Since the 5 Staz games were introduced in 2020, there is an increase in Division 1 and 2 high scores among students from three out of four all schools.* * See Appendix Section I for the complete set of data.
8
Visualizing Our Impact
95% of students say 5 Staz helped them make friends at school.
8
80% of students say that 5 Staz has
The 5 Staz game has eased my teaching in class. My learners often run their own self study groups and research new content on their own
helped them
200
prepare for exams.
-Kugonza Emmanuel
65%
I enjoy learning with the game and it helps us make more friends at school
of teachers say
-Irafasa Munyarucuro
that 5 Staz eased
Student at Kinakyeitaka
Science Teacher at Planning for Tomorrow
or improved their teaching.
The 5 Staz game allows schools to incorporate child-centered and “gamified” learning approaches in their classrooms. Both teachers and students found that this approach positively impacted their educational experiences.
9
Key Themes The My Home Stars team identified key findings and patterns throughout the assessment. The three main takeaways are that the game helped form positive relationships, increase academic competency, and improve leadership and self-confidence.
2
Academic Competency The majority of the students expressed that the 5 Staz helped them to prepare for their studies and exams. The game's Q & A format allows students to learn effectively and expand their knowledge of the curriculum. Teachers also echoed similar thoughts, and many of them mentioned that the game improved learning and the performance of students. They asserted that the game also eased teaching and helped them to cover the required material.
1
3
Leadership & Self-confidence
All of the students agreed that the 5 Staz game helped them to make friends at school. Many of them mentioned that they enjoyed the group discussions and the competition among peers.
Students said they enjoyed asking each other questions from the 5 Staz game as well as engaging in group discussions. In addition to promoting friendship among students, this activity helps them build confidence in leading discussions and expressing their thoughts.
The teachers brought up the point that the game promotes positive relationships between students. They also mentioned that students are learning how to better collaborate and communicate with peers and the teachers.
Teachers also acknowledged that students communication skills are improving since the introduction of 5 Staz. Since the game adopts a team-oriented and studentcentered learning approach, students are developing strong leadership qualities.
Forming Positive Relationships
10
Acknowledgements A big thank you to the teachers and students from our partner schools who took time out of their busy day to provide valuable feedback in this assessment.
Haircut Shampoo Hair Color
Ngurwe Primary School Coburwas Primary schools Kinakyeitaka Primary School Rukundo Intergrity Primary School Planning For Tomorrow Primary School
Highlights Hair Extensions
And thank you to our fantastic team of academics that unselfishly contributed their expertise to this impact assessment--especially during the assessment design stage, coding efforts and data analysis phase.
Special Occasion Hair
Ashley Welsh - University of British Columbia Ellen Flournoy - Capilano University Kwesi Yaro - University of Alberta Clovis Hatungimana - Quest University, class of 2023 Rahaf Youssef - Caltech University Erick Ziponi - Project Manager, My Home Stars
Connect & Learn More About Our Work
Our Website
Our Instagram
Our LinkedIn
Our Facebook
“Inspiring Excellence & Lighting the Star Within Every Child”
11
Appendix I. Historic PLE Scores 2019 - 2022 KINAKYEITAKA PRIMARY SCHOOL Number of Students Per Division Per Year PLE Grades
2018
2019
2020
2021
2022
Division 1
01
01
12
-
12
Division 2
08
30
38
-
40
Division 3
09
02
05
-
02
Division 4
15
01
01
-
00
U-Failed
06
00
00
-
01
TOTAL
39
34
56
-
55
COBURWAS AND PRIMARY SCHOOL Number of Students Per Division Per Year PLE Grades
2018
2019
2020
2021
2022
Division 1
04
09
13
-
19
Division 2
16
17
18
-
06
Division 3
00
00
01
-
00
Division 4
00
00
00
-
00
U-Failed
00
00
00
-
00
TOTAL
20
26
32
-
25
12
Appendix NGURWE PRIMARY SCHOOL Number of Students Per Division Per Year PLE Grades
2018
2019
2020
2021
2022
Division 1
00
00
00
-
00
Division 2
08
20
12
-
20
Division 3
10
07
12
-
09
Division 4
06
01
00
-
03
U-Failed
00
00
00
-
00
TOTAL
24
28
24
-
32
PLANING For TOMORROW (P4T) PRIMARY SCHOOL Number of Students Per Division Per Year PLE Grades
2018
2019
2020
2021
2022
Division 1
-
01
05
-
03
Division 2
-
14
14
-
22
Division 3
-
06
01
-
00
Division 4
-
O1
00
-
00
U-Failed
-
01
00
-
00
TOTAL
-
23
2O
-
25
13
II. Coded Teacher Survey Responses Teacher Question #1
How has the 5 Staz game impacted your teaching experience in your classrooms?
Code
Number of respondents (out of 20)
Eased teaching
9
Improved/expanded learning & experience
6
Speak to Q&A format
5
Improved teaching
5
Teacher Question #2
Has the 5 Staz impacted students' behavior in class? If yes, how?
Code
Number of respondents (out of 20)
Increased interaction
6
Improved relationships
6
Collaboration
5
Confidence
3
14
Teacher Question #3
Do you think the 5 Staz is a good resource? If yes, what excites you about using the 5 STAZ educational game? If no, why?
Code
Number of respondents (out of 20)
Encourages participation/ collaboration
7
Covers the syllabus
6
Eases teaching
4
Makes learning fun
4
Teacher Question #4
What have you liked most about the game? What would you change about the game?
Code
Number of respondents (out of 20)
Content arrangement
5
Relevance of content
4
Arousal of interest
4
Facilitation of teaching & learning
2
15
Teacher Question #5
5. What feedback do you have for My Home Stars team? What does the team need to improve?
Code
Number of respondents (out of 20)
Thank you (appreciation for MHS!)
9
Other (countless play; t-shirts; stakeholder awareness; improve motivation)
6
Increase reach (schools/grades/access)
4
Inlcude diagrams/images
3
16
III. Coded Student Survey Responses Student Question #1
What do you enjoy most about the 5 Staz educational game?
Code
Number of respondents (out of 20)
Enjoy Q&A format
9
Make friends
5
Play/fun
4
Expand knowledge
3
Student Question #2
Do you ever use the game to prepare for exams? Why?
Code
Number of respondents (out of 20)
Exam preparation
13
Better performance
4
Revision
3
Improve spelling
2 17
Student Question #3
How do you use the game to study for exams?
Code
Number of respondents (out of 20)
Revision
11
Playing
5
Group discussions
5
Practice spelling
1
Student Question #4
Does the game help you make friends at school?
Code
Number of respondents (out of 20)
Yes
19
Group discussions
7
Bring pupils together
3
Playing in groups
3
18
Student Question #5
Please tell us one thing you would like to change about the 5 Staz game.
Code
Number of respondents (out of 20)
Nothing
4
More questions
3
Question setting
2
Game movements
2
19
IV. Code Interpretation & Examples Description
# of Respondents
% of Respondents
Relevant Quotes
Eased teaching
Students had to do prework; content/learning not solely on the teacher; helped cover content faster
9
45%
The 5 Staz game has eased my teaching in class in that learners are exposed to self study, interaction and research of new content.
Improved, expanded learning and performance
Students get to revise content and ask questions, which eased learning and ultimately improved their academic performance.
6
30%
Drastic change in learners' performance / their academic performance boosted. A lot of topics are revised hurriedly. Children listen to each other's ideas. Improvement of creativity, imaginative and cognitive skills in students.
Increased interaction
The game allows more peerto-peer interactions compared to traditional classroom learning
6
30%
The 5 Star impacted students behavior since they interact in groups discussing with other learners, creating friendship for each other hence good behavior.
The students learn how to Cooperation work with one another to / solve problems and develop Collaboration a sense of friendship and teamwork.
5
25%
Yes, by associating with the other in groups. Students help the other in case of reading difficulties.
Code
Covers the syllabus
The game content captures the Ugandan primary education currciculum that covers four core subjects: Science, English, Math, Social Studies.
6
30%
Yes, absolutely good. The game content covers a wide area in syllabus and even go beyond what a teacher may teach in class. Besides academics, the game develop other skills like leadership, effective communication skills in students.
Arousal of learning interest
Students are encouraged to learn and ask questions, which helps create a positive attitude towards learning
4
20%
Children interact without discrimination; Winning motivates learners to learn.
Increase reach (schools/gra des/access)
Teachers want to see the game being made available to more schools in refugee camps and also to the lower primary students
20%
I really recommend the games to be played countless times. Supply game boards to all the schools in the settlement. Kindly motivate the patrons as they execute their duties. Increase the number of times interschool competitions are held. Leadership conferences should be held twice each term.
More competitions / conferences
Teachers really enjoyed having the inter-school game competitions as well as teacher conferences where students and teachers can come together to share their experiences
15%
So far I am appreciative of the support rendered by My Home Stars and I hope you continue supporting us. Focus on organizing competitions and conferences where teachers shares experiences. Provide T-shirts to class coordinators.
4
3
20