AFTER SCHOOL Impact Report 2013-2014 experienceBELL.org
3,756 Scholars
After School Impact Report
K-8 Grades Served
87% Average Daily Attendance
2013-2014 BELL (Building Educated Leaders for Life) exists to transform the academic achievements, selfconfidence, and life trajectories of children living in under-resourced, urban communities. In the BELL After School program, scholars in grades K-8 engage in academic instruction led by certified teachers and hands-on enrichment courses. The program model engages parents and positive adult role models from the community. BELL partners with school leaders and teachers to expand learning time and reach the scholars who need it most. During the 2013-14 school year, 3,756 scholars participated in BELL After School at 33 school-based sites in Boston & Chelsea, MA; New York City, NY; Newark, NJ; Charlotte, NC; and San Jose, CA. BELL partnered with school districts, public schools, and community organizations to strategically target for enrollment those students who were struggling in school and who could benefit the most from academic and social development opportunities. BELL scholars participated in small-group academic instruction in reading and math and enrichment activities for 120180 minutes per day, 3-5 days per week, for 20-36 weeks. School leaders & teachers measured program outcomes against three major goals for scholars: increased academic performance, enhanced selfconcept, and improved social skills. Additionally, assessment activities sought to measure parent engagement and satisfaction with the program.
72% Scholars with scaled score gains in reading
69% Scholars with scaled score gains in math
93% Parents reporting that they are more engaged with their child’s education because of BELL
95% Parents expressing high satisfaction with BELL’s services
97% Teachers reporting that expanded learning time in BELL After School helped them develop their professional skills Page 1 of 5
Schools
Scholars
Boston & Chelsea, MA
Grade
584
K-6
1,237
K-8
614
K-8
Baltimore, MD
1,037
K-8
Charlotte, NC
256
3-5
San Jose, CA
87
7-8
3,756
K-8
New York City, NY Newark, NJ
TOTAL
Average Daily Attendance 90% 85% 82% 91% 83% 85% 87% 50%
75%
100%
IMPACT MEASUREMENT
SCHOLAR ACHIEVEMENT
ATTENDANCE DATA: BELL manages scholar
SCALED SCORE INCREASE: At the start of the
enrollment and tracks attendance data in an online Scholar Management System powered by Salesforce.com. Program leaders integrate enrollment and attendance data with staffing and quality indicators to optimize the learning environment.
BELL After School program, formative assessments provided teachers with baseline indicators of scholar performance along with individualized learning plans. Approximately 72% of scholars started the program performing below grade level in reading, and 61% were behind in math.*
TEST DATA: Teachers use STAR Enterprise Assessments on a pre- and post-program basis. These computer adaptive assessments are built for measuring progress against Common Core standards. Scholars complete multiple-choice assessments via computer, laptop, or iPad, at the start and end of the after school program. The assessment adapts to scholar answers. If a scholar selects the correct answer, the next question becomes more difficult. If the scholar answers incorrectly, the next question becomes easier. Each assessment includes approximately 40 items.
Assessment data from the end of the BELL After School program show that the majority of scholars increased their scaled scores in reading and math. Scaled scores are useful in comparing student performance over time and are calculated based on the difficulty of items and the number of correct responses. Because the same range is used for all students, scaled scores are also useful for comparing student performance across grade levels. Increases in scholars’ scaled scores suggests that participating in after school activities could have a positive impact on academic achievement. The data also hints that expanded learning opportunities may have the greatest impact on scholars who stand to benefit the most: those who are performing below grade level.
TEACHER SURVEYS: BELL developed a teacher
instrument to assess parent observations of scholar improvement, program quality, and parent satisfaction. Parents complete the anonymous survey on-site during the final week of the program.
Math
PARENT SURVEYS: BELL developed a parent survey
Reading
and tutor survey instrument to assess program implementation and efficacy from the perspective of instructional staff. Teachers and tutors complete the anonymous survey at the end of the program, which includes questions regarding scholars’ progress, training, curricula, BELL staff and service, and parent engagement.
50% *
Underperforming scholars score in the lowest two quartiles on STAR assessments: Urgent Intervention (1-25th percentile) or Intervention (26-50th percentile)
72% 76%
69% 72% 75%
100%
% of all scholars increasing their scaled scores % of underperforming scholars increasing their scaled scores
Page 2 of 5
GAINS IN SELF-CONFIDENCE & SOCIAL SKILLS: Scholars’ academic success is rooted in their selfefficacy skills, determination, and their social skills, including their ability to communicate clearly and work well with others. Teachers and parents consistently reported that scholars’ participation in the BELL After School program helped them make important gains in these areas. Such a positive impact can be attributed to a culture of high expectations, high-quality staff, a small staff to scholar ratio, and an individualized learning environment.
PARENT & TEACHER ENGAGEMENT & SATISFACTION: Teachers and parents consistently reported that BELL’s programming was of high quality and met or exceeded their expectations. Teachers reported that the program structure and resources helped scholars achieve the goals set forth for them. They also reported that working in BELL After School helped them develop their professional skills. Parents reported that the model boosted their involvement in their child’s education – an outcome that is proven to have significant long-term impact on student achievement in school and beyond.
Teacher Observations of Scholar Achievement 88%
Scholars are reading better.
84%
Scholars are better at math.
Scholars are better at asking for help.
89%
Scholars are better at expressing their ideas.
89%
Scholars are better at taking part in group activities.
89%
Scholars have more confidence in themselves.
89% 78%
Scholars are displaying better self-control or self-management skills.
83%
Scholars are better at relating to their peers.
Parent Observations of Scholars Achievement 92%
Scholars are reading better.
89%
Scholars are solving math problems better.
90%
Scholars are more excited to learn.
86%
Scholars are better at expressing him/herself.
91%
Scholars are more confident in their abilities.
Scholars have a positive attitude about school.
91%
Scholars have a positive attitude about learning.
92% 88%
Scholars are cooperating more with children.
93%
Scholars are willing to participate in activities
50%
75%
100%
Page 3 of 5
Teacher Observations of Scholar Engagement 97%
Scholars were engaged in the literacy curriculum. BELL’s literacy resources effectively advanced scholars toward the accomplishment of their goals.
96% 95%
Scholars were engaged in the mathematics curriculum. BELL’s math resources effectively advanced scholars toward the accomplishment of their goals.
92% 93%
Community Time activities were structured and interactive. All scholars in my cluster participated in parent-teacher conferences.
74% 86%
All scholars in my cluster developed a scholar portfolio.
78%
My program celebrated College Pride Day
Teacher Engagement & Satisfaction 97%
Staff displayed a positive attitude.
95%
Staff modeled BELL's mission.
93%
My expectations about working for BELL were met or exceeded. The communication, information, and support I received during the recruitment and hiring process was of a high quality.
96%
Working in the BELL After School Program has helped me develop my professional skills.
97%
I would recommend this program to parents looking for an educational afterschool program for their child(ren).
97%
I would recommend teaching in this program to my peers (e.g., other college students, teachers, etc.)
95%
I find working at BELL personally rewarding.
95%
If my circumstances stayed the same, I would want to work in a BELL program next year.
96%
Parent Engagement & Satisfaction 93%
I am more involved in my child's education
96%
My scholar enjoyed BELL
95%
I recommend BELL to other parents.
93%
I would like to enroll my child again.
92%
BELL met or exceeded my expectations for academic impact.
92%
BELL met or exceeded my expectations for social development.
96%
BELL met or exceeded my expecations for my scholars' fun and enjoyment.
92%
I am highly satisfied with my BELL experience.
50%
75%
100%
Page 4 of 5
EVALUATION ADVISORY BOARD BELL’s evaluation activities are guided, advised, and endorsed by an external, interdisciplinary group, BELL's national Evaluation Advisory Board. The responsibility of the Evaluation Advisory Board is to oversee the application of principles and standards established in the evaluation and assessment fields. Dr. Duncan Chaplin, Senior Researcher Mathematica Policy Research, Inc. Dr. Leslie Goodyear, Senior Research Associate Educational Development Center, Inc. Ms. Priscilla Little, Research & Evaluation Consultant Dr. Beth Miller, Director of Research & Evaluation Nellie Mae Education Foundation
experienceBELL.org