BELL Summer 2013 Evaluation Report

SUMMER

2013

Report Card

experienceBELL.org

SUMMER Report Card 2013 BELL (Building Educated Leaders for Life) exists to transform the academic achievements, self-confidence, and life trajectories of children living in under-resourced, urban communities. BELL fulfills its mission by partnering with schools to expand learning time in the summer and during the school year.

BELL’s summer learning program blends rigorous, small-group academic instruction in reading and math with hands-on enrichment courses in science, technology, creative arts, and fitness and health. The program brings the classroom into the community, and the community into the classroom, by incorporating field trips, service projects, and parent engagement.

BELL Summer is designed for students who are performing below grade level and who face the greatest risk of academic under-performance. At the start of the program, data from computer-adapative assessments showed that scholars were performing below grade level. BELL and its school and community partners set the following goals: ¤ Scholars will avoid summer learning loss. ¤ Scholars entering BELL Summer below grade level will gain one or more months of grade-equivalent skills. ¤ At least 75% of all scholars will demonstrate improved selfconfidence and social skills. ¤ At least 75% of parents will report that they became more engaged in their child’s education through BELL Summer.

This report summarizes outcomes achieved in the 2013 BELL Summer program. The program served 7,942 scholars at 63 school and community sites in San Jose & San Rafael, CA; Hartford, CT; Orlando, FL; Chicago, IL; Baltimore, MD; Boston & Salem, MA; Charlotte & Winston-Salem, NC; Newark, NJ; and New York City, NY. According to assessments administered at the start of the BELL Summer program, approximately 70% of participants entered the program performing below grade level in reading and 54% were below grade level in math.

7,942 Scholars

K-8 Grades Served

85% Average Daily Attendance

+1.4 Months, Reading +1.7 Months, Math Average Grade-Equivalent Gains All BELL Scholars

89% Parents reporting that BELL Summer increased their child’s self-confidence

91% Parents reporting they are more engaged in their child’s education because of BELL

94% Parents expressing satisfaction with BELL’s services Page 1 of 5

METHODOLOGY TEST DATA: BELL measured academic outcomes using two assessment tools. At sites in CA, MD, MA, and NC (approximately 70% of all BELL Summer sites), teachers administered STAR Enterprise assessments. These computer-adaptive assessments are built for Common Core standards. At the start of BELL programs, scholars took a 40-question multiple choice test on a computer, laptop, or tablet. The questions adjust to scholar responses, providing BELL’s team of educators with deep and accurate insight into scholars’ academic needs.

STAR’s powerful reporting tool boosted teachers’ ability to differentiate instruction during the program. At the end of BELL Summer, STAR assessments quantified scholars’ improvement in reading and math.

At sites in NY & NJ, as well at sites operated in partnership with YMCA associations in CT, FL, and IL, BELL measured academic impact using the Stanford Diagnostic Reading Test (SDRT-IV) (Harcourt Educational Measurement), a paper-based diagnostic assessment of grade-level literacy skills for grades K-8; and Stanford Diagnostic Mathematics Test (SDMT-IV) (Harcourt Educational Measurement), a diagnostic assessment of grade-level mathematics skills for grades 2-8. These paper-based tests were also administered at the start and end of the program.

OUTCOMES GRADE-EQUIVALENT GAINS: In BELL Summer, scholars gained grade-equivalent reading and math skills.

Grade-equivalent gains are an important indicator of student success because they provide an absolute measure of how much a student has progressed up the proficiency ladder at a time when most children are not engaged in structured learning activities.

According to data from STAR Enterprise assessments, scholars in BELL Summer achieved average gains of 1.4 months of reading skills and 1.7 months of math skills.

Gains were even greater for scholars who started the summer below grade level: those scholars gained 2.4 months of grade-equivalent skills in reading and 3.9 months of math skills. Scholars in or entering middle school generally started the summer further behind grade level than their younger peers, and achieved greater gains.

Data from Stanford Diagnostic Tests also demonstrate that BELL scholars gained academic skills. According to those paper-based tests, scholars gained 5.1 months of reading skills and 5.8 months of math skills.

In sum, BELL Summer helped scholars become more prepared to succeed in the new school year.

SELF-EFFICACY DATA: BELL uses the Youth Outcome Toolkit, a validated, youth-respondent instrument, to measure changes in scholars’ academic behaviors, sense of self, positive core values, and life skills, because such indicators are highly predictive of future school persistence and success.

TEACHER SURVEYS: BELL developed a teacher and tutor survey instrument to assess program implementation and efficacy from the perspective of instructional staff.

Teachers and tutors complete the anonymous survey at the end of the program, which includes questions regarding scholars’ progress, training, curricula, BELL staff and service, and parent engagement.

PARENT SURVEYS: BELL developed a parent survey instrument to assess parent observations of scholar improvement, program quality, and parent satisfaction.

Parents complete the anonymous survey on-site during the final week of the program.

SALESFORCE.COM: BELL developed a customized scholar management system based on the Salesforce.com platform to track and report on scholar enrollment, attendance, and a wide range of quality metrics.

GRADE-EQUIVALENT GAINS IN BELL SUMMER, IN MONTHS All Grades Scholars K-4

Grades 5-8

Reading Data source: STAR Enterprise Assessments All Scholars

1.4

0.7

1.2

Underperforming Scholars

2.4

1.6

2.0

5.1

N/A

N/A

Data source: SDRT-IV All Scholars

Math Data source: STAR Enterprise Assessments All Scholars

1.7

1.6

2.0

Underperforming Scholars

3.9

2.2

4.0

5.8

N/A

N/A

Data source: SDMT-IV All Scholars

Page 2 of 5

AVERAGE SUMMER LEARNING GAINS OF UNDERPERFORMING SCHOLARS VS SUMMER LEARNING LOSS

Gain / Loss of Grade-Equivalent Skills in Months

4

+3.9 Months, Math Skills

3

+2.4 Months, Reading Skills

2

Average Grade-Equivalent Gains Scholars starting BELL Summer below grade level

1 0 -1

-2 Months

-2 Summer Start

Average Summer Learning Loss Disadvantaged Students without Summer Learning*

Summer End

The growth in academic skills measured by computeradaptive assessments and paper-based tests is significant given that most disadvantaged children lack summer learning opportunities and, as a result, lose two months of skills. Summer learning loss accumulates over time, and has been shown to account for up to 2/3 of the academic achievement gap between disadvantaged students and their higher-income peers by the time students reach the 8th grade.

At BELL Summer, you get more attention and help than you would usually get in school. I wasn’t that good in reading in 6th grade. But when my BELL Summer teacher went over it and explained it to me, I got the hang of it!

-

Lasayne 6th grade BELL scholar

* Sources McCombs et al. (2011). Making Summer Count: How Summer Programs Can Boost Children’s Learning. Rand Education & The Wallace Foundation.

Cooper, Harris (2003). Summer Learning Loss: The Problem & Some Soluations. ERIC Clearinghouse on Elementary & Early Childhood Education.

Page 3 of 5

PARENT & TEACHER SURVEY DATA Parent & Teacher Observations of Scholars' Academic Performance 81%

Teachers reporting that scholars improved their reading skills Teachers reporting that scholars improved their math skills

86%

Parents reporting that scholars improved their' reading skills

83%

Parents reporting that scholars improved their math skills

83%

Parent & Teacher Observations of Self-Efficacy Skills 92%

Teachers reporting that scholars increased their self-confidence Teachers reporting that scholars improved their ability to ask for help

88%

Teachers reporting that scholars improved their ability to express their ideas

90%

Parents reporting that scholars increased their self-confidence

89%

Parents reporting that scholars improved their attitude toward school

89%

Parents reporting that scholars improved their attitude toward learning

91%

Parent & Teacher Observations of Community Skills Teachers reporting that scholars improved their peer relations skills

83%

Teachers reporting that scholars improved their self-control skills

82% 91%

Parents reporting that scholars are better at group activities Parents reporting that scholars cooperate more with other children

84% 0%

25%

50%

75%

100%

Page 4 of 5

Parent Engagement & Satisfaction Parents reporting that they felt more engaged in their child's education"

91% 96%

Parents reporting that their child enjoyed BELL"

94%

Parents expressing satisfaction with BELL's services"

91%

Parents reporting that they would enroll their child in BELL again" Parents reporting that they would recommend BELL to other parents"

94%

Teacher Engagement & Satisfaction Teachers reporting high satisfaction with their experience in BELL Summer."

88%

Teachers reporting that BELL's team modeled BELL's mission."

94%

Teachers reporting that BELL's team displayed a positive attitude."

94%

Teachers reporting that BELL Summer developed their professional skills."

94%

Teachers reporting that they found working for BELL personally rewarding."

97%

Teachers reporting that they would recommend BELL to parents."

97%

0%

25%

50%

75%

100%

EVALUATION ADVISORY BOARD BELL’s evaluation activities are guided, advised, and endorsed by an external, interdisciplinary group, BELL's national Evaluation Advisory Board. The responsibility of the Evaluation Advisory Board is to oversee the application of principles and standards established in the evaluation and assessment fields.

Dr. Duncan Chaplin, Senior Researcher Mathematica Policy Research, Inc. Dr. Leslie Goodyear, Senior Research Associate Educational Development Center, Inc. Ms. Priscilla Little, Research & Evaluation Consultant Dr. Beth Miller, Director of Research & Evaluation Nellie Mae Education Foundation

Scan or click to see the 2013 BELL Summer Photo Album.

experienceBELL.org