BELL Summer 2012 New York Evaluation Report

Summer Evaluation Report NEW YORK CITY, NY

2012

2,295 Scholars Served

K-8 2012 Report Card NEW YORK CITY, NY

Grades Served

162.5 - 250 Total Hours of Summer Learning

90% Average Daily Attendance

27th à 35th Percentile Rank Increase, Reading

25th à 32nd Percentile Rank Increase, Math

3.9 MONTHS Grade-Equivalent Gain, Reading

4.3 MONTHS Grade-Equivalent Gain, Math

81% Teachers expressing high satisfaction with their participation in BELL Summer

85% Parents reporting that they are more engaged with their child’s education because of BELL

93% experienceBELL.org

Parents expressing high satisfaction with program quality and impact.

2012 Figure A:

Percentile Rank Scores 50%

Summer

40%

+8%

Evaluation Report

+7%

30% 20% 10%

NEW YORK CITY, NY Average % Rank

BELL Scholars Reading

Math

Start: 27%

Start: 25%

Finish: 35%

Finish: 32%

0%

ABOUT THIS REPORT

PERCENTILE RANK SCORES (Figure A)

BELL (Building Educated Leaders for Life) exists to transform the academic achievements, self-confidence, and life trajectories of children living in under-resourced, urban communities. The nonprofit organization partners with schools to expand learning time, and serves students in grades K-8. The BELL Summer program operated for 6.5 - 10 hours per day, 5 days per week, for 5 - 6 weeks: a total of 162.5 – 250 hours of summer learning.

In BELL Summer, scholars increased their percentile rank scores. Percentile rank scores range from a low of 1 to a high of 99, with 50 representing the middle score and denoting average performance. They provide a relative measure that compares BELL scholar performance to national norms. When enrolling in BELL Summer, scholars were underperforming: they entered at the 27th percentile in reading and the 25th in math. In BELL Summer, scholars increased their percentile rank scores to the 35th and 32nd percentiles in reading and math, significantly reducing their deviation from the norm and narrowing the achievement gap.

In BELL Summer, scholars engaged in literacy and math instruction and hands-on enrichment courses. They learned in a small-group environment, with instruction led by certified teachers and supported by teachers assistants.

Scholars participated in service projects and field trips.

The program also engaged parents and strengthened the connection between the school and home.

In 2012, 2,295 scholars in grades K-8 participated in BELL Summer at 14 New York City partner schools, including 2 Summer Quest sites in partnership with the NYC Department of Education. Approximately 75% of participants were performing below grade level at the start of the program, according to standardized diagnostic reading and math tests. BELL measured program outcomes against three major goals for scholars: improved academic performance, enhanced self-concept and selfefficacy, and increased social skills. Additionally, evaluation activities sought to assess parent engagement and satisfaction with the program.

GRADE-EQUIVALENT GAINS In BELL Summer, scholars made significant gradeequivalent gains in reading and math and avoided summer learning loss. Grade-equivalent gains are an important indicator of student success in the summer because they provide an absolute measure of growth about how much a student has progressed up the proficiency ladder at a time when most children are not engaged in structured learning activities. In BELL Summer, scholars gained 3.9 months of grade-equivalent skills in literacy and 4.3 in math.

TEACHER & PARENT SURVEY DATA Parent & Teacher Observations of Scholars' Academic Performance 86%

Teachers reporting that BELL improved scholars' reading skills!

87%

Teachers reporting that BELL improved scholars' math skills!

92%

Parents reporting that BELL improved scholars' reading skills!

91%

Parents reporting that BELL improved scholars' math skills!

Parent & Teacher Observations of Self-Efficacy Skills 90%

Teachers reporting that BELL improved scholars' self-confidence.!

88%

Teachers reporting that BELL improved scholars' ability to ask for help!

92%

Teachers reporting that BELL improved scholars ability to express their ideas.!

91%

Parents reporting that BELL improved their child's self-confidence.!

92%

Parents reporting that BELL improved their child's attitude toward learning. !

Parents reporting that BELL improved their child's attitude toward school.!

90%

Parents reporting that BELL improved their child's ability to express their ideas.!

91%

Parent & Teacher Observations of Community Skills 91%

Teachers reporting that scholars were better at group activities.!

84%

Teachers reporting that scholars were better with peer relations.!

80%

Teachers reporting that scholars were better at self-control.!

92%

Parents reporting that scholars are better at group activities.!

90%

Parents reporting that scholars cooperate more with other children.!

Parent Engagement & Satisfaction 85%

Parents reporting that they felt more engaged in their child's education.!

98%

Parents reporting that their child enjoyed BELL.!

93%

Parents expressing satisfaction with BELL's services.!

Parents reporting that they would enroll their child in BELL again. !

97%

Parents reporting that they would recommend BELL to other parents.!

97%

0%

25%

50%

75%

100%

EVALUATION ADVISORY BOARD BELL is nationally recognized for its rigorous approach to program evaluation. Evaluation activities are guided, advised, and endorsed by an external, interdisciplinary group, BELL's national Evaluation Advisory Board.

Dr. Duncan Chaplin, Senior Researcher Mathematica Policy Research, Inc. Dr. Leslie Goodyear, Senior Research Associate Educational Development Center, Inc. Dr. Denise Huang, Project Director & Senior Researcher National Center for Research on Evaluation, Standards, and Student Testing (CRESST) Ms. Priscilla Little, Research & Evaluation Consultant Dr. Beth Miller, Director of Research & Evaluation Nellie Mae Education Foundation

METHODOLOGY TEST DATA: The following standardized, validated instruments were administered pre- and post-program and used to measure program outcomes: the fourth edition of the Stanford Diagnostic Reading Test (SDRT-IV) (Harcourt Educational Measurement, 1996), a diagnostic assessment of grade-level literacy skills, was administered to all scholars; and the fourth edition of the Stanford Diagnostic Mathematics Test (SDMT-IV) (Harcourt Educational Measurement, 1996), a diagnostic assessment of grade-level mathematics skills, was administered to scholars in grades 2 and higher.

SELF-EFFICACY DATA: BELL uses the Youth Outcome Toolkit, a validated, youth-respondent instrument, to measure changes in scholars’ academic behaviors, sense of self, positive core values, and life skills. These attitudes, skills, and behaviors are highly predictive of future school persistence and success. In addition, at the beginning and end of BELL programs, scholars in grades 3 and higher complete a brief survey designed by BELL to assess their academic aspirations.

TEACHER SURVEYS: BELL developed a teacher and tutor survey instrument to assess program implementation and efficacy from the perspective of instructional staff.

Teachers and tutors complete the anonymous survey at the end of the program, which includes questions regarding scholars’ progress, training, curricula, BELL staff and service, and parent engagement.

“

BELL Summer helped her – the smaller groups are better for her. I think BELL gave her an advantage going back to school.

Before, in math and science in particular, she had trouble focusing in school, and her principal recommended she participate. In the program, she got extra help in math – her weakest subject – and she excelled here in BELL. She likes the program.

-­‐ BELL Parent

“

“I’m learning all the stuff I had trouble with in school – square roots, scientific notation.

I had to work on my math to be ready for high school”

-­‐ BELL Scholar

“

I believe the BELL experience will boost scholars’ confidence when they return to school after the summer break.

-­‐ BELL Teacher

PARENT SURVEYS: BELL developed a parent survey instrument to assess parent observations of scholar improvement, program quality, and parent satisfaction.

Parents complete the anonymous survey on-site during the final week of the program.

experienceBELL.org