Teach For America Study

teachforamerica

Case Study

of

Effectiveness

Analysis and Compilation by

T h e Sc h u lt z C e n t e r for Teaching

and

Leadership

Jacksonville, Florida 路 July 2011

teach for impact for leadership teach for change Teach for all Of Us Teach for the future teach for transformation teachforamerica teach for family for opportunity teach for impact for leadership teach for change Teach for all Of Us Teach for the

Who are we?

Tennessee state report finds that Teach For America corps members outperform their peers

Teach for America (TFA) is a nonprofit organization committed to placing top recent college graduates into local teaching forces of beginning teachers currently serving in urban and rural school districts across the country. As part of their commitment to work as TFA corps members, these teachers agree to teach for two years in public schools, and in the process it is hoped that these individuals evolve into strong instructional leaders who seek to expand educational opportunities for all students. Teach for America – Jacksonville currently has corps members placed in 23 Duval County schools at the elementary, middle, and high school levels.

In March 2011, Teach for America – Jacksonville requested that an external analysis be done of current performance by corps members serving in Duval County schools. The Schultz Center for Teaching and Leadership was asked to provide this third party review of available data to inform TFA work in several ways:

• To provide TFA teachers, staff members, and stakeholder groups with relevant information about the progress made to date by corps members in comparison with other novice teachers in Duval County schools

• To make information available about various sources of student data and their relative utility in offering teachers and TFA staff members with reliable data for decision making about performance of students; and

• To make recommendations for an ongoing plan for data collection and analysis that will allow TFA and the Duval County School Board with timely and relevant information about the progress of students and the effectiveness of Teach For America corps members.

Because Teach For America corps members are at work in elementary, middle, and high schools, student achievement data from all levels was reviewed with the goal of answering the following question:

How does the student performance of students in TFA classrooms compare with that of similar experience level teachers in non-TFA classrooms?

What did we do?

Teach For America is one of only ten new organizations on FORTUNE’s list of 100 Best Companies to Work For

While student achievement in all areas is important, the most commonly cited indicators of student proficiency discussed widely in Florida are those found on the Florida Comprehensive Assessment Tests (FCAT) in grades 3-10 in reading and mathematics. These statemandated accountability measures were used for most of the analyses conducted for this report. In addition, we included a recently developed assessment of reading at the primary grade levels called FAIR (Florida Assessments for Instruction in Reading) to allow us to include data for TFA corps members assigned to primary grade levels. In all, we were able to include 101 corps members, among whom about one half were serving in grade levels and subject area assignments which permitted the inclusion of student performance data allowing for the analysis of student gains.

The question at hand was whether the student gains of TFA corps members compared favorably with that of all novice (beginning) teachers, and in particular, the group of beginning teachers placed in similar schools. After clarifying questions of TFA staff members and discussing reasonable comparison groups that might be used to answer questions about efficacy, we arrived at several primary groups which would serve as the source of contrasts made in the analysis of student achievement gains:

• Students in TFA Classrooms

• Students in Non-TFA Classrooms (in the same schools)

• Students in Novice (Non-TFA) Classrooms

• Students in Title I Schools

• Students in Turnaround Schools

• Students in Duval County (Sample) Student achievement gains available for students in Grade 2 (FAIR), grades 4-8 (FCAT) in reading, and grades 3-10 (FCAT) in mathematics were used to carry out statistical analyses on differences that were found among the gains made by grade level and by named comparison groups of teachers. Because state percentile norms do not exist for the FCAT, all Duval County scores were ranked to create “local norms” and percentiles which could then be used

Harvard study finds that Teach For America is producing a significant number of education entrepreneurs

to compare the relative gains made by teachers in various groups. Local percentile ranks were assigned to all students in Grade 2 for whom FAIR Probability of Reading Success (PRS) scores were available (n = 8700) and for all students at grades 4 - 10 in reading and mathematics for whom previous year and current year data created viable “gain scores” (approximately 60,000 scores in reading and 76,000 in mathematics). Local percentile ranks were calculated by standardizing gain scores and ranking them within grade levels. Allowing for the uniqueness of grade level patterns of test content and difficulty, the rank ordering of gains within grade level would give us a good representation of what “typical” growth looks like currently in our schools.

Based on the percentile ranking process in reading and mathematics, gains produced for all students were then sorted into four quartile groupings:

• Quartile 1: 1st through 25th local percentiles

• Quartile 2: 26th through 50th local percentiles

• Quartile 3: 51st through 75th local percentiles

• Quartile 4: 76th through 99th local percentiles Using the placement of student gains within local percentile ranks and quartiles, we can determine whether students made growth placing them in the 1st through the 4th quartile of gains seen in Duval County. This process allowed us to use commonly available measures (FAIR and FCAT) in reading and mathematics to qualify the relative size of gains made by students of teachers within various comparison groups.

The average (or typical) performance in Duval County was determined by local percentiles calculated with all students, so variations would occur randomly across all. If gains differ significantly by subgroups of teachers identified by other factors, then, this may provide evidence that the effectiveness of these groups may similarly differ. Based on these sources of information, we were able to make distinctions in two ways:

1. Average gains of students made in classrooms of TFA teachers with other comparison group teachers.

2. Proportions of student gains falling in the top (3rd and 4th) local quartiles of performance (i.e., above average performance).

What did we learn?

Following are results tables reflecting comparisons of FAIR PRS score and FCAT gains made by students in classrooms of comparison groups in contrast with student results in Teach for America classrooms. Top Quartile (3rd and 4th Quartile) Gains were based on local percentile ranks calculated on Duval County total group scores in reading and mathematics. Cells in these tables are shaded to reflect findings using tests of statistical significance: NBC’s TODAY show recently featured Wendy Kopp’s Washington Post bestseller, A Chance to Make History, and TFA alumnus Brett Kimmel

Green = statistically significant results in a positive direction (+) Yellow = no significant difference or comparable results (≈) Blue = Duval County Sample (random sample of all students) Gray = Indicates the teacher comparison group used in statistical analysis

“ I am indebted to my father for living, but to my teacher for living well.”

Al e x a nd e r t h e G r e at

What did we learn?

Reading Comparisons by Teacher and School Groups. readi ng gai ns i n loca l 3rd & 4th quarti les

Number/percent of students top quartile gains Novice Novice TA Title 1 TFA T1 Teachers Schools Schools Teachers Teachers

Duval County Sample

Grade 2

57 75.3%

174 55.8%

278 52.6%

201 55.2%

293 54.7%

1925 50.0%

Grade 4

103 54.2%

157 32.5%

300 41.1%

250 40.6%

406 42.4%

1077 50.0%

Grade 5

40 40.8%

159 40.0%

301 47.0%

212 41.3%

347 44.1%

1012 50.0%

Grade 6

124 44.8%

158 30.3%

263 38.1%

95 22.0%

322 40.6%

700 50.0%

Grade 7

81 52.3%

116 33.1%

242 41.1%

75 26.1%

292 45.4%

612 50.0%

Grade 8

99 44.4%

141 31.2%

220 38.6%

106 28.3%

324 43.1%

651 50.0%

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

5,200 new Teach For America teachers recently joined the national effort to expand educational opportunity

Reading Summary Statements: Percent of students in TFA classrooms making gains which were above average locally (Q3 & Q4): • Were significantly higher at Grade 2 (75.3%) than other comparison groups and the district as a whole (+).

• Were also significantly higher than comparison groups at grade 4, 6, 7, and 8 (+).

• Were not significantly different at Grade 5.

• Improvement most needed at grades 5, 6, and 8.

Math Comparisons by Teacher and School Groups. math gai ns i n loca l 3rd & 4th quarti les

Number/percent of students top quartile gains Novice Novice TA Title 1 TFA T1 Teachers Schools Schools Teachers Teachers

Duval County Sample

Grade 4

57 52.3%

174 56.3%

278 52.2%

201 58.0%

293 56.2%

589 50.0%

Grade 5

57 45.2%

109 44.7%

181 48.2%

159 45.3%

246 47.9%

552 50.0%

Grade 6

91 66.9%

147 57.6%

219 57.3%

178 61.2%

332 55.2%

537 50.0%

Grade 7

42 70.0%

165 59.9%

249 56.1%

140 59.6%

298 56.2%

514 50.0%

Grade 8

92 64.4%

164 55.8%

202 54.2%

170 54.8%

326 54.7%

553 50.0%

Grade 9

39 69.6%

126 60.9%

212 53.0%

279 56.0%

258 56.3%

445 50.0%

Grade 10

74 56.4%

135 54.8%

190 49.5%

285 52.1%

266 53.0%

446 50.0%

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Count & % w/in grade

Math Summary Statements: Percent of students in TFA classrooms making gains which were above average locally (Q3 & Q4): • Were significantly higher at grades 6, 7, 8, and 9 than other comparison groups (+).

• Were not significantly different at grades 4, 5, and 10.

• Improvement most needed at Grade 5.

In conclusion.

Teach For America’s 20th Anniversary Summit draws nearly 11,000 committed to closing the achievement gap

The goal of this study was to determine the impact of TFA teachers on student growth in reading and mathematics as compared with that of similar experience teachers in nonTFA classrooms. The answer must also be put in context as it relates to “customer satisfaction” and district “staying power”.

Reviewing student gains, it’s clear that TFA corps members are having a positive impact on student achievement in Duval County. Among 13 individual statistical comparisons made between students in TFA classes with those of other Novice Title I classrooms, 9 were found to be statistically significant in a positive direction, while 4 were found to be comparable. In only one instance (4th grade math) did the proportion of students making above average gains (locally) fall below that of the most similar comparison group. These findings mirror national studies which demonstrated that student achievement under corps members was great or greater than that of other new teacher groups.

Principals in schools where TFA corps members were assigned provided an important source of information about the quality of corps members. The goal of ensuring that effective teachers are placed in every classroom is critical. Duval County principals expressed high levels of satisfaction on a survey completed in 2009:

• 92% said that TFA teachers ‘have made a positive difference in the school environment’

• 75% rated the preparation given to TFA teachers as ‘above average’

• 92% would recommend hiring of TFA teachers

There is a concern about the commitment and staying power of corps members in Duval County. By the close of school year 2009-10, 109 corps members had been placed in schools. Among those, 89 completed two-year commitments. Within that group, 26 continued teaching in Duval County, and 63 did not. Of that 63, 45 stayed in the field of education. In total, 71 of the 89 (80%) who completed their two-year commitment are still involved in education. Efforts are ongoing to enhance the TFA corps members commitment to Jacksonville.

teach for impact for leadership teach for change Teach for all Of Us Teach for the future teach for transformation teachforamerica teach for family for opportunity teach for impact for leadership teach for change Teach for all Of Us Teach for the

4019 Boulevard Center Drive Jacksonville, Florida 32207 904 348 7239 www.schultzcenter.org

ABOUT THE AUTHOR: Katherine Divine, Ph.D. is an experienced assessment and evaluation specialist currently serving in a shared position with Duval County Public Schools as Executive Director for Research Design and Evaluation at the Schultz Center. Her past experience includes tenure as a Research Associate at the University of North Florida (Florida Institute of Education); Director of Research, Evaluation and Accountability for the District School Board of Pasco County; and Director of Research, Communications, and Student Services for Portsmouth Public Schools in Virginia. For more information about how programs are evaluated for effectiveness, or for other reports and publications, visit www.schultzcenter.org.