Page 1

NORTH CAROLINA OPPORTUNITY SCHOLARSHIPS Countering a Flawed Duke Report and Setting the Record Straight

pefnc.org


EXECUTIVE SUMMARY

Since their establishment by North Carolina lawmakers in 2013, Opportunity Scholarships have elicited intense, wide-ranging reactions from the state’s K-12 stakeholders. Supporters have greeted these statefunded private school scholarships with great enthusiasm, applauding the educational freedom and choice they offer to low-income families. Yet opponents have been equally fierce in their criticism, claiming these scholarships, capped at $4,200, represent an unaccountable reform with limited potential for promoting student success. Debate over the academic merits of private school scholarships, also known as vouchers, has become particularly heated this spring. A report released in March by the Children’s Law Clinic at Duke Law School claims the Opportunity Scholarship Program is “unlikely” to promote better academic outcomes for students. Decrying Program design, Duke researchers write that “an ‘apples-to-apples’ comparison between public school and voucher students” is not possible since voucher students do not take state tests.1 Moreover, references to national school choice research, in the Duke report and in public commentary, increasingly dismiss the weight of evidence favoring vouchers, focusing instead on several newer studies that show negative findings. We believe it’s time to set the record straight—on North Carolina’s program, and on national research. We’ve written this paper to do just that, focusing on three key areas: • First, we counter the flawed academic analysis offered to the public by the Duke report. We also present the facts to date, which offer no evidence that the Program is unlikely to produce better academic outcomes for students long-term. • Second, we correct misinformation about school choice research nationwide. We review representative research on parental school choice programs, showing that participation is generally linked with positive academic outcomes for students, especially after several years of participation. Benefits are most evident for African American students, and in the areas of educational attainment and reading achievement. • Third, and finally, we identify the framework for the kind of evaluation that provides an “apples-to-apples” comparison of scholarship and public school students in North Carolina. In fact, a newly announced academic impact analysis, already underway, will soon give state taxpayers and K-12 stakeholders an early snapshot of scholarship students’ performance. While we understand this study will not provide the definitive word on student outcomes linked with scholarship participation, we hope it establishes a foundation for future, more permanent evaluations of the Opportunity Scholarship Program.


I’m a single mom of two and cannot afford to enroll my children in a private school without the Opportunity Scholarship Program. Though I am economically challenged and the primary caregiver to my children, I’m not a statistic. I, too, care deeply about my children’s future in this world. This Program has made all the difference in the world for our family. Kaylan McCaskill, Gastonia Opportunity Scholarship Parent

ACADEMIC OUTCOMES IN NORTH CAROLINA AND THE CASE FOR ACCOUNTABILITY

Publicly-funded programs created to expand educational opportunity for students should, without a doubt, be accountable to the public. Lawmakers establishing North Carolina’s Opportunity Scholarship Program understood this, and codified provisions in statute to require schools enrolling more than 25 scholarship recipients to report aggregate standardized test scores to the North Carolina State Education Assistance Authority (NCSEAA). This information is a public record. Statute allows school administrators to choose which test to administer, but specifies that it must be “a nationally standardized test or other nationally standardized equivalent measurement.”2 To date, schools have reported scores for two academic years: 2014-15 and 2015-16; 2016-17 test scores are due in July 2017. The first year (2014-15), just six schools met the threshold required for data reporting. Of these six schools, two reported that half or more of their scholarship students were at or above the 50th National Percentile Rank (NPR) in reading, language, and mathematics. A third school reported verbal and nonverbal reasoning scores, showing that half or more of students were at or above the 50th NPR in those areas. Schools reporting aggregate data used two different standardized assessments: the Iowa Tests and the TerraNova Achievement Test.3 The next year, in 2015-16, 34 schools enrolled more than 25 Opportunity Scholarship students. Of these 34 schools, 13 schools indicated that half or more of their scholarship students were at or above the 50th NPR in reading and mathematics. Eleven of 34 schools reported that half or more of Opportunity Scholarship students were at or above the 50th NPR in reading, language, and mathematics. Schools reporting aggregate data administered at least five different tests: the Iowa Assessments/Tests, the TerraNova Achievement Test, the Stanford Achievement Test, the California Achievement Test, and a test from the Northwest Evaluation Association.4 What do these data tell us? Not much. We still do not know how Opportunity Scholarship students perform relative to public school students from similar backgrounds. We do not know how

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

2


scholarship students compare to other scholarship students at various private schools, given the different tests students take. While data clearly show there is room for improvement in performance, they provide no definitive evidence that the Opportunity Scholarship Program is not helping students, that the Program is poorly designed to produce positive educational outcomes, or that participants are not keeping pace with public school students from similar economic backgrounds.

IN NORTH CAROLINA, A FLAWED ACADEMIC ANALYSIS

Nonetheless, Duke researchers attempt to draw conclusions about scholarship students’ performance in their report, writing in the executive summary, “Based on limited and early data, more than half the students using vouchers are performing below average on nationallystandardized reading, language, and math tests. In contrast, similar public school students in NC are scoring above the national average.”5 This second sentence was later removed from the report, as was the inequitable direct comparison researchers made between scholarship students’ and public school students’ performance. This comparison was structured using different tests and different national comparison groups. We commend Duke researchers for correcting some of their misstatements and for removing this direct comparison, but believe there is merit in fully setting the record straight in this paper.

National Average versus National Percentile Rank

Even the corrected Duke report provides no explanation for how researchers came up with a “national average” for scholarship students; private schools reported scores for scholarship students on different tests based on a national percentile rank (NPR), not a national average. The corrected Duke report indicates of scholarship students that a majority scored below the “national average” for both 2014-15 and 2015-16.6 It’s unclear, however, how this “average” percentile rank was determined; to do this correctly, researchers would need to convert each percentile rank to a different metric, such as a normal curve equivalent, average that number, and then convert the score back to a percentile rank for reporting purposes. There is no explanation in the report for whether such calculations were carried out, and if so, how. CTB McGraw-Hill, the publisher of TerraNova, one of the tests used by North Carolina private schools, offers the following caution:

The National Percentile (NP) is the best score for describing test results to persons outside the test and measurement community … The 50th NP is the middle score—also called the Median—above and below which half of the students scored … The NP is a non-equalinterval scale, meaning that the distance between two NPs has different meaning at different points in the scale. NPs should not be treated arithmetically; that is, they should not be averaged directly.7

3

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


Comparison Groups

“Comparisons” in both the initial and corrected versions of the Duke report are of little value and provide no meaningful context for making determinations about scholarship students relative to their public school peers from similar backgrounds. Immediately following analysis of scholarship student aggregate data, the corrected Duke report includes a “public school comparison” page, showing that public school students in North Carolina are performing above the national public school average on a different test, the National Assessment of Educational Progress (NAEP). This test is administered to a sample of students, both nationally and at the state level. It’s worth noting that the original uncorrected Duke report, which was reported on by the media, compared free and reduced lunch eligible (FRL) public school students in North Carolina only to other FRL public school students nationally—and then summarized this way, “Although it is not an ‘apples-to-apples’ comparison, the most recent data shows that comparable students who remained in public schools are scoring better than the voucher students on national tests.”8 This comparison was never equitable for a variety of reasons. The two groups of NC students (voucher and FRL public school), and the students they were measured against, represented different groups of students. Scholarship students, who are themselves low-income (eligibility for the scholarship is based on having a family income that does not exceed 133% of the amount to quality for free and reduced lunch), were evaluated against a nationally representative sample of students across the country—from both public and private schools, and not just other low-income students. As mentioned above, North Carolina FRL public school students, on the other hand, were not compared in the Duke report to a nationally representative sample of students taking NAEP. They were compared to other FRL public school students nationwide who took NAEP. Here is how the Iowa Testing Programs of the University of Iowa, test developer for the Iowa Assessments (used by a number of North Carolina private schools reporting aggregate data on scholarship students), characterizes their sample for determining NPR:

One metric, National Percentile Rank (NPR), indicates the status or relative rank of a student’s achievement compared with that of a nationally representative sample of students … The most recent NPRs for the Iowa Assessments are based on the standardization studies conducted in 2010-2011 in which nationally representative samples of public and private school students were assessed in all content areas. As a result, information from the Iowa Assessments allows educators and parents to compare an individual student’s or a local group of students’ performance to the most current estimate of national student performance available.9

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

4


The corrected version of the Duke report, which removed the direct comparison between scholarship and public school students, instead provides— as a representation of public school performance—a “public school comparison” page of two groups of students: 1) public school students in North Carolina measured against public school students nationwide; and 2) public school students in North Carolina who are eligible for the federal school lunch program (FRL students) measured against other public school students nationwide who are eligible for the school lunch program. Text in the corrected report summarizes data this way:

For 2015, the NAEP data shows that, as a group, the North Carolina public school children scored above the national public school average in 2 of 4 categories: 4th grade reading, 4th grade math; exactly at the national average for 8th grade math; and slightly below the national average for 8th grade reading. This held true for North Carolina public school children who are eligible for the federal school lunch program.10

Since researchers point out that both FRL public school students in North Carolina and overall public school students in North Carolina performed better, on balance, than a national comparison group, they seem to imply that public school students in the state are better off academically than scholarship students. Presenting data this way is problematic and somewhat misleading, however, given the presentation of data in the report on scholarship students. Neither version of the Duke report compares FRL public school students in North Carolina directly to a nationally representative sample of all public school students taking NAEP. (NAEP is also administered to students at private schools, but these schools “are not included in state-level results which are solely focused on public schools,” according to NAEP guidelines about sample determinations.11) Here is how NAEP defines some sample guidelines: To ensure that a representative sample of students is assessed, NAEP is given in a sample of schools whose students reflect the varying demographics of a specific jurisdiction, be it the nation, a state, or a district. Within each selected school and grade to be assessed, students are chosen at random to participate in NAEP. Every student has the same chance of being chosen—regardless of race/ethnicity, socioeconomic status, disability, status as an English language learner, or any other factors.12

5

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


What would a direct comparison between FRL public school students in North Carolina and a nationally representative sample of all public school students show? Such a comparison reveals that FRL public school students in North Carolina score below the national public school average on NAEP in both reading and math in both 4th and 8th grades. A “public school comparison” evaluating the average NAEP scores of North Carolina FRL public school students relative to the average NAEP scores of public school students nationwide would have shown the following:13

Public School Comparison, 2015 NAEP Nat. Average (Public Overall)

Nat. Average (FRL)

NC Public School (FRL)

4th grade math

240

229

234

4th grade reading

221

209

215

8th grade math

281

268

268

8th grade reading

264

253

249

Fundamentally, NAEP average scale scores cannot be evaluated or equated as comparable, either directly or indirectly, to NPRs on a range of different nationally standardized tests. It remains unclear why Duke researchers chose to include this information on NAEP public school performance at all, even in their corrected report. The bottom line is this: We do not yet know how most scholarship students in North Carolina are performing on nationally standardized tests, and we do not know how scholarship students compare to other low-income students not using scholarships. A student attending private school on an Opportunity Scholarship could score below the 50th percentile nationally on one of these standardized tests but still outperform another low-income student who did not receive a scholarship. We also don’t know how these Opportunity Scholarship students, whose aggregate scores were reported to the NCSEAA, were testing before entering private school, and whether their test scores since they began using a scholarship evidence growth. For a student performing at the 37th percentile, for example, a jump to the 45th percentile in one year might indicate a notable gain—but still leave this student with a score below the 50th percentile. We can, however, make some assumptions about the value and viability of school choice programs based on student performance in similar programs nationwide. A review of the research helps

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

6


identify what we might expect to see in North Carolina, and also provides compelling justification for school choice reforms.

THE NATIONAL VIEW: WHAT RESEARCH ON SCHOOL CHOICE PROGRAMS SHOWS

Critiques of North Carolina’s Opportunity Scholarship Program cite several recently released studies as evidence that school choice is linked with negative outcomes for students. However, these studies are not representative of the majority of outcomes identified by research thus far. Overarching findings from a large body of research show that school choice programs, on balance, are linked with positive academic outcomes for students, especially after several years of participation. Benefits are sometimes found in the area of improved academic achievement on standardized tests (especially in reading), or with certain groups of students. Earlier findings of smaller privately-funded voucher programs in Washington, DC, Dayton, Ohio, and New York point to test score gains for African American participants in particular.14 Even more strikingly and definitively, positive outcomes for choice participants have been identified in the realm of educational attainment—high school graduation rates, as well as enrollment and persistence in college. What about school choice programs across the world? A 2016 meta-analysis of international school choice research found “overall positive and statistically significant achievement effects of school vouchers that vary by subject (math or reading), location (US v. non-US) and funding type (public or private).”15 A summary of research studies follows. This summary is not exhaustive, but the studies cited below are representative of the type of findings demonstrated by school choice programs nationwide.

Milwaukee Parental Choice Program

The most reliable and comprehensive feedback on school choice programs comes, understandably, from evaluating some of the longest-running programs in the country. The nation’s first contemporary choice program is the Milwaukee Parental Choice Program (MPCP); this program has been in effect since the early 1990s. What have researchers found? In general, MPCP participants have not differed substantially from their public school peers on annual standardized test scores. But the effects of voucher participation on students’ level of educational attainment have been substantial.

7

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


In 2015 testimony before the US Senate Homeland Security Committee, John Witte, who served for years as the official state evaluator of MPCP, said of the program:

The results were quite simple. Compared to the control group, students receiving vouchers beginning in the 2006 cohort graduated from high school and attended four year colleges at between 4 percent and 7 percent higher rates than the comparison group of 2006 public school students. The colleges that they attended also appeared to be of higher status than the ones attended by public school students. And, for the 2006 freshmen cohort the voucher students had a high persistence rate into their sophomore year of college. Although 7 percent may not appear to be a high number to some, it is an extremely steep increase for large urban city school districts that have reached seeming limits in graduation success in the last several decades.16

Another study of MPCP compared voucher students with public school students, again revealing positive outcomes related to educational attainment. Researchers found that 8th and 9th grade students who used a voucher to attend a private school were more likely to graduate from high school. Researchers note that voucher students were also “more likely to enroll in a 4-year postsecondary institution after graduating and, when

applicable, to persist in that 4-year

institution beyond the first year of enrollment.”

17

Washington, DC’s Opportunity Scholarship Program

The long-term research on Washington, DC’s federally-funded Opportunity Scholarship Program is positive, both in terms of test scores and educational attainment. However, a new short-term federal study of one-year impacts revealed negative findings. A summary of overall findings is included here. A 2017 study of Washington, DC’s Opportunity Scholarship Program affirmed that participation in the program had a positive effect on reading outcomes, especially during students’ fourth year of participation. Using a scholarship to attend private school had no effect on math outcomes.18 An earlier evaluation of Washington, DC’s program found that the program had a significant and favorable impact on high school graduation rates and a modest but positive effect on reading test scores. Regarding high school graduation rates, researchers write, “The impact of using a scholarship was an increase of 21 percentage points in the likelihood of graduating. The positive impact of the program on this important student outcome was highly statistically significant.” While researchers did not observe any significant impacts of program participation on mathematics test performance, they found that “the calculated impact of using a scholarship was a reading gain of 4.8 scale score points or 3.4 months of additional learning.”19

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

8


Patrick Wolf of the University of Arkansas, who has conducted much of the research on Washington, DC’s Opportunity Scholarship Program as well as a number of other school choice programs, had this to say in his 2015 testimony before the US House Committee on Oversight and Government Reform:

Students in our pioneering study graduated from high school at significantly higher rates as a result of the OSP. Specifically, the use of an Opportunity Scholarship increased the likelihood of a student graduating by 21 percentage points, from 70 percent to 91 percent. In scientific terms, we are more than 99 percent confident that access to school choice through the OSP was the reason why students in the program graduated at these much higher rates and not some statistical fluke.20

Newer findings on the short-term test outcomes of voucher students have been more negative. A study conducted by the Institute of Education Sciences at the US Department of Education, released recently, found that students who used a scholarship to attend private school had significantly lower math test scores after one year than students who applied for scholarships but stayed in public school. Students coming into the program from low-performing schools did not show statistically significant negative test score declines in math, but those from better performing schools did—in both math and reading. Researchers write:

There were no significant achievement impacts, positive or negative, for students applying from low performing schools (those designated as ‘in need of improvement’ or SINI), to whom the SOAR Act gave priority for scholarships. Negative impacts for both mathematics and reading scores were statistically significant for students who were not attending SINI schools when the students applied for the scholarship and also for students in grades K-5.21

What do these findings mean? It’s hard to say for sure, but longer-term evaluations of this program and others point to positive outcomes as students stay in the program, especially in the area of educational attainment. Clearly, more study is needed to track effects over a longer span of time.

New York’s Scholarship Program

A number of studies have been conducted evaluating school choice in New York. One study by Matthew Chingos and Paul Peterson of a privately funded voucher program (New York School Choice Scholarships Foundation Program) for low-income families found that the offer of a voucher did not impact college enrollment overall, but did have a statistically significant impact on college enrollment and bachelor’s degree attainment for African American students.22

9

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


Children’s Scholarship Fund-Charlotte, North Carolina

An older study by researcher Jay Greene of Charlotte’s privately-funded scholarship program found participation in CSF was linked to higher test scores. Dr. Greene writes, “After one year, the results show that students who used a scholarship to attend a private school scored 5.9 percentile points higher on the math section of the ITBS [Iowa Test of Basic Skills] than comparable students who remained in public schools. Choice students scored 6.5 percentile points higher than their public school counterparts in reading after one year.”23 Another study of Children’s Scholarship Fund-Charlotte recipients who transitioned back into public high school found these students were more likely to pass annual state tests than their peers in Charlotte Mecklenburg Schools who had not used scholarships to attend private school. 24

Florida’s Tax Credit Scholarship Program

The latest evaluation of Florida’s Tax Credit Scholarship Program shows that school choice participants held their own relative to a more advantaged national sample of students. This program, which enables corporations to receive a tax credit for contributions to a scholarship funding organization, has been in effect since 2001 and has been assessed through a series of nine evaluations. The latest evaluation was published in 2016 and was conducted by the Florida State University Learning Systems Institute. Researchers found that “FTC students scored at the 47th national percentile in reading and the 46th national percentile in mathematics. These scores are similar to previous years’ scores.”25 While these students are below the 50th percentile, context is essential in evaluating scores. Researchers note that students receiving scholarships through the Florida Tax Credit (FTC) Scholarship Program were “relatively more disadvantaged and lower-performing prior to entering the FTC program than free- lunch eligible, non-participant students. Moreover, they tend to come from lower performing public schools.” Researchers, affirming that FTC students tend to maintain their “relative position in comparison with all students nationally” in math and reading, also point out a critical caveat: “It is important to note that these national comparisons pertain to all students nationally, and not just students from low-income families.” 26

Ohio’s EdChoice Scholarship Program

One of the most widely cited studies by voucher opponents is the evaluation of Ohio’s EdChoice Scholarship Program, conducted by researchers David Figlio and Krzysztof Karbownik. This study, which was published in 2016 by the Thomas B. Fordham Institute, examined voucher students’ test outcomes between 2003-04 and 2012-13. Researchers found that program participation had a negative effect on state reading and math test scores. The negative impacts were more significant for math than English Language Arts.27 What are the reasons for these findings? It’s unclear. Certainly, some of the private schools participating in the program may not have performed well. But findings may have been impacted

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

10


by other factors such as programmatic design. For example, researchers were limited “to studying students who attended (or had left) public schools that were just above or below the state’s cutoff for ‘low-performing.’” They add:

By definition, this group did not include the very lowest-performing schools in the state. It’s possible that students who used a voucher to leave one of the latter schools might have improved their achievement; we simply cannot know from this study. The negative effects could also be related to different testing environments—higher stakes for public than private schools—or to curricular differences between what is taught in private schools and the content that’s assessed on state tests.”28

Researchers did find evidence that the competition from vouchers improved public schools. They note that the program “improved the achievement of the public school students who were eligible for the voucher but did not use it… In other words, the voucher program has worked as intended when it comes to competitive effects.”29

Louisiana Scholarship Program

The Louisiana Scholarship Program began in 2008 as a pilot program in the city of New Orleans and then expanded to become a statewide program in 2012. Evaluation of the program has linked participation to negative academic outcomes, especially in math. Researchers note the following, “Our results indicate that the use of an LSP scholarship has negatively impacted both ELA [English Language Arts] and math achievement, although only the latter estimates are statistically significant.” Negative effects were smaller as time went on. Researchers write, “While not conclusive, these results suggest the negative impacts of the program may dissipate over time.”30 What’s the reason for Louisiana’s poor outcomes? Lower scores may be attributed to the performance of private schools in the state, but program design may also be a factor: Louisiana’s program places a high regulatory burden on private schools. Researchers offer a number of possible explanations for findings, including the scale of the program as well as program rollout: The program’s first year of statewide implementation did not give schools much time to sign on from the law’s passage to implementation at the beginning of the school year. In fact, in the first year of statewide implementation, less than a third of the state’s private schools chose to participate, as researchers point out.31 Another possible explanation for negative findings is that the voucher students were required to be assessed using state public school tests; their performance may have suffered in part due to their attendance at schools not aligned with public school curricula.

11

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


Regardless, if the Louisiana program follows a similar trajectory to other choice programs, test scores could be expected to improve as time passes. Patrick Wolf, one of the authors of the Louisiana evaluation study, characterized the overall impacts of voucher programs on test scores this way at a recent debate at Harvard University:

Combining all of these experimental results into a meta-analysis, as my graduate students and I have done, indicates that the average effect of private school choice on student test scores is positive in both reading and math but modest in size and on the border of statistical significance. Effects tend to be null or negative the first few years, as students adjust to their new schools, and then turn more clearly positive after three years.32

But, as Dr. Wolf sums up, long-term findings in the educational attainment realm are far more significant and show greater impact:

Four empirical studies have examined the effects of choice on educational attainment. When it comes to education, how far you go matters more in life than how much you know, so educational attainment is the best measure of the effectiveness of a choice program. All four studies conclude that disadvantaged students attain more education if they have access to school choice.33

What’s the bottom line? School choice research to date is more positive, promising, and nuanced than many critics indicate. Student outcomes as measured by educational attainment are particularly encouraging. Evaluations of programs in other states are extremely useful in understanding the design and characteristics of school choice programs, and what may have contributed to students’ performance. But the fundamental question remains: What’s next for North Carolina’s Opportunity Scholarship Program?

THE WAY FORWARD: AN “APPLES-TO-APPLES” ACADEMIC COMPARISON

Lawmakers crafted the statute establishing the Opportunity Scholarship Program with independent, meaningful outside accountability in mind. Statute sets forth the requirement that the NCSEAA report annually to the North Carolina General Assembly’s Joint Legislative Oversight Committee on the “learning gains or losses of students receiving scholarship grants.” Specifically:

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

12


The report shall include learning gains of participating students on a statewide basis and shall compare, to the extent possible, the learning gains or losses of eligible students by nonpublic school to the statewide learning gains or losses of public school students with similar socioeconomic backgrounds … This report shall be conducted by an independent research organization to be selected by the Authority, which may be a public or private entity or university.34

The annual Program reporting requirement for an independent evaluation was scheduled to begin this coming 2017-18 school year. To date, however, no such evaluation has been conducted, largely because funding was not set aside statutorily to ensure it could take place. The public deserves to know if taxpayer dollars are being directed toward a K-12 program that is linked with successful student outcomes. To that end, and recognizing the critical importance of an independent evaluation, public and private K-12 stakeholders in North Carolina have joined together to ensure our state continues to move forward. In addition to PEFNC, this collaboration includes the North Carolina State Education Assistance Authority, the North Carolina Department of Public Instruction, the North Carolina Division of Non-Public Education, private schools, and public school districts. As a result, independent researchers from a public university are now in the process of conducting an early academic impact analysis of North Carolina’s Opportunity Scholarship Program, directly comparing the test performance of a sample of scholarship students with public school students from similar backgrounds. Student performance for both groups will be evaluated using the same nationally standardized achievement test. Full findings will be released in the fall of 2017. While we know this study will not offer definitive feedback on the Opportunity Scholarship Program, we do know this: It will provide an early snapshot of scholarship students’ performance compared to their peers in public schools. We believe future evaluations based on the parameters of this initial study represent the best way forward in measuring test outcomes associated with the Opportunity Scholarship Program. Such a framework provides for a true “apples-to-apples comparison,” using the same yardstick (a nationally normed standardized achievement test) to measure scholarship students against public school students. This approach ensures fairness and impartiality and removes any advantage to one group or another based on test alignment with a particular curriculum. Thus we hope this study lays a foundation for future evaluations of the Opportunity Scholarship Program. Those evaluations will also be informed by the work of a new state task force. During the 2017 Legislative Session, lawmakers in the North Carolina General Assembly passed a budget creating this task force—established to study the evaluation of students receiving Opportunity Scholarships, and to set clear parameters for test-based comparisons with similar public school students. As a standing member of this new task force, PEFNC looks forward to collaborating with private school organizations, public and private school leaders, and independent research organizations, as we work together to affirm the Program’s accountability to the state’s K-12 stakeholders.

13

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight


CONCLUSION

North Carolina’s Opportunity Scholarship Program was created with the purpose of expanding the number of K-12 options for low-income families statewide. In this regard, the Program has enjoyed tremendous success. Nearly half of the state’s 740-plus private schools currently have Opportunity Scholarship students enrolled. The most important stakeholders of all—the families themselves—seem overwhelmingly satisfied. More than 90 percent of recipients choose to renew their scholarships each year. Popularity grows with each successive year. Since the Program’s inception, families have submitted over 33,000 applications.35 As consistent supporters of the Opportunity Scholarship Program specifically, and proponents of school choice generally, we celebrate these developments as evidence that the Program is on track to fulfill its objectives and keep its commitments. But we also support independent evaluations of students’ learning. Nationally, such evaluations have provided stakeholders with essential information on the successes or shortcomings of school choice initiatives, as well as the various program features or designs that may hinder or contribute to student achievement. Numerous school choice programs across the country are showing the value of these K-12 reform efforts, which are helping disadvantaged students graduate from high school and enroll in college with greater frequency. Other school choice programs have not delivered promising results for participating students. They may yet do so. We know from research that academic benefits often take years to accrue. Here in North Carolina, a newly announced academic impact analysis offers a template for equitable evaluation of scholarship students and their public school peers. We hope it serves as a framework for evaluations moving forward. We look forward to the findings of this study and of future evaluations—and to seeing the myriad other ways in which the Opportunity Scholarship Program impacts the educational outcomes of students across our state.

North Carolina Opportunity Scholarships: Countering a Flawed Duke Report and Setting the Record Straight

14


NOTES 1 Jane R. Wettach, “School Vouchers in North Carolina: The First Three Years,” Children’s Law Clinic, Duke Law School, March 2017, 1, 18, https://law.duke.edu/childedlaw/School_ Vouchers_NC.pdf. 2 NC General Statutes Chapter 115C Article 39, accessed April 26, 2017 from http://www.ncga. state.nc.us/EnactedLegislation/Statutes/pdf/ByArticle/Chapter_115C/Article_39.pdf. 3 North Carolina State Education Assistance Authority, “2014-15 Opportunity Scholarships Aggregate Test Report,” 2015. 4 North Carolina State Education Assistance Authority, “2015-16 Opportunity Scholarships Aggregate Test Report,” 2016. 5 Wettach, “School Vouchers,” 1 (original uncorrected report). 6 Wettach, 11 (corrected report). 7 C TB McGraw-Hill, “Beyond the Numbers: A Guide to Interpreting and Using the Results of Standardized Achievement Tests,” http://www.ctb.com/netcaster/ncmedia/12890/Beyond_ the_Numbers_with%20TN3.pdf. 8 Wettach, “School Vouchers,” 12 (original uncorrected report). 9 University of Iowa, Iowa Testing Programs, “Interpreting National Performance Using the Iowa Assessments,” http://itp.education.uiowa.edu/ia/documents/Interpreting_National_ Performance.pdf. 10 Wettach, “School Vouchers,” 12 (corrected report). 11 N  ational Center for Education Statistics, “NAEP State Assessment Sample Design Frequently Asked Questions,” accessed May 18, 2017 from https://nces.ed.gov/nationsreportcard/ about/samplesfaq.aspx. 12 Ibid.


13 National Center for Education Statistics, “NAEP State Comparisons,” accessed April 26, 2017 from https://nces.ed.gov/nationsreportcard/statecomparisons/withinyear.aspx?usrSelections= 0%2cRED%2c4%2c0%2cwithin%2c0%2c0, https://nces.ed.gov/nationsreportcard/ statecomparisons/withinyear.aspx?usrSelections=0%2cMAT%2c4%2c0%2cwithin%2c0%2c0, https://nces.ed.gov/nationsreportcard/statecomparisons/withinyear.aspx?usrSelections=1%2c RED%2c4%2c0%2cwithin%2c0%2c0, https://nces.ed.gov/nationsreportcard/ statecomparisons/withinyear.aspx?usrSelections=1%2cMAT%2c4%2c0%2cwithin%2c0%2c0. 14 Harvard Kennedy School of Government, “African American Students Increase Test Scores in School Voucher Programs, Study Shows,” press release, August 28, 2000, https://www.hks. harvard.edu/news-events/news/press-releases/african-american-students-increase-testscores-in-school-voucher-programs,-study-shows. 15 Danish Shakeel, Kaitlin Anderson, and Patrick Wolf, “The Participant Effects of Private School Vouchers Across the Globe: A Meta-Analytic and Systematic Review,” EDRE Working Paper No. 2016-07, May 10, 2016, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2777633. 16 John Witte, “Evaluating Voucher Programs: the Milwaukee Parental Choice Program,” written testimony, US Senate Homeland Security Committee Hearing on the Milwaukee Parental Choice Program, July 20, 2015, 16. 17 Joshua Cowen, David Fleming, John Witte, Patrick Wolf, and Brian Kisida, “School Vouchers and Student Attainment: Evidence from a State-Mandated Study of Milwaukee’s Parental Choice Program,” The Policy Studies Journal 41, no. 1 (2013): 161, http://onlinelibrary.wiley. com/doi/10.1111/psj.12006/full. 18 Kaitlin Anderson and Patrick Wolf, “Evaluating School Vouchers: Evidence from a Within-Study Comparison,” University of Arkansas Department of Education Reform, April 3, 2017, http:// www.uaedreform.org/downloads/2017/04/evaluating-school-vouchers-evidence-from-awithin-study-comparison.pdf. 19 Patrick Wolf, Brian Kisida, Babette Gutmann, Michael Puma, Nada Eissa, Lou Rizzo, “School Vouchers and Student Outcomes: Experimental Evidence from Washington, DC,” Journal of Policy Analysis and Management 32, no.2 (2013): 260-265, http://onlinelibrary.wiley.com/ doi/10.1002/pam.21691/abstract.


20 Patrick Wolf, “The DC Opportunity Scholarship Program: Making the American Dream Possible,” written testimony, US House Committee on Oversight and Government Reform, May 14, 2015, 3, https://oversight.house.gov/wp-content/uploads/2015/05/Wolf-writtentestimony-on-the-DC-OSP-2015-final-final-version.pdf. 21 Mark Dynarski, Ning Rui, Ann Webber, and Babette Guttman, “Evaluation of the DC Opportunity Scholarship Program: Impacts After One Year,” National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education, April 2017, xiii, https://ies.ed.gov/ncee/pubs/20174022/pdf/20174022.pdf. 22 Matthew Chingos and Paul Peterson, “Experimentally Estimated Impacts of School Vouchers on College Enrollment and Degree Attainment,” Journal of Public Economics 122 (2015): 7, http://www.sciencedirect.com/science/article/pii/S0047272714002461. 23 Jay Greene, “Vouchers in Charlotte,” Education Matters, Summer 2001, 57, http://media. hoover.org/sites/default/files/documents/ednext20012_46b.pdf. 24 UNC Charlotte Urban Institute, “Supported Students’ High School Outcomes, 2006-2014,” Children’s Scholarship Fund-Charlotte, 2015, http://www.csfcharlotte.org/bw/wp-content/ uploads/2015/10/CSF-C-Student-Outcomes-Report_FINAL.pdf. 25 Zahid Kisa, Melissa Dyehouse, Toby Park, Brian Andrews-Larson, and Carolyn Herrington, “Evaluation of the Florida Tax Credit Scholarship Program Participation, Compliance and Test Scores in 2014-15,” Learning Systems Institute, Florida State University, June 2016, vi, http:// www.fldoe.org/core/fileparse.php/5606/urlt/FTC_FinalReport-14-15.pdf. 26 Ibid. 27 David Figlio and Krzysztof Karbownik, “Evaluation of Ohio’s EdChoice Scholarship Program: Selection, Competition, and Performance Effects,” Thomas B. Fordham Institute, July 7, 2016, https://edexcellence.net/publications/evaluation-of-ohio%E2%80%99s-edchoicescholarship-program-selection-competition-and-performance. 28 Ibid., 2. 29 Ibid.


30 Jonathan Mills and Patrick Wolf, “The Effects of the Louisiana Scholarship Program on Student Achievement After Two Years,” School Choice Demonstration Project, University of Arkansas & Education Research Alliance, Tulane University, February 24, 2016, 2, 4, https:// papers.ssrn.com/sol3/papers.cfm?abstract_id=2738805. 31 Ibid., 36, 38. 32 Opening Statement of Dr. Patrick J. Wolf, University of Arkansas, “School Vouchers: Friend or Foe?” Askwith Debate, Harvard University Graduate School of Education, Cambridge, MA, May 1, 2017, http://www.uaedreform.org/downloads/2017/05/wolf-opening-statementschool-choice-friend-or-foe.pdf. 33 Ibid. 34 NC General Statutes Chapter 115C Article 39, accessed April 26, 2017 from http://www.ncga. state.nc.us/EnactedLegislation/Statutes/pdf/ByArticle/Chapter_115C/Article_39.pdf. 35 North Carolina State Education Assistance Authority, “Opportunity Scholarship Program Summary of Data,” accessed June 23, 2017, http://www.ncseaa.edu/documents/OPS_ Summary_Data.pdf.


Parents for Educational Freedom in North Carolina (PEFNC) is a statewide organization that supports greater educational options through parental school choice, such as public charter schools and the Opportunity Scholarship Program. Our mission is to inform parents of the benefits of expanded options and empower them to exercise freedom in meeting their children’s needs, regardless of race, national origin, income or religion.

pefnc.org

OSP White Paper  

NORTH CAROLINA OPPORTUNITY SCHOLARSHIPS: Countering a Flawed Duke Report and Setting the Record Straight

Read more
Read more
Similar to
Popular now
Just for you