Re-Evaluating Standardized Testing in the Public Schools

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

Dissertation presented to the Faculty of the California School of Education Alliant International University San Diego In partial fulfillment of the requirements for the degree of Doctor of Education by Janie My Linh Dam, JD November 27, 2019

Approved by: Chris Pilkington, EdD, Chairperson Shirley Baker, EdD James Pulido, PhD

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

© Janie My Linh Dam, 2019

ii

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

iii

Dedication I dedicate this work to Dr. and Mrs. Phap and Lily Dam. Combined mom and dad contributed over 60 professional years to K-12 and higher ed in the United States, from classroom teaching all the way to district level management. Without my loving, devoted parents I would not be the person that I am today.

This paper was written applauding fellow public educators in California and everywhere else for their integrity and hard work. My decision to transition from courtroom to classroom in 2000 turned out to be the right one. While rule of law is foundational to the democratic state, an embrace of free public education for all will deliver democracy’s prosperity. As our great country proudly upholds the ideals of human diversity and dignity, it is an envisioned bright future that teachers continue to instill in future generations.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

iv

Acknowledgements As the chair of my dissertation committee, Dr. Chris Pilkington facilitated my doctoral progress and provided key insights and perspectives that helped me pull it all together. Many thanks to you, Dr. Chris, for your very positive energy and for recommending that I select student assessment as the dissertation topic. Sometimes one cannot see right under one’s nose; you always point me in the right direction.

Dr. James Pulido’s indispensable guidance as my methodologist bolstered the academic rigor and value of my study. In the relatively short period of time that I worked with Dr. Pulido I have grown significantly as a quantitative researcher. Richard Gardner I must also thank for the additional SPSS expertise. Being able to make statistical analysis so much fun: that’s real talent, Richard.

I very much appreciate Dr. Kathy Hayes and the Los Angeles Unified School District Committee for External Research Review (CERR) for their comments on the study proposal. I look forward to expanding my research post-doc.

Last but not least, I would like to acknowledge Dr. Shirley Baker, who has been instrumental in getting me back on track to finally complete my EdD. It has been a long and winding road, and without Dr. Baker’s support I would not have reached my destination.

Having arrived in the United States as a child political refugee at the age of 10, I became an English as a Second Language learner at the end of fifth grade, beginning of sixth grade. Dr. Baker is the embodiment of open-heartedness, kindness and unending patience—qualities often found in ESL teachers whom she teaches and trains, and who help to open that critical first door to the American dream. I am deeply grateful as a student.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

v

Abstract The PSAT, SAT, ACT and California Smarter Balanced standardized tests were analyzed and compared as predictors of college persistence. The study used test score and college enrollment data of 142 2016 graduation cohort members at a large, demographically diverse public high school in suburban Southern California. Results from MANOVA did not indicate significant differences between the tests in question. Results from MLR detected significance only between ACT ELA score and college persistence and between SAT math score and college persistence. Based on findings an argument was made for gleaning from, but replacing, the college admission tests with an improved version of the California Smarter Balanced as single new assessment better designed to measure and predict students’ future career success, whether via college or vocational school attendance. This simultaneous streamlining and realignment action serves to avoid the unhealthy overtesting of students, substantially save costs, and minimize instructional disruption while collecting the objective data needed for tracking state and national progress toward a better educated workforce.

Implementation of 21st century advanced technologies is also recommended to feasibly produce a more authentic and complete skills-set profile for each student.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

vi

Table of Contents Dedication ............................................................................................................................... iii Acknowledgements ................................................................................................................ iv Abstract .....................................................................................................................................v List of Tables ........................................................................................................................ viii List of Figures ......................................................................................................................... ix CHAPTER 1: Introduction .....................................................................................................1 Background of Problem .................................................................................................1 Statement of Problem .....................................................................................................6 Purpose of Study .............................................................................................................7 Research Question ..........................................................................................................8 Significance of Study.......................................................................................................8 Methodology ..................................................................................................................10 Definition of Terms .......................................................................................................10 Organization of Study...................................................................................................13 CHAPTER 2: Literature Review .........................................................................................14 Oldest Account of Standardized Testing ....................................................................14 Standardized Student Assessment in American Public Schools ...............................14 Testing for Equity in Higher Learning .......................................................................17 Affirmative action ................................................................................................24 Standardized Tests as Political and Economic Management Tools .........................27 Minimum-competency movement during the Cold War .................................27 A nation at risk. ....................................................................................................28

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

vii

NCLB, ESSA, and the Common Core Standards .............................................29 Predictors of College and Career Success ..................................................................32 Postsecondary Assessment Culture .............................................................................36 CHAPTER 3: Methodology ..................................................................................................38 Restatement of Problem ...............................................................................................38 Purpose of Study ...........................................................................................................39 Research Question ........................................................................................................40 Research Design ............................................................................................................40 Data Collection ..............................................................................................................46 CHAPTER 4: Findings ..........................................................................................................50 MANOVA ....................................................................................................................50 Multiple Linear Regression (MLR) .............................................................................51 CHAPTER 5: Discussion .......................................................................................................56 Major Findings and Implications ................................................................................56 Contributions to the Literature ...................................................................................59 Limitations .....................................................................................................................60 Future Applications ......................................................................................................60 Concluding Remarks ....................................................................................................63 REFERENCES .......................................................................................................................67

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

viii

List of Tables Table 1. Multivariate Tests: Means, Standard Deviation, Significance, Effect Size………51 Table 2 Multivariate Tests: Pillai’s Trace, Wilks’s Lambda, Hotelling’s Trace, Roy’s Largest Root ........................................................................................................... 52 Table 3. Tests of Between-Subjects Effect .......................................................................... 52 Table 4. Bonferroni Post Hoc Test. ............................................................................................. 53 Table 5. Multiple Linear Regressions: All Test Scores ....................................................... 54 Table 6. Multiple Linear Regressions: PSAT English, ACT English, SAT Math .............. 55

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

ix

List of Figures

Figure 1. Levels of college persistence…………………………………………………….53

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

1

CHAPTER 1 Introduction Background of Problem Access to higher education is recognized as being vital to the 21st century workforce (Perna, Finney, & Callan, 2014). The projection is that by the year 2020, over 60% of all jobs in the United States will dictate knowledge and training beyond high school. Furthermore, an additional 3.1 million jobs anticipated to be vacated at the retirement of the large baby boomers generation will need to be filled (Perna et al., 2014). Research has revealed an even greater demand for postsecondary education among populations historically underrepresented in higher learning (Perna et al., 2014). The 2013 Policy Brief of the American College Board cited that the share of U.S. jobs requiring a postsecondary degree will grow to 63% over the next decade. This translates to a quest for 22 million new college degrees and at least 4.7 million new workers with postsecondary certificates. Yet American institutions of higher learning are projected to only be able to produce roughly 19 million degrees at the current rate, three million short.

The forecasted shortage of qualified workers has fueled a national desire to assess high school students of all demographic backgrounds on a grand scale, in a standardized manner, for college and career preparedness (Every Student Succeeds Act, 2015). Standardized exam scores continue to be a key factor in the American college admissions process (GPA; Grove, 2010). As one admission officer expressed, “Let’s face it, it’s easier to compare numerical data than it is to decide whether a semester in France should be ranked higher than a state soccer championship” (Grove, 2010, p. 1).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

2

The two most widely recognized traditional college entrance exams come from the private sector: the SAT and the ACT. The SAT was first developed and administered by Educational Testing Service (ETS) in 1925, on behalf of the College Board, a private organization based in New Jersey (College Board, 2018c). The ACT was created in 1959, for American College Testing, also a private organization, to compete with the SAT. The ACT was authored by University of Iowa professor Everett Franklin Lindquist (ACT, 2018a). In 2011 the ACT surpassed the SAT in the number of test takers for the first time. That year 1,664,479 students took the SAT and 1,666,017 students took the ACT. By 2017 over 2.03 million high school students across the country took the ACT (Pope, 2012).

Since the 2017-18 school year, each SAT exam costs $50 without essay and $65 with essay (College Board, 2019b). Students are additionally charged $15 for registration by phone; $30 per change in test date, testing location or test type; $30 for late registration; and $53 to get onto a waitlist to test on standby. Additional score reports above the four copies included in the exam fee are $12 each; scores by phone are $15 per phone call. A score report rush order costs $31 and also $31 to retrieve archived older scores. The College Board charges $55 per test for verification of test results by handscoring multiple-choice answer sheets and essays (College Board, 2019b). ACT test fees are significantly higher, but include six copies of the score reports: $150 per exam with no writing and $166.50 per exam with writing. Late ACT registration costs $32 per exam, as does test date change. Test center change incurs a charge of $30 per exam (ACT, 2019a).

For high school students planning to take the SAT, the Preliminary SAT (PSAT) has for many decades served as a practice test (College Board, 2018c). The PSAT index score benchmarks the student’s progress toward college readiness. High school juniors with top scores

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

3

on the PSAT, also known as the National Merit Scholarship Qualifying Test (NMSQT), can qualify for National Merit scholarships that cover their college tuition. The PSAT/NMSQT currently costs $17 per test (College Board, 2019a). More recently in 2016, the Preliminary American College Test (PreACT) became available as a preparatory assessment for high school sophomores planning to take the ACT (ACT, 2018c). Since 2016 launching the PreACT is offered at $12 per test (ACT, 2016).

In the public schools, governmental educational agencies have been focusing on standardized student assessments that address academic achievement differences among K-12 demographic subgroups. The cost of state-mandated testing reached as high as $1.7 billion a year in 2012 (Ujifusa, 2012). The No Child Left Behind Act (NCLB), which became law in 2001 to reauthorize the Elementary and Secondary Education Act (ESEA) of 1965, was specifically drafted to measure and eliminate disparities in learning performance among student racial groups (Kim & Sunderman, 2005). The national legislation included school accountability provisions for closing “the achievement gap between high- and low-performing children, especially the achievement gaps between minority and non-minority students, and between disadvantaged children and their more advantaged peers” (No Child Left Behind, 2001, Sec. 1001 [3]).

Leveling standardized test scores between demographic subgroups became “the most significant educational challenge facing American society in the 21st century” (Kim & Sunderman, 2005, p.

3). Under NCLB each state was to establish its own annual measurable objective (AMO) targets based on standardized test scores, and a minimum group size, that is, the minimum percentage of students in each demographic subgroup required to meet or exceed the AMO targets (No Child Left Behind, 2001 Sec. 1111 [b][2][G][iii]). The federal mandate required 95% of students overall and 95% of each student subgroup within a school to take standardized reading and

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

4

mathematics tests. In compliance with this mandate, California created the Standardized Testing and Reporting (STAR) program to manage its public schools’ administration of the California standards tests (CSTs) in English Language Arts, Algebra I, Algebra II, Geometry, and in the science and social studies subjects (California Department of Education, 2018).

In 2015 the Every Student Succeeds Act (ESSA) replaced NCLB (Every Student Succeeds Act, 2015). Unlike preceding ESEA reiterations, ESSA shifted educational authority from the federal government to state and local education agencies (Sharp, 2016). The new act was intended to (a) ensure that states set high college and career standards; (b) enable states to maintain accountability by directing resources towards schools that require improvement; (c) empower states and local education agencies to use appropriate evidenced-based interventions that foster school improvement; (d) encourage states to preserve annual assessments as an informing mechanism that does not overshadow teaching and learning; (e) increase access to quality preschool programs for more children; and (f) secure new resources for identifying and investigating promising educational practices as well as replicating proven strategies that enhance students’ educational outcomes (The White House, Office of the Press Secretary, 2015 in Sharp, 2016). In California, the California Assessment of Student Performance and Progress (CAASPP) program replaced the STAR program. CAASPP provides standardized tests that measure students’ college and career preparedness. The Smarter Balanced Summative Assessments (SBAC) are CAASPP’s adaptive computer-based tests that rate students’ English and math literacy skills linked to college and career success (California Department of Education, 2018).

The average amount of time that American public school students currently spend on standardized testing is significant (Lazarin, 2014; Pandya, 2011; Ravitch, 2016). The reality

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

5

facing American school children today is the pressure to become college and career ready starting as early as the third grade (California Department of Education, 2018). For high school students there is an overlap of multiple high-stakes college admissions exams, statemandated testing (Amrein & Berliner, 2002), not to mention semester finals tied to report card marks, at the end of the Spring semester. Increasingly college bound students are taking both the SAT and ACT in order to raise their chances of being admitted and receiving college tuition scholarships (The Princeton Review, 2018).

In this context the body of literature has brought forth alarming evidence of students experiencing high stress levels and illnesses attributed to excessive high-stakes standardized testing (Moon, Brighton, & Callahan, 2002; Mulvenon, Stegman, & Ritter, 2005; Ryan, Ryan, Arbuthnot, & Samuels, 2007; Segool, Carlson, Goforth, Van der Embse, & Barterian, 2013; Wood, Hart, Little, & Phillips, 2016). Surveys of English learners further disclosed that overemphasis on standardized test preparation drills can take time away from essential instruction, including lessons designed to enhance students’ college and career critical thinking skills (Giambo, 2017). According to the Council of Chief State School Officers, 39 states now have to examine how to alleviate overtesting in response to public debates and outcries (Ujifusa, 2015).

Another factor driving the college and career readiness movement is the overwhelming amount of unpaid student loan debt. College tuition is now the second highest consumer debt category, higher than both credit cards and auto loans, and behind only mortgage debt (Friedman, 2018). There are over 44.2 million borrowers who collectively owe $1.52 trillion in student loan debt in the U.S. (Friedman, 2018). The average student in the 2016 class has $37,172 in student loan debt. Roughly 20% of all U.S. student loan borrowers are from

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

6

California, Florida, Texas, and New York (Friedman, 2018). In fact, Chou, Looney, and Watson (2017) argued that an institutional cohort’s student loan repayment rate, defined as the portion of a cohort’s initial loan principal that is repaid within five years after leaving school, is a good indicator of student outcomes, institutional quality, and the return on federal loan dollars.

Although for nearly a century scores on standardized college entry exams have been deemed indispensable objective measures for admissions (Grove, 2010), there are now over 150 accredited American colleges and universities declaring themselves test-optional and not requiring SAT and ACT scores to be submitted by applicants (FairTest, 2018; Saracino-Lowe, 2015). Since 1969 these institutions of higher learning have adopted Bowdoin College’s philosophy that standardized test scores are not definitive ratings of a person’s intelligence or skills as a college student (Bowdoin College, 2018). Top ranking schools in this list include Pitzer College, New York University, Hampshire College, Lewis & Clark College, University of Texas at Austin, Cornell College, Arizona State University, American University, Montana University, and George Washington University (Saracino-Lowe, 2015). This trend begs the question of whether SAT and ACT standardized test scores are truly needed.

Statement of Problem Given the potential impact on student wellness and instructional time, it is worthwhile to address excessive standardized testing. Clearly there is a need for an objective way to track the nation’s progress toward a diverse, college-educated workforce, so the elimination of standardized testing altogether is not a viable option. Streamlining would appear to be a more reasonable approach. The fact that traditional college admissions exams and emerging statemandated school accountability assessments are now both focused on students’ college readiness present an opportunity for streamlining. Are all the existing standardized assessments truly

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

7

necessary, or could one of them suffice? How do the California’s Smarter Balanced Summative Assessments, PSAT, SAT, and ACT compare and rank as predictive objective measures of college readiness that can help states track their progress toward meeting the large scale demands for highly educated workers?

Purpose of Study The purpose of this study was to investigate the correlation between students’ results on college readiness standardized tests and their success in college. College success is often equated with college persistence, that is, the student’s successful completion of his/her freshman courses and return for subsequent years of study in college. High college persistence rates among graduates and alumni are considered indicators of a high school’s effectiveness in preparing students for college (Stewart, Lim, & Kim, 2015). If, with respect to college persistence, standardized state-mandated assessments are found to possess a level of predictive power equal to or higher than that of traditional college entry exams, then an argument could be made for replacing scores from traditional college exams, such as the SAT and ACT, with results from the state assessments, such as the California Smarter Balanced, in the selection of college freshmen.

This is particularly pertinent to admissions into state universities such as the University of California or California State University systems. Such a substitution would help to greatly reduce the amount of time and effort that students typically spend on proving they are college and career ready, while still yielding the objective data needed for state government decisionmaking. Findings from this research also help to inform college and university regents, industry leaders, and policymakers in deciding whether and how to use PSAT, SAT, ACT, and/or California Smarter Balanced test scores in the college admissions process.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

8

Research Question The answer to the following research question was sought to meet the purpose of this study: How do the California Smarter Balanced Summative Assessments (California Smarter Balanced), Preliminary SAT (PSAT), SAT, and ACT compare as predictors of college persistence?

Significance of Study Having only been operative since the 2014-2015 school year, the California Smarter Balanced Summative Assessments (California Smarter Balanced) are relatively new.

Independent research on California Smarter Balanced that is not sponsored by the state is still very limited. This study has the potential to become a significant addition to the existing body of research on the capacity of state standardized testing to accurately predict students’ college and career readiness (Cimetta, d’Agostino, & Levin, 2010; Fina, Welch, & Dunbar, 2017; Koretz (2017). This research was intended to spur more in-depth research on the usefulness of California Smarter Balanced as a model for using a state assessment as a measure of college and career readiness as well as a college admissions tests based on college persistence.

By the time this dissertation study was conducted, there were only four years of archived Smarter Balanced student results available for analysis: test scores from school years 2014-2015, 2015-2016, 2016-2017, and 2017-2018. Students take this test during their junior (third) year of high school. This means that students who took the test in 2017-2018 were still completing their final senior year of high school during the study, and those who tested in 2016-2017, 2015-2016, 2014-2015 would fall in the age groups typically enrolled in the first, second, and third year of college, respectively. Therefore only test score data

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

9

from school years 2015-2016 and 2014-2015 belonging to students graduating from high school during the 2016-2017 and 2015-2016 school years (2016 and 2017 graduation cohorts) would potentially be usable for this study.

This study reflected upon Zwick’s empirical work over three decades on the role of College Board standardized test scores in American college admissions. In her Harvard University Press publication, Who Gets In: Strategies for Fair and Effective College Admissions, Zwick (2017) reviewed the equitability and effectiveness of the admissions process: Unlike these alleged [testing] experts, I am not dismissive of the value of tests. In the absence of comprehensive evidence about a test’s value, I am neither a believer nor a disbeliever. . . . Asking whether standardized tests are good is much like asking whether cars are good. If they’re constructed well and used intelligently, they can be very valuable. Some are just plain bad. Even good ones can be harmful if misused.

None are perfectly reliable. The decision to use a particular admissions criterion, whether it be tests, grades, income, or grit, can be evaluated in terms of its fairness and effectiveness. (Preface) If California Smarter Balanced scale scores and performance levels are measured to be just as predictive of students’ college persistence as compared to traditional college admissions exam scores, a claim can be supported that colleges and universities, even the growing number of test-optional institutions in the United States and worldwide, should accept state standardized test results in lieu of SAT and ACT test scores. This dissertation is particularly relevant to state university admissions. Tied specifically to state common core curriculum and instruction, state-mandated assessments are expectedly more aligned with the state’s academic goals than are privately developed national or international college admissions exams. Furthermore, unlike the ACT and SAT or other College Board exams, state testing is equally and equitably offered on the public school campus, during school hours, and at no cost to all

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

10

students of all demographic subgroups. Publicly funded schools under state mandate and support are currently equipped to facilitate at minimum a 95% student testing participation rate. The practice proposed by this study represents a recognizable step toward the higher goal of equal access to higher learning among all students of all backgrounds. New understanding gained from this research will help educators to empirically answer the question: Can state-mandated high school achievement tests serve to select college students? Whether yes or no, the answer can have impactful implications for the states’ commitment to K-12 standardized college and career readiness testing. It may influence postsecondary college admissions culture and protocol in California, nationally and worldwide.

Methodology Archival test score data from the California Smarter Balanced Summative Assessments, PSAT, SAT, and ACT were provided by one large demographically diverse California public high school with a student population of over 4,500. A MANOVA (Multivariate Analysis of Variance) was run in IBM SPSS to detect significant differences as to effect on college persistence between the independent variables (IVs) of California Smarter Balanced, PSAT, SAT, and ACT test scores, as defined in the Definitions of Terms section below. When the researcher determined that results from the MANOVA were not adequate to satisfactorily answer the research question, MLRs (Multiple Linear Regressions) were conducted in SPSS. Definition of Terms For the purpose of this study, the key terms are defined as follows:

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

11

ACT: National standardized test developed at the University of Iowa for American College Testing in 1959 (ACT, 2018a). ACT student results comprise of subject area scores in English, math, reading, and science, and a composite score that is the average of all the subject area scores. Raw subject area scores are converted to a scale score between 1 and 36.

The English section tests students on usage/mechanics and rhetorical skills. The math section is composed of questions in pre-algebra/elementary algebra, intermediate algebra/coordinate geometry, and plane geometry/trigonometry. The reading section assesses social studies/natural sciences reading skills, and arts/literature reading skills. The science section contains three distinct question types to evaluate students’ performance in data representation, research summaries, and conflicting viewpoints (ACT, 2018b). The recently made available ELA score is the rounded average of the English, reading, and writing scores; and the new STEM score is the rounded average of the math and science scores (ACT, 2019b).

CAASPP (California Assessment of Performance and Progress): State of California’s system for collecting K-12 student academic performance data for school accountability, using the Smarter Balanced Assessment Consortium (SBAC) assessment system. With the exception of students eligible for special education alternate assessments and English learners who are in their first 12 months of attending a school in the United States, all California students in Grades 3 through 8 and Grade 11 take the Smarter Balanced Summative Assessments. These web-based computer-adaptive assessments consist of a multiple choice test section and a Performance Task (PT) section (California Department of Education, 2018). Assessment results include the student’s overall scale score and overall achievement level in the subjects of English language/literacy and mathematics, claim

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

12

scores (scale scores in each of the skill areas of the tested subject), and claim achievement levels. The four English language/literacy claims or skill areas are reading, writing, listening and research/inquiry. The three mathematics claims or skill areas are concepts and procedures; problem solving and modeling and data analysis, and reasoning communication.

Grade 11 Smarter Balanced scale scores range from a minimum of 2299 to a maximum of 2795. The achievement levels are Level 1, Standard Not Met; Level 2, Standard Nearly Met; Level 3, Standard Met; and Level 4, Standard Exceeded (California Department of Education, 2018).

PSAT (Preliminary SAT): Preparatory version of the SAT typically taken by high school students. The PSAT total score is the sum of the Evidence Based Reading and Writing (EBRW) and math scale scores. The EBRW and math section scores range from 160 to 760 each. The PSAT total scale score range is 320-1520 in 10-point increments (College Board, 2018a).

SAT: A standardized test widely used for college admissions in the United States.

Originally called the Scholastic Aptitude Test by the College Board in 1925, the test was renamed the Scholastic Assessment Test, the SAT I Reasoning Test or SAT Reasoning Test, and now simply the SAT. The overall SAT score is the sum of the Evidence Based Reading and Writing (EBRW) score and the mathematics score, each of which ranges from a minimum of 200 to a maximum of 800. EBRW subscores indicate student mastery level in command of evidence, words in context, expression of ideas, and standard English conventions. The math subscores indicate student mastery level of algebra, problem solving and data analysis, and passport to advanced math. The SAT total score range is 400-1600 (College Board, 2018b).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

13

Organization of Study This dissertation study is presented in five chapters. Chapter 1 covers the background and statement of the problem, purpose of study, research question, significance of study, methodology, definitions of terms, and organization of study. Chapter 2 reviews the relevant body of literature to provide an overview and historical context of the research, as well as critical analysis of the problem. Chapter 3 describes in more detail the research design and statistical procedures used in the study. Chapter 4 presents findings of the study based upon statistical analysis of the data collected. Chapter 5 discusses the implications of the findings, and makes recommendations for practical applications and future expansion of the research.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

14

CHAPTER 2 Literature Review Oldest Account of Standardized Testing The first recorded occurrence of standardized testing in the world can be found in the history of Chinese civil servant selection, as early as 2200 B.C. Candidates for government jobs were required to demonstrate proficiency in literature and writing, arithmetic, music, horsemanship, archery, and public/social ceremonial rites. In the archery test, applicants were given three arrows to shoot at a human-shaped and sized object while riding on horseback. If all three arrows hit the target, a perfect score was earned; if two of them reached the target, the performance was graded as good; and if only one arrow met the target, it was a pass (Miyazaki, 1976).

Standardized Student Assessment in American Public Schools American standardized testing began in the mid-1800s, as a consequence of compulsory public education. In 1845 standardized written essay exams replaced the traditional practice of students individually and orally reciting from memory to their teachers to demonstrate what they knew. This change came with Horace Mann’s advocacy for more objective, efficient student testing. Oral examination was time-consuming and taxing on teachers, and criticized as being too subjective (Giordano, 2005; Rothman, 1995).

Mann was an education reformer known for the Common School Movement, which was a movement dedicated to a more equitable school system characterized by quality teachers and a nonsectarian approach (Cremin, 2019). As Secretary of the Massachusetts State Board of Education, Mann believed that a common public school system was needed to provide the growing number of immigrant youths the tools they needed to succeed. He argued that

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

15

standardization was the solution to burgeoning school enrollments and a diverse student body (Smith, 1934/2002). Despite the general attitude at the time favoring exclusive education for the elite, Mann’s ideas became appealing to the school system as compulsory matriculation continued to greatly expand and diversify the student population (Rothman, 1995). Mann further promoted that regular written tests could serve as instruments to compare the quality of teaching among schools (Caldwell & Courtis, 1925). Soon after, students’ written essays and short answers were read and scored to rate the effectiveness of their teachers and school programs (Resnick, 1982). Decades later, in the late 1880s, Emerson White, a leader of the National Education Association (NEA), began adamantly opposing the use of written exams to compare schools and teachers. White also fought against students’ writing scores being used as the sole indicator for student promotion and retention determination, and insisted that test-focused instruction was detrimental to education: They [written tests] have perverted the best efforts of teachers, and narrowed and grooved their instruction; they have occasioned and made well-nigh imperative the use of mechanical and rote methods of teaching; they have occasioned cramming and the most vicious habits of study; they have caused much of the overpressure charged upon schools, some of which is real; they have tempted both teachers and pupils to dishonesty; and last but not least, they have permitted a mechanical method of school supervision . . . They [the teachers] shut their eyes to the needs of the pupil and put their strength into what will “count” in the examination. (White, 1886, pp. 199-201) In 1888, White again rejected tying student promotion to written exams, and announced that NEA schools were returning to teacher recommendations based on student daily work (White, 1888).

White’s vigorous opposition, however, did little to deter the development of standardized IQ and achievement tests for school aged children. In 1905, French psychologists Alfred Binet and Theodore Simon published the first IQ test, a 30-item exam

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

16

to identify students unable to succeed in school in France (Wolf, 1973). In 1914, psychologists Henry Goddard, Edmund Huey, and Lewis Terman presented their American version of the Binet test, the Stanford-Binet, designed to measure the intelligence of American students (Resnick, 1982). Between 1908 and 1916, American psychologist Edward L. Thorndike and his students at Columbia University worked on measuring student achievement instead. They developed standardized achievement tests in arithmetic, handwriting, spelling, drawing, reading, and language (Wigdor & Gardner, 1982). The Thorndike handwriting test has been considered one of the first norm-referenced test, which had a strong impact on the development of future standardized achievement tests (Pearson & Stallman, 1994).

Reading was among the last subject areas to be assessed with a standardized test, with the very first one emerging in 1915 (Smith, 1934/2002). Test developers struggled to come up with a design that could adequately measure reading achievement. First, because reading is a highly complex process, researchers could not agree on what kind of skill reading actually was. Secondly, in the past, reading instruction had always involved reading aloud, so a silent standardized reading test would be incongruent with how students practiced reading in class. On the other hand, by its very nature oral reading required individual assessments, which defeats the purpose of standardized testing (Smith, 1934/2002). By the early 1900s, however, there was a shift from teaching oral reading to silent reading. A number of researchers made the case for silent reading as being superior to reading aloud in terms of both speed and comprehension. As printed materials became more available and the amount of text that needed to be read vastly increased, silent reading became important and standardized tests were developed specifically to assess silent reading

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

17

skills. Teachers began to focus their instruction more on silent reading (Smith 1934/2002).

Readence and Moore (1983) recognized three types of standardized test questions on standardized reading comprehension tests used in the early 1900s: reproducing a passage test, solving written puzzle tests, and answering question tests. An example of a reproducing a passage test is Starch’s test (1915) where the student is given a limited amount of time to read several passages silently and then write down everything from memory he or she could remember from the passages. The teacher then assigns a reading comprehension score based on the number of words that correctly retold the passage. Unlike Starch, Kelly (1916) did not believe that the ability to reproduce a passage is a good measure of reading comprehension. Instead, the Kansas Silent Reading Test that he designed included multiple puzzles of increasing difficulty, which students were required to read silently and answer within a certain amount of time. Arthur Otis’s Stanford-Binet multiple-choice questions were puzzle-solving questions based on the Kansas Silent Reading Test (Samelson, 1987).

Thorndike (1914) created the Scale Alpha test, which consisted of short paragraphs followed by questions of increasing complexity. Students were given as much time as needed to answer the questions, and the answers were scored on a scale of 0 to 4 to assess the degree of correctness. Thorndike’s answering questions format later became the standard model for reading comprehension assessment (Readence & Moore, 1983).

Testing for Equity in Higher Leaning The evolution of college admissions examination. Since the 1635 founding of Harvard University, the first American institution of higher learning, college admission requirements have passed through four distinct stages. The first era, the era of subjectivity, occurred between the 1600s and 1800s; the second era, search for uniformity, spanned from 1893 to the mid-1900s;

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

18

the reach for objectivity lasted through the 1960s; finally giving rise to a more holistic approach to student selection (Cabrera & Burkum, 2001).

With roots in Old World Europe, American higher education was in the beginning the pursuit of personal development by the elite few who possessed the means to do so. The vast majority of the population was resigned to a life of physical labor without opportunity for advanced learning. During the colonial period, it was the privileged “wealthy White boys” who attended university (Thut, 1957, p. 15). It could be said that at the time, the ability to pay one’s tuition was the primary criterion for college attendance. Beyond economic barriers, prospective college students in the era of subjectivity also faced numerous entrance requirements that were specific to each university. Early admissions criteria were often subjective and theological in nature. Religious denominational characterization of personal virtues led to a great variation in college entry standards during this period (Cremin, 1964). The diverse content of these entrance exams placed tremendous demands on the secondary schools as they attempted to adapt their curricula to university admission and matriculation. University applicants often traveled for a week or longer to distant campuses to take exams that were only offered during certain times of the year (Crouse & Trusheim, 1988).

The aim of the 17th century grammar school was to prepare pupils for college, and the purpose of the college was to cultivate an enlightened clergy for the people (Broome, 1963). For most of the 1700s, the college president conducted a personal interview of each freshman candidate. Carefully crafted interview questions evaluated applicants’ knowledge of classical languages and particular literary or theological readings, and determined their moral character (Cabrera & Burkum, 2001). Selection of the languages and readings and the definition of moral character were highly subjective. They varied widely from one university to another, from one

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

19

college president to the next. As colleges and universities grew in both number and size, it became less and less feasible for college presidents to manage all the personal interviews themselves. The passing on of interviewing tasks to faculty members generated even greater subjectivity and inconsistency, as academic departments tended to introduce their own admissions biases. Consequently, by the 19th century, because of the many admission standards as there were colleges and departments to which students applied, college preparatory schools were pressured to groom students (Linn, 1993). The most significant change to college admissions requirements during the colonial period was the addition of proficiency in vulgar arithmetic, or basic mathematics (Linn, 1993, p. 16). Yale University codified this requirement in 1745 (Broome, 1963). This introduction of objective and uniform protocol in the college admissions process marked the end of the era of subjectivity, and ushered in the era of uniformity (Cabrera & Burkum, 2001).

By 1845, under Horace Mann’s influence, American students were being tested for the first time in a consistent, standardized manner in the public schools. The Boston school system had begun using preprinted short-answer tests (California Postsecondary Education Commission [CPEC], 2001). Five decades later, in 1893, the leading private universities concurred to enact uniform admissions standards, including testing, and recommended a common curriculum for all college preparatory high schools. By 1899 the North Central Association (NCA) was founded to bring uniformity and standardization to higher education (Cabrera & Burkum, 2001). On November 17, 1900, 12 colleges and universities established the College Entrance Examination Board (The College Board) as a subdivision of the Association of Colleges and Preparatory Schools of the Middle States and Maryland. The College Board charter members were Barnard College, Bryn Mawr, Columbia, Cornell, Johns Hopkins, Rutgers, University of Pennsylvania,

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

20

Vassar, and the Women’s College of Baltimore (Fuess, 1950). In June 1901, 973 students wrote 7,889 essays that comprised the first standardized test administration by the College Board (Fuess, 1950). These tests were designed to rate applicants’ understanding of specific subjects, not to determine natural aptitude (Linn, 1993). Although the testing process was becoming more standardized, the scoring of student essays remained largely subjective; and no analyses were ever performed to verify whether the highest scoring students were ultimately admitted.

Spurred by the reported successes of the U.S. Army’s Alpha test as an objective classification tool for soldiers’ abilities during World War I, by 1919 the College Board became interested in developing objective tests of intelligence purely for use in college admissions. The Alpha test was based on the works of Binet and Simon in measuring human intelligence (Crouse & Trusheim, 1988). In their search for a reliable method to identify defective children within the French public school system, psychologist Alfred Binet and his physician assistant Theodore Simon proposed that children’s abilities could be ranked by the most difficult cognitive tasks that they are observed to perform correctly, as compared to their peers (Sacks, 1999). In 1912, German psychologist Wilhelm Louis Stern had proposed taking Binet and Simon’s measurement of mental age and dividing it by the child’s chronological age. Stern used the term IntelligenzQuotient or IQ for mental age divided by chronological age multiplied by 100 (Hunt, 1993). Four years later in 1916, psychologist Lewis Terman introduced the Stanford University Revision of the Binet-Simon Scale, later known simply as the Stanford-Binet. Robert M. Yerkes, as president of the American Psychological Association and newly appointed U.S. Army head psychologist, quickly engaged the new Stanford-Binet. Yerkes believed that intelligence was inherited and fixed in quantity. He was determined to carry out one of the largest intelligence test administrations in history (Sacks, 1999). Between 1914 and 1918, assisted by Carl Campbell

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

21

Brigham, a professor of psychology at Princeton University, Yerkes advocated the use of large scale mental testing using the Army Alpha test. The U.S. military granted Yerkes permission to test the intellectual abilities of over 1.75 million soldiers during this time period (Sacks, 1999).

Yerkes (1921) explained that his tests measured innate intelligence, which by definition was not influenced by either culture or education. He reported three major findings from the testing of army recruits: (a) the average mental age of White American adults was determined to be 13, which Yerkes classified as being at the top of the moronic category; (b) European immigrants could be graded by their country of origin, with the fair-skinned people of Northern and Western Europe scoring higher than the Slavs of Eastern Europe and the darker people of Southern Europe, and (c) the average score of Black men was considerably below the White men’s average. By the end of the war, army camps were using the Alpha test to screen GIs for officer training, and the practice was subsequently emulated by both industries and academia. Yerkes (1921) described the trend as a “steady stream of requests from commercial concerns, educational institutions, and individuals for the use of army methods of psychological examining or for adaptation of such methods to special needs” (p. 96). At the same time, his assistant Brigham used the data to support the idea of genetic differences among the races, and to claim the genetic superiority of the Nordic people from Northern Europe (Gould, 1982).

After the ending of the First World War, people began realizing that society as a whole would benefit from a better educated workforce overall. This belief began to replace the concept of college education as an exclusive pursuit of the privileged few or a particular race. For the first time, many institutions of higher learning were flooded with applications but unable to accommodate. Colleges and universities became focused on finding a way to more efficiently select incoming freshmen (National Research Council, 1999). Standardized test scores became

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

22

recognized as the way to instill objectivity into the college admissions process (Linn, 1993), mirroring what was happening with student assessment in the public schools. Following World War I, standardized testing flourished in education. It was estimated that the public schools and universities hired in the range of 200 to 300 psychologists with prior work experience in the army’s testing program. People were in awe of standardized tests, which they believed to be impartial, reliable instruments that could very quickly and accurately evaluate individual intellectual abilities (Chapman, 1988).

In April 1925, the College Board hired Carl Campbell Brigham to lead a commission to develop a standardized university admissions test. Brigham’s team selected the name Scholastic Aptitude Test, or SAT, for this new assessment, which very much resembled the Army Alpha.

The term aptitude was specifically selected to distinguish the SAT from the achievement exams used by other college admissions officers at that time (Crouse & Trusheim, 1988). Brigham encouraged colleges and universities to treat the SAT as an innate intelligence test. His commission argued that because the SAT was curriculum neutral, it was a tool for leveling the playing field for applicants who experienced inadequate secondary school preparation and therefore lower academic achievement. Another selling point the commission offered for the SAT was that it relieved secondary schools from having to teach to the test and to adapt their curriculum and instruction toward passing a test (Crouse & Trusheim, 1988).

The first administration of the SAT took place in 1926, attended by 8,040 college applicants. The first SAT edition with multiple-choice questions arrived in 1935 (Linn, 1993).

Although the invention of multiple-choice questions enabled large numbers of people to be tested at the same time, completed tests still had to be graded by hand using stencils.

Handscoring was time-consuming and vulnerable to human error. In 1931, Reynold B. Johnson,

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

23

a high school teacher in Ironwood, Michigan, invented the first test scoring machine, the Markograph (Lemann, 1999). By 1937, International Business Machines (IBM) successfully built and marketed devices that scored standardized tests by sensing electrical conductivity of lead pencil marks on paper (Arndt, 2011). This advancement in automation gave public schools and universities the capacity to process thousands of standardized tests in a cost-efficient manner.

In 1948, with the recommendation of Harvard University President James Bryant Conant, the Educational Testing Service (ETS) was established as the central testing agency for the whole nation. The College Board subsequently assigned ETS to develop all of its standardized assessments (Board of Admissions and Relations with Schools, 2002). By 1956, College Board SAT became entirely multiple-choice in format (Linn, 1993). Although the machine scoring was considered a college admissions miracle, promising to remove subjectivity and cronyism from the process, researchers have since pointed to the SAT’s disparate impact on minority and lowincome students (Healy, 1999; Olivas, 1997; Zwick, 1999). In 1999, the U.S. Department of Education’s Office for Civil Rights (OCR) published a warning to institutions of higher learning that the use of standardized tests may be considered discriminatory under certain circumstances (Gose & Selingo, 2001). Per the OCR, if an exam has a significant disparate impact on racial minorities and females, the institution is obliged to show that scores from the exam are an educational necessity and that no practical alternative mechanism exists of equivalent or better quality that could achieve the same educational goal with less disparate impact. Further questions were subsequently raised regarding standardized tests as accurate predictors of academic success (Adelman, 1999; Hiss, 1993; Miller, 2001).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

24

The most popular alternative to the SAT is the ACT. In contrast to the SAT, which is an aptitude test of reasoning and verbal abilities, the ACT was designed to be an achievement test that measures what students have learned in school. ACT is an acronym originally standing for American College Testing, and became operational November 1959. The current version of the SAT contains four sections—reading, writing and language, math, and essay (optional)—to assess students in evidenced-based reading and writing (critical reasoning) and mathematics. The ACT is comprised of five portions: English, math, reading, science reasoning, and essay (optional). Adapting to the ACT’s scoring structure, the SAT now no longer penalizes students by docking points for guessing incorrectly on a question (The Princeton Review, 2018).

Although college admissions committees review the score on each section of the SAT, they tend to consider the ACT’s overall composite score. Students weak in one content area could still end up with a very good ACT score by being strong in another (The Princeton Review, 2013).

Affirmative action. In 1961 President John F. Kennedy signed Executive Order No. 10925 into law to require that employers act affirmatively in hiring and treating employees fairly regardless of their race, creed, color, or national origin (Sovern, 1966). During President Lyndon B. Johnson’s tenure, the 1964 Civil Rights Act (1964) was passed to help low-income people through education and health programs, stimulate the economy and create new jobs, and remove race and gender barriers to educational and employment opportunity. Title VI of the act was subsequently applied to bar discriminatory practices in college admissions. Any educational institutions not in compliance with the federal requirements were at risk of losing financial assistance (Moses, 2001). Affirmative action in higher education admissions is a policy where the applicant’s race, ethnicity, and sex are taken into account in making the selection decision (Moses, 2001). In 1995 President Bill

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

25

Clinton solidified his support for affirmative action (Purdum, 1995), but it was during his term that the race-conscious policy came under strongest attacks (Moses, 2001).

Spearheaded by Ward Connerly, the University of California Board of Regents voted to ban the consideration of race and ethnicity in admissions decisions in 1995. During the November 1996 elections, Proposition 209, California’s ballot initiative titled the California Civil Rights Initiative (CCRI) was passed with 54% of the vote (Moses, 2001). The resulting constitutional amendment abolished all preferences based on race, ethnicity, and sex. The constitutionality of the amendment was challenged by opponents in court, but in 1997 the Supreme Court affirmed the 9th Circuit Court of Appeals’ ruling and upheld Proposition 209 (Moses, 2001).

From 1996 to 1998 University of California, Berkley’s African American freshman annual enrollment dropped by more than 60%, and Latino enrollment went down by nearly 50% (Weiss, 1998). Comparable trends were observed when Texas also eliminated affirmative action in its university admissions protocol (Card & Krueger, 2005). On March 18, 1996, in deciding the Hopwood v. Texas case against the use of race-based admissions criteria to attain diversity at the University of Texas Law School, the 5th Circuit Court of Appeals effectively reversed all affirmative action policies in higher education admissions in the 5th Circuit states: Texas, Louisiana, and Mississippi (Moses, 2001). In 1998, Washington state was similarly grappling with Initiative 200, approved by 59% of voters (Moses, 2001), to prohibit state action that discriminates against or grants preferential treatment in public education to individuals or groups based on race, sex, color, ethnicity, or national origin (Holland, 1999). This occurred in the context of legal contest against the rejection White students’ applications to the University of Washington Law School, and two

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

26

other similar lawsuits against the University of Washington and the University of Michigan Law School (Moses, 2001).

The steep decline in enrollment of historically underrepresented student populations prompted counteractive steps by University of California administrators (Zwick, 1999). The role of privately developed college admissions tests came under scrutiny. Eugene Garcia, dean of the School of Education at Berkley, was quoted as stating, “We . . . have evidence that the SAT loses 2,000 Latino students this year alone” (Colvin, 1997, p. B-2). In a March 1999 interview, Richard C. Atkinson as president of the UC system declared that he would be willing to replace the SAT if the newly approved California high school exit exam (CAHSEE) proved to be a better test (Cabrera & Burkum, 2001).

Per Zwick (1999), In popular press, the existence of bias in admissions tests is typically assumed to be demonstrated by the persistent pattern of differences between racial groups in average test scores. . . . When academic researchers investigate the fairness of the SAT and the ACT, however, they don’t ordinarily focus on the average scores achieved by each ethnic group. Instead, they consider another aspect of the test results: How well does the SAT predict college grades for each group? (p. 2) For instance, the College Board conducted a study in 1994 to evaluate the SAT’s predictive power. That study found that contrary to popular belief, SAT scores on average tended to predict lower college grades than actually attained by Asian and White American students, and higher college grades for Native, African, and Hispanic American students who actually achieved.

Perhaps the most prominent debunking of the SAT was Dr. Atkinson’s announcement in 2001 that high school students seeking to enter the University of California system would no longer have to take the SAT. Putting the spotlight on the new holistic era of college admissions, Dr. Atkinson (2001) proposed that it was time that “all campuses move away from admission

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

27

processes that use narrowly defined quantitative formulas and instead adopt procedures that look at applicants in a comprehensive, holistic way” (p. 1). The College Board subsequently modified the SAT in 2005. ETS eliminated certain types of questions from the exam that were considered ambiguous, particularly cultural analogies. To address the rising number of perfect scores, the test was also made slightly more challenging, and a new writing section with an essay was added. The new SAT, named SAT Reasoning Test or SATII, was first offered on March 12, 2005. The mathematics section was expanded to cover three years of high school mathematics, and the verbal section’s name was changed to the critical reading section (College Board, 2009a). SAT Suite of Assessments, the latest version of SAT, is described by the College Board as “an integrated system of tests that measure what students are learning in class, and what they need to succeed in college” (College Board, 2018b, p. 2). The same College Board brochure refers to assessment designs based on “what research and evidence have shown to be essential for college readiness and success” (p. 2), but it is not clear which research and what evidence.

The SAT is a three-hour to three-hour and 50-minute test of critical reading, writing, and problem-solving ability wholly developed and administered by ETS and proprietarily owned by the College Board.

Standardized Tests as Political and Economic Management Tools Minimum-competency movement during the Cold War. During the 1950s and 1960s standardized testing clearly became an instrument of national political and economic management, no longer just an assessment tool (Koretz, 2008). National security and education concern rose as the Cold War escalated. After the Soviet Union’s launching of Sputnik in 1957, Congress passed the National Defense Education Act of 1958 (NDEA) and then the Elementary and Secondary Education Act of 1965 (ESEA; Haney, 1984). Both of these legislative acts

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

28

mandated expanded use of standardized testing. The focus on standardized testing can be attributed to two factors. First, prior to World War II, 25% of high school students were on a college track, another 25% were on a vocational track, and the remaining 50% were expected, and accepted, to drop out (Berliner & Biddle, 1995). After World War II, however, public alarm over low graduation rates led to curricular adjustments such as more engaging courses, relaxed academic standards and grading procedures designed to avoid student failure were made to keep students in school. While these adjustments did result in lower dropout rates, growing distrust of the public schools ushered in the minimum-competency movement. This movement called for standardized tests that were constructed to guarantee that high school graduates could read, write, and solve basic arithmetic problems. Between 1963 and 1964, 73 state laws were passed to hold public schools accountable for raising student achievement (Wise, 1979). By the mid1970s, 36 states have mandated minimum-competency tests to measure basic skills (Rothman, 1995). The California High School Exit Exam (CAHSEE) later became such a test. Second, the national average scores on the SAT began to drop in 1963, and by 1977 they have plummeted by 49 points on the verbal section and by 31 points in the mathematics section. The public’s fear over this 14-year decline in scores motivated the College Board and ETS to conduct a study. The research panel suggested that soft educational standards, deterioration of the family unit, and excessive television watching were to blame for this downward trend in standardized test scores, and recommended even more standardized student assessment (Wirtz, 1977).

A nation at risk. Under President Ronald Reagan, Secretary of Education Terell H. Bell appointed the National Commission on Excellence in Education to report on the condition of public schools and universities in the United States. In response to growing concern that the American education system was inherently flawed, the commission released its report in April

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

29

1983: A Nation at Risk: The Imperative for Educational Reform (National Commission on Excellence in Education, 1983). In a severe tone, the commission portrayed the state of affairs as being very bleak, warning that “if an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war” (National Commission on Excellence in Education, 1983, p. 5). The commission cited multiple reasons for alarm during the 1980s: declining SAT scores, poor performance by American students on standardized international comparative student assessments such as the Programme for International Student Assessment (PISA), and low high school test scores. It also identified grade inflation, social promotion, and lax standards as further evidence of educational malaise. The commission urged educators to return to teaching the basics and to incorporate standardized tests at various grade levels. A Nation at Risk tremendously influenced state education policies, and spurred the growth of standardized testing nationwide. By 1987-1988, 45 states and the District of Columbia had implemented annual statewide standardized testing to assess student achievement (Marks, 1989).

NCLB, ESSA, and the Common Core Standards. Since 1965 the Elementary and Secondary Education Act (ESEA) has remained the primary codification of federal spending on K–12 schools, articulating the nation’s commitment to equal education opportunity for all students, regardless of race, ethnicity, disability, English proficiency, and income (Essex, 2015).

The act was the most extensive federal educational bill the US Congress ever passed, and was enacted as part of President Lyndon B. Johnson’s War on Poverty during the Viet Nam War and civil rights era (Essex, 2015). Both the No Child Left Behind Act (NCLB) and the Every Student Succeeds Act (ESSA) are reiterations of ESEA (Every Student Succeeds Act, 2015; No Child Left Behind, 2001). Strongly recommending greater academic rigor at the K-12 level,

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

30

and alignment of high school graduation requirements to college entrance requirements, College Board researchers presented that a substantial portion of the American student population is insufficiently prepared for postsecondary education. The Board also proposed that inadequate preparation can result in great costs to the nation, particularly in the form of remedial education and failure to complete college. In California, the Public Policy Institute of California (PPIC) is still reporting that most of the state’s high school graduates are not prepared for college (Johnson, 2017). In spite of the vast majority of California parents (85%) desiring their child to eventually earn a bachelor’s or graduate degree (Baldessare, Dean Bonner, Kordus, & Lopes, 2017), it is estimated currently that only 30% of California ninth graders will receive a bachelor’s degree. The data shows that California needs 1.1 million more skilled workers with at least a bachelor’s degree by 2030 to keep up with current economic demand. Because skilled labor shortfall often entails declining tax revenues, rising unemployment, and increases in costly social services (Johnson, Cook, & Mejia, 2017), understandably college and career readiness has become a priority educational policy in California.

When at the federal level No Child Left Behind was transitioning to Every Student Succeeds in 2010, public school accountability tied to government funding was taking on a new mission: to ensure that every student graduating from a public high school is college and career ready (CCR; Every Student Succeeds Act, 2015). The legislative spirit of ESSA was to retain the hallmark annual standardized testing requirements of NCLB, while shifting federal accountability provisions to the states (Sharp, 2016). In compliance to ESSA, California officially established the California Assessment of Student Performance and Progress (CAASPP) system on January 1, 2014 (California Department of Education, 2018).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

31

CAASPP replaced the Standardized Testing and Reporting (STAR) Program, which had complied with the No Child Left Behind Act (California Department of Education, 2018).

In 2009 a group of state governors and education commissioners launched a national set of state standards called the Common Core. Two multi-state consortia, the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (SBAC), were established to develop assessments aligned with the Common Core State Standards (CCSS) that measure whether students are college and career ready (CCR; Camara, 2013). The U.S. Department of Education’s 2009 Race to the Top competition awarded $362 million in grants to these two consortia on the conditions that higher education is involved in the assessment design, and the standardized tests would be used to determine student readiness for freshman level credit-bearing college courses (Camara, 2013).

Performance-level descriptors (PLDs), cut scores, and benchmarks were set for high school assessments, corresponding to college and career readiness. At the lower grades, assessments were designed to determine whether students were on track to be CCR by the end of high school (Smarter Balanced Assessment Consortia, 2012). The elementary and middle school tests served to provide early warning to students who were not on track for credit-bearing college coursework in English and mathematics, and provide them with an opportunity to correct deficiencies to avoid the need for remediation in college (King, 2011).

By the end of 2010, 43 states, two territories and the District Columbia had adopted the Common Core State Standards (CCSS) for K-12 English Language Arts and mathematics, and had joined either PARCC or SBAC (Partnership for Assessment of Readiness for College and Careers, 2012). The first Common Core test administration took place during the 2014-15 school year. In 2016-17 eight states used SBAC and six selected

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

32

PARCCs to measure college and career readiness at the high school level; 20 conducted the ACT, 10 chose the SAT, and two administered both the PSAT and SAT. Seven states used a combination of SBAC or PARCC and ACT or SAT tests. The remainder of the states conducted their own standards-based assessments (Gewertz, 2017). California used the SBAC assessments, and did not mandate the ACT nor SAT. Predictors of College and Career Success College entry exams such as the SAT and ACT have for many decades served as the primary indicators of college success. The relationship between admissions test scores and performance in college has historically been studied based on the grade point average (HSGPA; Sawyer, 2013). With state-mandated assessment now linked to college and career readiness, there is growing research to gauge the predictive strength of high school standardized tests against that of the traditional college admission exams with respect to CCR. McGhee (2003) used linear regression methods to analyze the capacity of the Grade 10 Washington Assessment of Student Learning (WASL) and the Scholastic Aptitude Test (SAT) to predict freshman year grade point averages (FGPA) among students entering five Washington state university campuses in 2001. Both tests accounted for approximately the same proportion of variance in first year GPA. However, the WASL was found to not be as strongly predictive of grades in specific colleges compared to the SAT. Coelen and Berger (2006) measured the predictive value of Grade 10 Connecticut Academic Performance Test (CAPT) scores against that of SAT scores. They discovered that the CAPT and SAT English tests similarly predicted college GPA, accounting for unique proportions of GPA variance when included as simultaneous predictors; but the two mathematics tests diverged in predictive power. CAPT math scores and SAT quantitative scores were equally effective as

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

33

indicators, accounting for the same proportion of variance in predicting college GPAs when used separately. Neither measure added uniquely to the other when used simultaneously in the prediction. Although McGhee (2003) found the WASL comparably potent to the SAT in predicting students’ college GPAs, Coelen and Berger (2006) concluded that the CAPT was limited compared to the SAT as it showed differentiated prediction depending on institution type and subject area. Cimetta and colleagues (2010) investigated whether Arizona’s high school exam, the Arizona Instrument to Measure Standards (AIMS), could accurately predict first- year and cumulative college grade point averages (FGPA and CGPA) among students enrolling in the University of Arizona in 1999 and 2000. Scores from the mathematics and writing sections of AIMS were discovered to be significant positive predictors of freshman GPA. Contrasting what might be anticipated from the Brown and Conley’s (2007) previous finding of strong content alignment between state standards-based tests in reading and the college standards, the reading section of the Arizona state test did not contribute to the variance of GPAs explained. However, although Cimetta and colleagues (2010) concluded that the Arizona data suggested that the state high school exam was not an adequate substitute for the SAT, they still maintained that the state’s exam could potentially serve as a useful indicator of college readiness. Kingston and Anderson (2013) looked at the Kansas State Assessment (KSA) as predictor of student success in dualenrollment college courses, that is, courses for which students receive both high school and college credit. High school KSA scores were examined as reliable indicators for the success of high school students taking college level algebra and English at a Kansas community college. Results positively showed that the KSA accurately predicted both college course grades and scores on the dual-enrollment course placement tests.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

34

More recently Nichols-Barrer, Place, Dillon, and Gill (2015) compared the PARCC consortium assessments to the Massachusetts Comprehensive Assessment System (MCAS) tests as predictors of student success in college. The MCAS and PARCC test scores were found to predict students’ first-year GPAs equally well and were comparable to SAT scores. In English language arts, the MCAS proficiency benchmark and the PARCC College and Career Readiness (CCR) benchmark represented the same level of student performance. However, Nichols-Barrer and colleagues discovered that the CCR benchmark set by the PARCC consortium in its mathematics assessment predicted a higher level of college performance than the standards-based proficiency benchmark set by the state in the MCAS. This observation was not surprising; many states were likely setting low standards because in general students did not perform as well in math, and state proficiency benchmarks were being set under the NCLB mandate that every student must be proficient by 2014 (for example, Bandeira de Mello, Bohrnstedt, Blankenship, & Sherman, 2015). Nichols-Barrer and colleagues (2015) therefore recommended that if the MCAS tests were to be used for valid college readiness interpretations, its proficiency benchmark level would need to be sufficiently raised. College entry exam scores and GPA data were not included in this study because at that point no students who had taken these state tests had yet progressed to college. The findings from this limited study were nevertheless consistent with those in earlier literature. d’Agostino and Bonner (2009) focused on whether the student performance levels established by the Arizona standard-setting panel for the state’s high school test were at a level suitable to properly define college readiness, based on GPAs corresponding to B and C levels of academic performance. They found that the proficiency standards in writing and mathematics were set at or even higher than collegiate expectations. However, the reading proficiency standard in reading appeared less rigorous. Additional analysis revealed that the

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

35

Arizona reading test provided little useful information about CCR, consistent with the finding of Cimetta and colleagues (2010).

While d’Agostino and Bonner (2009) and Nichols-Barrer and colleagues (2015) studied college and career readiness using existing cut scores, other researchers and assessment programs have set benchmarks for CCR using other methods. For example, as discussed by Furgol, Fina, and Welch (2011) and Theaker and Johnson (2011), college readiness benchmarks were indirectly set by linking high school assessment scores to scores from a college admissions test, and then identifying the point on the high school test distribution that corresponds to the readiness cut score on the college admissions test. The Texas Education Agency, on the other hand, set benchmarks by administering the state’s end-of-course tests to college students and subsequently linked the results to the performance of those same students in the corresponding college courses (Texas Education Agency, 2013).

Prompted by increasing national emphasis on assessing college and career readiness, Koretz and his research team at Harvard Graduate School of Education (Koretz et al., 2016) analyzed the data of students entering the City University of New York (CUNY) and the Kentucky public colleges in 2010, 2011, and 2012. The team examined the outcome variable of FGPA in both the New York and Kentucky samples. In the New York sample the predictors studied were HSGPA, SAT scores, and New York State Regents’ mathematics and English test scores. In the Kentucky sample, HSGPA, Kentucky Core Content Test (KCCT), scores in mathematics and English were the investigated predictors.

In the New York system, correlation to FGPA was found to be similar for the Regents and SAT tests. In English, the correlation between freshman grade point average and Regents

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

36

test scores was slightly higher compared to SAT scores (r = .33 compared with r = .29). In mathematics they were nearly the same (r = .35 and r = .36). Yet correlations between the two tests by subject differed substantially; the correlation between the two mathematics test scores was r = .77 versus r =.57 in English. In the Kentucky system correlations with FGPA were also similar between the state and college entry tests. However, in the New York SAT data, critical reading had the weakest relationship with FGPA; whereas, in the Kentucky data the ACT English test had the strongest ACT English scores and also showed a substantially stronger connection with HSGPA than did the SAT critical reading test.

In their recent University of Iowa testing programs study, Fina and colleagues (2017) agreed with Camara (2013) that the establishment of empirical links between high school assessments and college outcome is essential to college readiness interpretations. Using Iowa Assessments (IA) test scores, ACT scores, college course grades, and first-year college GPA from students in the 2009 high school graduation cohort who subsequently enrolled at the University of Iowa, Fina and colleagues (2017) sought to show that by setting the appropriate benchmarks at the high school level, a state’s standards-based high school achievement test can be empirically validated as a supportive college readiness predictive tool for high school students. Higher scores on the IA in reading, mathematics and science, the three content areas at the center of NCLB and ESSA regulations, were found to be significantly associated with higher grades in General Education Requirements (GER) college courses and higher college freshman GPA. Postsecondary Assessment Culture The body of research cited above is part of the growing literature on the use of results from high school standardized testing for college readiness and success inferences. The

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

37

empirical evidence is beginning to reveal that existing state-mandated summative assessments can or potentially can be on par with traditional college admission exams as indicators of college readiness and success. As interest in the CCR body of research expands, other aspects of college readiness predictors have been brought to the forefront.

For years the disconnect between K–12 standards and postsecondary standards has concerned leading college readiness experts Michael Kirst and David Conley. Their main focus was to encourage college faculty and postsecondary systems to develop a coordinated set of standards so K-12 educators and students can know what it means to be college ready and what is expected of graduating high school seniors (Conley, 2007; Kirst & Bracco, 2004; Kirst & Venezia, 2006). At a KoKoHs International Conference held in Berlin in April 2016, Koretz, spoke of the mistakes made in the past by large scale K-12 standardized testing efforts that should be avoided in building a postsecondary education culture of assessment and evaluation. Per Koretz (2016), that culture is currently virtually nonexistent, but holds great promise. The researcher specifically pointed to the assessment challenges of (a) constructing instruments that measure specific competencies, (b) matching tests to specific uses and inferences, and (c) preventing and addressing function creep, or the gradual over-widening of the use of data beyond the purpose for which it was originally intended. Per Koretz (2016), there are already severe incidences of function creep. For example, the Chicago and Iowa tests of basic skills, which were designed as student diagnostic tools, were being used to hold schools accountable and as criteria for grade level promotion; and the SAT college admissions test, which is meant to predict students’ postsecondary performance, has been used to compare state educational systems.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

38

CHAPTER 3 Methodology Restatement of Problem The body of research points to a significant shortage of college-educated workers in the United States over the next decade. There is a particular need for helping student populations traditionally underrepresented in postsecondary education to obtain higher level knowledge and skills in order to gain employment (Perna et al., 2014). This has resulted in a costly national effort both at the state and federal level to objectively measure the college readiness of all and all demographic subgroups of high school students, an effort that has been criticized by some researchers as being excessive standardized testing negatively impacting student health and instructional programs (Amrein & Berliner, 2002; Lazarin, 2014; Ravitch, 2016). At the same time, a significant number of leading universities has removed traditional college admissions test scores from the freshman selection process (FairTest, 2018; Saracino-Lowe, 2015), based on the belief that standardized test scores are not sufficient indicators of an individual’s potential for success in higher education (Bowdoin, 2018). Moreover, there is a looming $1.5 billion in student college loan debt (Friedman, 2018) that seems to imply a disconnect between the completion of a college degree and the ability to get a job or a high enough paying job. This challenges the whole notion that upon completing their degree program college graduates will become adequately qualified and promptly hired for the millions of jobs anxiously waiting for them to fill. As the designers of both traditional college admissions exams and emerging statemandated high school achievement tests hone in on the very same measure of college and career readiness, the question becomes whether it is truly necessary or worth the expense to continue administering the SAT and ACT alongside California’s Smarter Balanced or other state

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

39

assessments. Or would it suffice, perhaps be more cost-efficient, to simply utilize student results on the state tests both for tracking schools’ effectiveness in delivering student outcomes aligned to state workforce needs, and for making college admissions decisions. This consideration is particularly pertinent to student enrollment in state-funded higher institutions of learning.

Purpose of Study This study aimed to determine how strong student results on the mandated California Smarter Balanced assessments were as predictors of college success compared to traditional PSAT, SAT, and ACT college entrance exam scores. As it is the current practice in the field to focus on student college persistence and retention (Seidman, 2019), college success is defined in this study as college persistence measured by the length of time (number od semesters) that the student remains enrolled in college. Findings from this research help to inform decisions by colleges and universities whether and how to use scores on the PSAT, SAT, ACT, and/or California Smarter Balanced in the admissions process in the future. If the state-mandated assessment is found to possess a level of predictive power equal to or higher than that of traditional college entry exams, then an argument could be made for replacing test scores from the SAT and ACT with student results on the California Smarter Balanced assessments in the college freshman selection protocol, especially at California state universities. Such a substitution would result in a reduction as to the amount of time and effort, not to mention exam fees that students and their families typically spend on college admissions testing. California for instance already has the structure in place to fund and to support the public schools to equitably administer standardized testing to all students of all demographic backgrounds. As such, the use of student results on the California Smarter Balanced Assessments has the potential to contribute to the leveling of the higher learning playing field, post-affirmative action.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

40

Research Question The purpose of the study led to the following research question: How do the California Smarter Balanced Summative Assessments, PSAT, SAT, and ACT compare as predictors of college persistence?

Research Design The study used a quantitative research design to collect and analyze archival data.

Quantitative design was selected for its capacity to analyze larger sets of data and to produce reliable objective findings. To answer the research question, IBM SPSS Multivariate Analysis of Variance (MANOVA) was used to test the following: Null hypothesis 1 (NH1): There is no statistically significant difference between levels of college persistence between California Smarter Balanced, PSAT, SAT, and ACT test scores.

Alternate hypothesis 1 (AH1): There is a statistically significant difference between levels of college persistence between California Smarter Balanced, PSAT, SAT, and ACT test scores.

MANOVA is a procedure to test hypotheses on the effects of one or more independent variables (IVs or IV groups) on multiple (two or more) dependent variables (DVs). In this research the IVs are test scores on the California Smarter Balanced, PSAT, SAT, and ACT. The DVs are the three levels of college persistence based on the number of semesters that the student enrolled and remained in college. However, the reader should note that due to the design of the IBM SPSS software for MANOVA, which allows for data entry of multiple DVs, but for only one IV at a time, the researcher could only make use of the SPSS software by entering the test scores as DVs and college persistence levels as the IVs.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

41

This workaround was only a reversal in the labeling of variables in SPSS, and still yielded the needed analysis of the significant difference as to group means. In interpreting the findings to answer the research question, however, test scores were appropriately treated and referred to as IVs impacting the DV of college persistence. MANOVA uses the F-test to generate statistics to calculate p-values. A p-value that is below 0.05 indicates significant difference (Muijs, 2011). The p < 0.05 significance cut off point is advisable for smaller sample size such as the one in this study (Muijs, 2011). Because the F-test is a global test (Muijs, 2011), it is unclear between which groups the significance lies. SPSS Tukey and LSD post hoc tests were thus applied to help clarify. Effect size (ES) is the size of the treatment effect that the researcher chooses to detect with respect to a given power (Mertler & Reinhart, 2016). It is the measure of the amount of difference between any two groups in standard deviation units (Mertler & Reinhart). Eta Squared (Ρ2 ) is the ES index in SPSS MANOVA (Muijs, 2011). It is calculated by dividing the within-groups sum of squares by the total sum of squares. Eta Squared values range from 0 to 1, where 0-0.1 is a weak effect, 0.1-0.3 is a modest effect, 0.3-0.5 is a moderate effect, and > 0.5 is a strong effect (Muijs, 2011). The SPSS Eta Squared values of each variable was considered. Furthermore, SPSS R Squared and Adjusted R Squared measures were looked at to assess the study modelâ&#x20AC;&#x2122;s overall predictive power (Muijs, 2011). To address Type 1 error, Bonferroni Error Correction was utilized. The more tests are performed on a set of data, the more likely the null hypothesis will be rejected even though it is true. This is referred to as a Type 1 error (Abdi, 2007). The larger the number of tests, the more likely it is to find rare events and therefore the easier it is to make the mistake of thinking that there is an effect when there is none. This can result in erroneously rejecting the null hypothesis. This problem has to do with inflation of the alpha level

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

42

(Abdi, 2007). To be protected from this error, the study corrects the alpha level by using Bonferroni as it performs its 30 tests (ten IVs times three DV groups). The researcher anticipated making this adjustment should the SPSS output suggest the need for it.

In anticipation of potential limitations of the small sample size and of SPSS MANOVA, the researcher decided to also proceed with Multiple Linear Regression (MLR) in SPSS to test the following hypotheses: Null hypothesis 2 (NH2): There is no statistically significant correlation between California Smarter Balanced, PSAT, SAT, and ACT test scores and college persistence.

Alternate hypothesis 2 (AH2): There is statistically significant correlation between California Smarter Balanced, PSAT, SAT, and ACT test scores and college persistence.

MLR serves to describe the relationship between one effect or outcome dependent variable (DV) and multiple predictor independent variables (IVs). Regression works by fitting a straight line between data points (IV and DV coordinate pairs on a scatterplot) so that the overall distance between the points and the line is minimalized, using the least squares statistical method (Muijs, 2011). Each MLR linear equation calculates a p-value that indicates whether the relationship between the DV and IVs is statistically significant. Each of the 10 test score types is entered as a predictor IV. Because linear regression requires that there be a single DV that is represented by continuous data, the college persistence ordinal data analyzed in the previous MANOVA procedure (Level 1 = 0-4 enrolled semesters, Level 2 = 5 enrolled semesters, Level 3 = 6-10 enrolled semesters) were replaced with the number of semesters enrolled.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

43

MANOVA was initially chosen over MLR for this research for several reasons. However, the combined findings from both statistical models served to shed more light on the research question. MLR assumes that the sample is random (Muijs, 2011), which was not guaranteed to be the case here. School catchment areas tend to be more homogeneous than society in general (Muijs, 2011), and the school site data set analyzed here represented a sample school rather than a random sample. Per Aldrich and Cunningham (2015), linear regression results are more reliable where the relationship between IVs are linear or nearly linear. This linear relationship may or may not have existed in this study. Scores from the SAT and the ACT, for instance, may not relate in a straight line as these are two very different standardized tests by definition; SAT is an aptitude test while the ACT is an achievement test; albeit it could be argued that they were evolving in the same direction, toward measuring college readiness skills. There was also the issue of homoscedasticity; that is, the requirement that the variances in the DV are the same for each IV (Aldrich & Cunningham, 2015). In this research homoscedasticity could not be assumed because, for example, students scoring 2400 on the California Smarter Balanced and 26 on the ACT may or may not have the same variability in college persistence level as students scoring 2600 on California Smarter Balanced and 30 on the ACT. A case could be made that because statemandated testing holds the school accountable but not the test taker, in the eye of the student, the California Smarter Balanced assessment is low stakes. Results on the PSAT, SAT, and ACT, on the other hand, directly impact the studentâ&#x20AC;&#x2122;s prospect of being admitted into college, and would therefore be perceived by the student as high stakes. Furthermore, state-mandated tests are administered at no cost to the student, while college admissions exams come with hefty exam fees. The perception of whether the test is high stakes or low

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

44

stakes along with the exam cost to the student can impact and lead to a dichotomy in student attitude and effort on the two types of standardized testing. Consequently, homoscedasticity between the test score IVs may not be strongly present here. As such, MLR was run in SPSS with assumption checks, estimated regression coefficients, Model Fit, R Squared Change, Descriptives, Part and Partial Correlations, Collinearity Diagnostics, and Durbin-Watson residuals. Mahalanobis distance was also applied to detect outlier effects. With these statistical safeguards, the researcher felt satisfied that MLR could still yield useful information to supplement the MANOVA findings in this case.

A number of other related SPSS tools and methods discussed by Muijs (2011) and Aldrich and Cunningham (2015) were considered but not selected for the present research: logistic regression, ANCOVA, and MANCOVA. Logistic regression (LR), also referred to as binary regression in SPSS, is used to measure the DV at the nominal (binary) level (Aldrich & Cunningham, 2015); that is, whether something either does or does not happen when the IV event occurs. In the context of this study, college persistence level was potentially treated as the DV and scores on the 10 standardized tests as the IVs for logistic regression. LR procedures could then be applied to determine college persistence categorically, such as 1 = the student persisted, 0 = the student did not persist, based on scores on the California Smarter Balanced, PSAT, SAT, and ACT. However, LR assumed sample size would have been an issue for the present research (too small a sample size for the number of IVs). Multicollinearity resulting from IVs having a high correlation among one another may also be problematic. Pre-tests would need to be conducted first to ensure that there were weak correlation coefficients between the IVs and a moderate to strong correlation between the DV and each of the IVs (Aldrich & Cunningham, 2015).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

45

Considering that sample size may ultimately invalidate the findings, it was decided that logistic regression was not the ideal method for this research.

ANCOVA (Analysis of Covariance) was also considered by the researcher. This is a method to statistically equate groups in order to strengthen the power of a statistical test. By removing the effect of a known control continuous IV, referred to as a covariate, scores on a single DV may be adjusted to account for any differences in another IV (Aldrich & Cunningham, 2015). The researcher considered assigning college persistence level as the DV, California Smarter Balanced test scores as the primary IV, and test scores on the other three standardized tests as covariates. This could possibly yield some information regarding the effects of the covariate test scores on the DV of college persistence, but this approach was not adequately aligned to the research question in order to yield an answer to it.

ANCOVA demands a linear relationship (homogeneous regression slopes) between the covariate and the IV (Aldrich & Cunningham, 2015), which in this case would be between test scores on the three college entry exams and test scores on the California Smarter Balanced. But that assumption is the very question that this research was attempting to answer.

ANCOVA, however, could be helpful in future related research. Effects on college persistence may possibly be further analyzed by demographic or other continuous IVs. For instance, the influence of standardized test scores on college persistence can be further examined by adding studentsâ&#x20AC;&#x2122; age as a covariate. Also, because students are allowed to take the PSAT in preparation for the SAT, as well as to attempt the SAT multiple times, pre-test scores from PSAT or previous SAT attempts may be assigned as covariates to study effects of latest SAT scores on college persistence.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

46

MANCOVA (Multivariate Analysis of Covariance) as an extension of ANCOVA investigates group differences among several DVs while controlling for effects of IV covariates that may affect the DVs (Mertler & Reinhart, 2016). Potentially a MANCOVA could be used, and would have been more relevant than an ANCOVA to treat multiple DVs. However, MANCOVA was not chosen for this dissertation along the same line of reasoning individual ANCOVAs were not chosen.

Data Collection Archival standardized test scores were obtained at a demographically diverse comprehensive public high school in a Southern California suburban area. The school had a population of nearly 4,800 students in Grades 9 through 12 at the time of the study. Data were collected from the entire 2016 and 2017 high school graduation cohorts. These student groups would have taken the California Smarter Balanced summative assessment as juniors during the Spring semester of the 2014-15 and 2015-16 school years and possibly the PSAT, SAT, and ACT at some point during their enrollment in high school from 2012 through 2017. There were a total of 1,475 students in the 2016 graduation cohort and 1,369 in the 2017 cohort. Participants were randomly assigned unique numerical identifiers to link and compile all test score data into a single data file for SPSS analysis. The data provided by the school included its entire sets of the following test scores for all students in the 2016 and 2017 graduation cohorts: 1. PSAT English reading and writing and math scale scores with test dates 2. SAT English reading and writing and math scale scores with test dates 3. ACT English reading and writing and math scale scores with test dates

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

47

4. California Smarter Balanced (SBAC) Summative English language arts/literacy and math scale scores with test dates 5. California Smarter Balanced (SBAC) Summative English language arts/literacy and math achievement levels with test dates 6. Alumni’s college of attendance descriptors, college semester enrollment begin-andend dates, degree awarded, and graduation status and date.

Because the College Board had redesigned and modified its scoring structure of the PSAT and SAT over the time period which the participants would have taken this test, the research referred to all the applicable PSAT and SAT Concordance Tables (The College Board, 2016, May 9) to concord and convert all PSAT and SAT scores to the most recent College Board college and career readiness scoring structure. For ACT, all English and math scores were collected. However, because all participants were found to have ELA and STEM scores, which are scores most recently designed by ACT to specifically measure college and career readiness, only these scores were included in the analysis, not the traditional math scores and not the traditional English, reading, writing and combined English/writing scores. The ACT ELA score is the combination of English, reading and writing scores, and the ACT STEM score is the average of math and science scores (ACT, 2019b). Because many participants attempted the PSAT, SAT and ACT multiple times, a decision needed to be made whether to use students’ highest scores or most recent scores in the analysis. The decision was to analyze the most recent scores, which would reflect the students’ level of college readiness closest to the time of their entry into college, and hence more relevant to the research question. For California Smarter Balanced, or SBAC, student scores on the 11th grade English language arts/literacy and math summative assessments were collected. Both the scale scores and the 1 through 4 achievement

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

48

levels were included in the analysis. For the same group of students (2016 and 2017 graduation cohorts), college persistence data were collected from National Student Clearinghouse StudentTracker® for High Schools alumni outcome data reports. Semester enrollment beginning and ending dates were used to calculate the total number of semesters that the students enrolled in and completed in college, and to assign them to the appropriate college persistence level group for the study. Upon the first data review in SPSS, it became evident that college enrollment data were still not available for many students in the 2017 graduation cohort.

Out of all students, 142 in the 2016 cohort and 133 in the 2017 cohort were found to have at least one English and one math score in every standardized test being studied. Because the 2016 cohort data set offered a large enough sample size to meet all the assumptions of MANOVA across all the tests, a decision was made to exclude the 2017 cohort data and limit analysis to just the 142 participants in the 2016 graduation cohort.

Based on the assumption that each school year is comprised of two semesters, IV College Persistence levels were originally defined for this quantitative research as follows: •

Level 1 = the student enrolled in one year, or 0 to 2 semesters, of college after graduating from high school.

•

Level 2 = the student returned to college for a second year of study, or enrolled for 3 to 4 semesters.

•

Level 3 = the student returned to college for a third year of study, or enrolled for 5 to 6 semesters.

•

Level 4 = the student completed all four years of college, or enrolled in 7 to 8 semesters.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

49

However, while the sample size of N = 142 was large enough overall for a MANOVA, the natural spread of the ordinal data set in question would not have met the size requirement for each individual IV college persistence level group. That is, for each IV level a MANOVA requires 30 subjects, so four levels of college persistence would require at least 120 subjects. Because the sample size was 142, this assumption was met. But because the data were very unevenly spread among the four levels, not all four levels as originally defined would have had at least 30 subjects for analysis. As such, the IV groups were modified as follows: •

Level 1 = 0 to 4 enrolled semesters (n = 51)

•

Level 2 = 5 enrolled semesters (n = 38)

•

Level 3 = 6-10 enrolled semesters (n = 53)

Typically archival data provided by the California Department of Education come demographically aggregated. Such aggregated data can be used for demographic subgroup analysis should this study lead to future research in that direction. The student college persistence data were collected from National Student Clearinghouse StudentTracker® for High Schools can also be aggregated by demographic subgroups as needed for future research arising out of this study. The collection and treatment of data in the study strictly adhered to the requirements of the Family Educational Rights and Privacy Act (FERPA) from beginning to end to protect the privacy of individual students and their families, and to preserve the confidentiality of the reported data. The selected databases, data, and methods of data collection satisfy all requirements of the Institutional Review Board (IRB) and any applicable standards and assessment research proposal review panel(s).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

50

CHAPTER 4 Findings MANOVA To answer the research question of whether the California Smarter Balanced Summative Assessments (California Smarter Balanced), PSAT, SAT and ACT compare as predictors of college persistence, a Multivariate Analysis of Variance or MANOVA was used to test the Null hypothesis (NH1) that there was no statistically significant difference between levels of college persistence between California Smarter Balanced, PSAT, SAT, and ACT test scores. The alternate hypothesis 1 (AH1) was that there was a statistically significant difference between levels of college persistence between California Smarter Balanced, PSAT, SAT, and ACT test scores. The MANOVA output supported Null Hypothesis 1 that there was no statistically significant difference between levels of college persistence between California Smarter Balanced, PSAT, SAT, and ACT test scores. The cut-off value for significance was p < .05, hence test results (displayed in Table 1) reveal that only ACT English (p = .017) and SAT Math (p = .014) had a statistically significant difference among the college persistence levels.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

51

Table 1 Multivariate Tests: Means, Standard Deviation, Significance, Effect Size __________________________________________________________________________ Variable Mean SD Mean SD Mean SD F p η2 __________________________________________________________________________ English PSAT

570.980

11.214

607.368

12.991 580.566

11.000

2.340

.100

.033

SAT

582.157

11.698

622.632

13.552 609.057

11.48

2.776

.066

.033

ACT

24.275

.643

27.053

.745 25.981

.631

4.198

.017

.057

SB-SS

2686.216

9.919

9.731

2.333

.101

.032

SB-AL

3.510

.093

3.585

.091

.997

.372

.014

2718.947 11.492 2701.792 3.711

.108

Math PSAT

582.941

11.475

616.579

13.293

611.509

11.256

2.334

.101

.03 2

SAT

583.529

11.572

622.368

13.406 628.679

11.351

4.390

.014

.05 9

ACT

24.608

.625

26.237

.613

2.190

.116

.03 1

SB-SS

2709.020

9.955

2728.816

9.766

2.322

.102

.03 2

SB-AL

3.314

.099

3.447

.097

1.919

.151

.02 7

.724

26.0

11.533 2738.679

.114

3.585

Note. PSAT, SAT, ACT refer to scale scores from these tests. SB-SS refers to scale scores on the California Smarter Balanced Summative Assessment and SB-AL refers to achievement level results on this assessment.

Multiple Linear Regression (MLR) Based on the very low levels of significance seen in the MANOVA Multivariate Tests, Roy’s Largest Root being the closest to showing significance (p = .058), the researcher proceeded with Multiple Linear Regression (MLR) to test Null Hypothesis 2 (NH2): There was

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

52

no statistically significant correlation between California Smarter Balanced, PSAT, SAT, and ACT test scores and college persistence (Table 2).

Table 2 Multivariate Tests: Pillai’s Trace, Wilks’s Lambda, Hotelling’s Trace, Roy’s Largest Root __________________________________________________________________________ College Persistence Level Effect p Pillai’s Trace .089 Wilks’s Lambda .092 Hotelling’s Trace .094 Roy’s Largest Root .058 __________________________________________________________________________ A closer look at Tests of Between-Subjects Effect results (Table 3) helped to verify that a significant difference was found in ACT English scale score (p = .017) and SAT math scale score (p = .014) relative to the three levels of college persistence as shown next (Figure 1) after Table 3.

Table 3 Tests of Between-Subjects Effect ____________________________________________________________________ Variable Mean square F p η2 ____________________________________________________________________ English PSAT 15004.368 2.340 .100 .033 SAT 17372.735 2.776 .066 .038 ACT 88.529 4.198 .017 .057 SB-SS 11707.211 2.333 .101 .032 SB-AL .441 .997 .372 .014 Math PSAT 15671.292 2.334 .101 .032 SAT 29977.847 4.390 .014 .059 ACT 43.627 2.190 .116 .031 SB-SS 11736.730 2.322 .102 .032 SB-AL .956 1.919 .151 .027 ____________________________________________________________________

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE Level 1 0-4 semesters (n = 51) Figure 1. Levels of college persistence

Level 2 5 semesters (n = 38)

53

Level 3 6-10 semesters (n = 53)

A Post Hoc Test was then run to determine where exactly the small significant difference lies. Results revealed that it was between college persistence Levels 1 and 2 for ACT English, and Levels 1 and 3 for SAT Math (Table 4).

Table 4 Bonferroni Post Hoc Test __________________________________________________________________________ Variable: Level 1 - Level 2 Level 1 - Level 3 Level - Level 3 p value p value p value __________________________________________________________________________ English ACT .016 .181 .823 Math SAT .090 .018 1.000 __________________________________________________________________________

Results from the MLR included an R Square Change value of .113 and a Significance F Change value of .095 for the study model. The significance p-value was .095, which was greater than .05, and therefore indicates no significance in the regression model. Overall MLR results support Null Hypothesis 2 that there was no statistically significant correlation between California Smarter Balanced, PSAT, SAT, and ACT test scores and college persistence (number of semesters enrolled in college). Closer examination of the output in SPSS Model Summary, however, revealed that only the PSAT English scale score had a significant correlation to the number of semesters enrolled in college (.009), and that it was a negative correlation (-.515 Beta standardized coefficient, -.011 β unstandardized coefficients). This was a surprising finding as it implied that the higher the studentâ&#x20AC;&#x2122;s score,

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

54

the fewer the number of semesters he/she will remain in college. The ACT English scale score approached significance at .058 (+.384 Beta standardized coefficient, +.141 β unstandardized coefficients) and the SAT math scale score approached significance at .084 (+.312 Beta standardized coefficient, .006 β unstandardized coefficients). All other test scores including the California Smarter Balanced scale scores and achievement levels were far from being significantly correlated to the number of semesters that the student remained in college (Table 5).

Table 5 Multiple Linear Regressions: All Test Scores __________________________________________________________________________ Independent Variable Unstandardized Standardized (predictors) Coefficients B Coefficient Beta t p __________________________________________________________________________ English PSAT -.011 -.515 2.654 .009 SAT .003 .135 .663 .508 ACT .141 .384 1.911 .058 SB-SS .003 .108 .498 .619 SB-AL -.407 -.157 -.860 .391 Math PSAT .001 .041 .232 .817 SAT .006 .312 1.739 .084 ACT -.129 -.337 -1.596 .113 SB-SS .003 .114 .529 .597 SB-AL .142 .059 .321 .749 __________________________________________________________________________ Note. Dependent variable: College semester count; R Square Change = .113; Sig. F Change = .09 After removing the test scores found to be insignificant, a second round of MLR was applied just to the PSAT English scale score, ACT English scale score, and SAT math score data. The resulting R Square value of .082 (Table 6) meant that significance was detected in this predicting model comprised of these three test scores. All three proved to be significant,

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

55

the strongest predictor being PSAT English scale score with a significant value of .004 with a reverse correlation (-.445 Beta standardized coefficients, -.009 β unstandardized coefficients). ACT English scale score was in second place with a significant value of .042 (+.290 Beta standardized coefficients, -.106 β unstandardized coefficients), and SAT math scale score in third with a significant value of 0.15 (+.290 Beta standardized coefficients, +.006 β unstandardized coefficients).

Table 6 Multiple Linear Regressions: PSAT English, ACT English, SAT Math __________________________________________________________________________ Independent Variable Unstandardized Standardized (predictors) Coefficients B Coefficient Beta t p __________________________________________________________________________ English PSAT ACT

-.009 .106

-.445 .290

-2.892 2.048

004 .042

Math SAT .006 .290 2.460 .015 __________________________________________________________________________ Note. Dependent variable: College Semester Count; R Square Change = .082; Sig. F Change = .008

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

56

CHAPTER 5 Discussion Major Findings and Implications The purpose of this study was to compare the PSAT, SAT, ACT, and California Smarter Balanced as predictors of college success, defined as college persistence and measured by the number of semesters the student remains in college. This was based on the continued focus in the field on minority college and university student attrition and retention (Seidman, 2019). Which of these standardized tests can best forecast high school studentsâ&#x20AC;&#x2122; persistence in higher learning, to help track progress toward meeting a critical national demand for college educated workers? The findings in this research led to more questions than answers.

There is no clear winner in this competition, and as such, the indication is that additional research and test design work still need to be done. Results from the MANOVA show no statistically significant difference in predictive power among the standardized tests. MLR output revealed some significant correlation between test scores and college persistence, but not across the board for any of the tests (PSAT and ACT for English language arts only, and SAT for math only).

If all the tests in question were equally sufficiently predictive, then the most practical approach would be to streamline and only administer the one test that is least costly, least disruptive to the K-12 instructional program, most equitably accessible to all students from all demographic subgroups, most aligned to state labor force demands. If all the tests in question were equally not sufficiently predictive, then it would still be advisable to administer the one test that is least costly, least disruptive to the K-12 instructional program, most equitably accessible to all students from all demographic subgroups, most aligned to state labor force demandsâ&#x20AC;&#x201D;with

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

57

the caveat that the selected test be further field tested and developed to become a better indicator of college and career readiness. Per the findings of this small study, that one test could be the California Smarter Balanced Summative. Since 2014-15 this school accountability test in English and math has been successfully administered on school campuses during the school day to assess 95% or more of public high school juniors from all demographic backgrounds, without cost to the student and his/her family. Accompanying interim assessments and data dashboards have also been made readily available to teachers and school administrators by the state (California Department of Education, 2018). However, looping back to the methodology discussion in Chapter 3, student’s attitude toward state testing may be an important factor affecting the fidelity of state testing data, and should be further studied. This is particularly pertinent if the data is to be trusted for school accountability reporting. According to Knekta & Eklöf (2015), the test taker’s attitude toward a test can significantly impact its effectiveness as a measuring instrument.

In this regard, there may be a need to shift students’ perceptions of the Smarter Balanced assessments from being low stakes to being high stakes testing. Students not taking statemandated assessments seriously may be an influence that if removed can help to improve the reliability and meaningfulness of the data produced by these standardized tests. Perhaps a study should be conducted with test taker control groups to investigate the effect of high school test takers’ attitude on the reliability or predictive power of state and College Board standardized test scores. Naturally if results from the California Smarter Balanced become the primary objective measure for college admissions (or trade school and job placement), and students are made well aware of this, the anticipation would be that students will take these assessments seriously. As it stands, high schools juniors may not find state-mandated tests to be directly relevant to their entry into college or future success. Furthermore, having to take multiple challenging

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

58

standardized tests at the same time, students will likely have to prioritize their testing efforts.

They expectedly will be choosing between focusing energy on the college exams that their family has invested many dollars in, versus on the state tests, which they may view as serving to hold the school, not students accountable, and which their parents or legal guardians may opt them out of.

SPSS MLR output pointed to a potentially useful predictive model that combines PSAT English scale score, ACT English scale score, and SAT math score data. The resulting R Square value of .082 meant that significance was detected in this predicting model comprised of these three test scores. However, as the strongest predictor, the PSAT English scale score with a significant p value of .004 had a reverse, negative correlation to number of semesters enrolled in college (-.445 Beta standardized coefficients, -.009 β unstandardized coefficients), whereas the ACT English scale score was in second place with a significant p value of .042, which had a direct, positive correlation with the number of semesters enrolled in college (+.290 Beta standardized coefficients, -.106 β unstandardized coefficients). The SAT English scale score had a direct, positive correlation with the number of semesters enrolled in college (+.003 Beta standardized coefficients, +.135 β unstandardized coefficients), but its significant p value of .508 in the initial regression was unacceptable, so it was removed. This was a very surprising discovery to the researcher, particularly in light of recent efforts by the College Board and American College Testing to concord SAT Suite of Assessments test scores with ACT test scores (ACT, 2018d; The College Board, 2018). Although it appears that the creators of SAT and ACT believe that the two tests are aligned, the PSAT, which is the Preliminary SAT, was found upon closer examination to predict college persistence trends in the reverse direction as the ACT. One

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

59

possible explanation for the reverse correlation is that students can earn college course credits for high SAT scores (McWilliams, 2016). That is, students with higher scores are more likely to spend fewer semesters in college because they have placed out of freshman courses with these scores. However, that would apply only to SAT scores, not PSAT scores, which can qualify students for national merit and other scholarships, but not earn them college course credit. Furthermore, the PSAT is purportedly the official practice test for the SAT—so why is the effect of PSAT on college persistence in the opposite direction as SAT effect on college persistence? Is college persistence defined simply as number of semesters enrolled in college an adequate definition for college success? Is PSAT an unnecessary layer of standardized testing that needs to be peeled off? Can college scholarships be tied to state test scores, in lieu of PSAT scores? There are enough unanswered questions to warrant further study. But if the PSAT English scale score is removed from the MLR predictive model, then the remaining combination of the ACT English scale score and the SAT math scale score can potentially serve to inform the design or re-design of the state’s Smarter Balanced assessments in English literacy and math.

Contributions to the Literature This study was built upon the existing body of research as to the capacity of state standardized testing to accurately predict students’ college readiness. As discussed in Chapter 2, studies conducted from 2010-2017 were typically focused on the college freshman GPA as measure of college readiness. The present study, however, investigated college persistence as a measure of college success. This was in the recognition of college student retention as a pervasive theme in higher ed (Seidman, 2019). This research effort

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

60

also offered an analysis of data generated by the most up-to-date versions of the college admissions exams and state-mandated assessments.

Limitations This study was limited by a small sample size, and data that were collected at a single school campus, albeit a demographically diverse one. The research would benefit from a larger number of participants from different school campuses. The study could yield richer findings specific to individual demographic subgroups by adding student demographic data in the analysis. Longitudinal studies are also called upon to track studentsâ&#x20AC;&#x2122; progress over time, covering the full college and university four-year enrollment cycle and beyondâ&#x20AC;&#x201D;toward both career and college success, not just college success. College persistence measured by number of semesters or length of time that the student remains in college may or may not be appropriate or adequate as a definition for college success aimed at career success. First, high performing students may exceed the need of taking college courses based on high school standardized test scores, and therefore enroll in fewer semesters in college. Second, there is a population of students who are better suited for trade school or immediate vocational employment upon graduation from high school than college attendance. Finally, this quantitative study would yield greater understanding if supplemented by an ancillary qualitative study. Findings would have been more deeply informative had the researcher subsequently surveyed and interviewed the 142 participating students on their standardized testing, college admission, and college attendance perspectives and experiences.

Future Applications Horace Mannâ&#x20AC;&#x2122;s vision of a positive, inclusive (non-sectarian), stronger staffed common schooling experience for children of all backgrounds was an important educational

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

61

paradigm shift and contribution to the 19th century First Industrial Revolution workforce and economy. As an essential component of this common school movement, a more efficient and objective student assessment system blossomed and reached maturity with the technical advancements of 20th century Second and Third Industrial revolutions. As we now enter a 21st century characterized by the Fourth Industrial Revolution, also referred to as Industry 4.0 (Prisecaru, 2016), the social goal is still to properly educate youths and prepare them to succeed in life through gainful employment. However, this time around, advancements in artificial intelligence (AI), Internet of Things (IoTs), and Blockchain technologies have the potential to fulfill education reformer Mannâ&#x20AC;&#x2122;s wishes beyond his dreams. Smart phone apps and AI robots, for instance, can well assist human teachers to more feasibly test students in both oral and written format to produce a more authentic and comprehensive skills profile of the student. This profile can then be used to individualize instruction, and to more expeditiously and accurately match the student with the appropriate postsecondary program of study, or with a job upon completion of the instructional program.

The Common School Movement had sacrificed oral recitation for the sake of objectivity and feasibility. There were too many students from highly diverse linguistic and cultural backgrounds to be assessed, and not enough teachers, let alone professionally objective and well-trained multicultural teachers, to individually verbally test them. With todayâ&#x20AC;&#x2122;s tech savvy students clearly seen as preferring silent modes of communication such as texting and emailing, this researcher would recommend special attention to possible decline in oral communication skills. Restoration of oral recitation instructional and assessment methods may be in order. Danish Grundvigian folkschool (Gillespie, 2013) and American MOOC (Massive Open Online Courses) (Moe, 2015) woven together hold

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

62

promise as a more balanced lifelong learning approach. School campus language laboratories currently used every school year to administer the College Board’s Advanced Placement (AP) language and culture exams can be technologically upgraded for this purpose. Technologies such as Pearson’s Intelligent Tutoring Systems (ITS) should be (critically) considered (Luckin, Holmes, Griffiths, & Forcier, 2016). Estonia’s e-governance (Heller, 2017) and higher student performance on PISA (Programme for International Student Assessment) (Sulkunen, 2017), along with Japan’s Society 5.0 aimed at healthy human-centered culture integrating cyber and physical space (Japan Cabinet, 2019), are also worth studying.

Findings in this research can be applied to effectively streamline and improve existing standardized testing. A new single student assessment model can be constructed and implemented to replace all existing standardized tests. Components of the PSAT, SAT, and ACT found to be predictive and useful can be incorporated into the new assessment model, or at least be more closely studied to inform its design. The advantages web-based California Smarter Balanced testing can be weighed against those of traditional paper-based college entrance exams.

The new model would serve to more efficiently and meaningfully measure student performance and progress toward career success via college attendance. Streamlining can potentially significantly reduce all the costs associated with testing currently borne by the school and by the student’s family. It could also help to eliminate the negative effects of overtesting on student wellness and learning.

This limited study is a small but recognizable step toward further leveling the playing field among all student demographic subgroups in the 21st century. Administering the new assessment during the school day on the school campus, aligned to daily classroom instruction

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

63

and at no cost to the student, would afford all students of all backgrounds fair opportunity to demonstrate their academic performance and progress as future contributors to the state’s and nation’s workforce and economy. Being admitted into college should not be dependent upon one’s ability to pay for expensive entrance exams, access to transportation, or inherited knowledge of college attendance culture. The new career readiness assessment would also need to measure the skills of students more suited for trade school or vocational training upon graduation from high school. Quoting a study in England that found the ability to read to be the single best indicator of future economic success (British Broadcasting Corporation News, 2002), Schmoker (2002) wrote that literacy is pivotal to acquiring the type of education that is the path to economic and political power (p. 57). As such, the assessment of reading skills should be prioritized in the design of standardized testing made equitably accessible to all students of all demographic subgroups.

Concluding Remarks Nearly 20 years ago, in the year 2000, Clarke, Madaus, Horn, and Ramos reviewed what they referred to as the very profitable 100-year-old testing industry in the United States as the top two testing companies, ETS and NCS, both hit the $500 million mark in annual revenues by 1998. Clarke and colleagues concluded that technical innovations might have contributed to the U.S. testing marketplace’s rapid growth, but did not necessarily produce better tests or better outcomes for students who use them. As Zwick (1999) continued her work on the fairness and effectiveness of College Board tests, Clarke’s team recommended more careful monitoring of the effects of high-stakes tests on teaching and learning (Clarke et al, 2000, Abstract).

For two decades since the year 2000 the writer of this dissertation experienced first-hand the enormous growth in student assessment and data collection. She can offer substantial

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

64

anecdotal and archival insight in this regard. At the height of No Child Left Behind and California Standardized Testing and Reporting (STAR), she was responsible, as a typical school site testing coordinator, for managing literally a ton of secured testing materials every school year: over 2,200 pounds of California State Test (CST) and California High School Exit Exam (CAHSEE) test booklets and answer sheets were packed and shipped back on a pallet to the state for scoring every Spring. In 2014-15, state testing became computer-based, involving the management of the stateâ&#x20AC;&#x2122;s online test delivery system and a secured network of approximately 2,500 teacher and student devices. At the same time, school wide PSAT 8/9 and PSAT/NMSQT continued to be administered on paper to over 3,000 students in Grades 9-11 every October, with the juniors competing for National Merit scholarships. PSAT test fees alone can cost a single high school well over $40,000 per year if all exam fees are covered for all students. Based on the number of tests cores returned to the school, students and their families are estimated to be investing roughly $70,000 on SAT and ACT a year, not including the costs of test prep programs. Additionally, with the increasing promotion of the Advanced Placement and Capstone programs by the College Board, a single school community can approach $400,000 in AP exam fees and related expenses annually without the College Board and state rebate programs.

Students taking 500 exams toward an International Baccalaureate (IB) diploma in lieu of an AP Capstone certificate would incur at minimum another $60,000 in exam fees every November.

Costly standardized testing has not stopped expanding, but can at times compete against, rather than support and inform, classroom instruction. The American Federation of Teachers (AFT) studied two mid-sized urban school districts in 2013. The research determined that in the most tested grade levels (such as 11th grade in high school) students typically spend 20 to 50 hours per year on testing, and 60 to 110 hours more on high-stakes test prep. Nelson, who

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

65

reported on this study, called it an American obsession with testing (2013). Based on the rate of $6.15 per student to extend the school day by one hour to make up for lost instructional time, AFT calculated the testing cost per pupil to range from $700 to over $1000 per school year. With projections of shortage in the millions of highly-educated workers, and the $1.5 billion in student college loan debt in the backdrop, all eyes are on the cost-effectiveness of K-12 college and career readiness programs, and on the prudence of college admission decisions. Standardized test scores as objective predictors of student outcomes should be a part of the cost-effective solution, not part of the problem of excess and waste. In these times of national economic crisis it is imperative that policymakers and testing experts come together to find the right solution. The recent SAT-ACT concordance between historically competitive College Board and American College Testing reflects good will and is a promising sign.

Per Koretz (2016), student assessments should be instruments to measure specific competencies, and should be well matched to specific uses and inferences. This is to avoid “function creep,” or misguided data use beyond the purpose for which it was originally intended. If the ACT English test and the SAT math test are to inform the state assessments, what specific student competencies do their scale scores reflect? Do these measured competencies support higher learning toward the development of skills in demand at the workplace? Or can they directly translate to workforce skills? Standardized testing is nothing new. Whether it is shooting three arrows from horseback four thousand years ago to earn an archery score of 0, 1, 2 or 3, or taking today’s state-mandated tests to be rated as English literacy and math achievement level 1 = Not Met; 2 = Nearly Met; 3 = Met; or 4 = Exceeded standard, it is still the same concept: demonstration of skill level aimed to meet a proficiency target. The difference, however, is that the ancient horse riding-shooting test was

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

66

much more authentic, perfectly aligned to the job skills involved (however gruesome). How closely related are the test questions or essay prompts on the PSAT, SAT, ACT, and California Smarter Balanced to real-world tasks that students will be expected to accomplish on their future jobs? What exactly do these test question items measure, how do they function? Do they authentically forecast the student’s competence and productivity in the workplace, or do they simply reflect his or her test-taking skills? With respect to school accountability, can the student results collected thus far for California’s School Accountability Report Card (SARC) data dashboard be trusted to drive the state’s decisions and policies, given student test takers’ attitude toward state testing? The phrase “quality over quantity” comes to mind in discussing standardized testing in California and the United States. And when it is clear that things have gone overboard and off mark, it would make sense to streamline and realign.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

67

REFERENCES Abdi, H. (2007). Bonferroni and Sidák corrections for multiple comparisons. Encyclopedia of Measurement and Statistics, 3, 103-107.

ACT (2016). ACT launches new PreACT test for 10th graders. Retrieved from https://www.act.org/content/act/en/newsroom/act-launches-new-preact-test-for-10thgraders.html ACT (2018a). ACT history. Retrieved from http://www.act.org/content/act/en/about- act.html ACT (2018b). How do tests, questions, and reporting categories relate to one another?

Retrieved from http://www.act.org/content/act/en/products-and-services/theact/scores/understanding-your-scores.html ACT (2018c). PreACT for k-12 educators & administrators. Retrieved from http://www.act.org/content/act/en/products-and-services/preact/preact-classroom.html ACT (2018d). ACT/SAT concordance. Retrieved from https://www.act.org/content/act/en/products-and-services/the-act/scores/act-satconcordance.html ACT (2019a). Current ACT fees and services. Retrieved from https://www.act.org/content/act/en/products-and-services/the-act-nonus/registration/fees.html ACT (2019b). Understanding your scores. Retrieved from https://www.act.org/content/act/en/products-and-services/the-act/scores/understandingyour-scores.html Adelman, C. (1999, November 5). Why can’t we stop talking about the SAT? The Chronicle of Higher Education, 46(11), B4.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

68

Aldrich, J. O., & Cunningham, J. B. (2015). Using IBM SPSS statistics (2nd ed.). Thousand Oaks, CA: Sage.

Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing and student learning. Education Policy Analysis Archives, 10, 18.

Arndt, R. (2011, June 5). IBMâ&#x20AC;&#x2122;s history of innovative tech. Fast Company Magazine. Retrieved from Fast Company.com web site:http://www.fastcompany.com/1752598/ibms-historyinnovative-tech Atkinson, R. H. (2001, February 18). Standardized tests and access to American universities: UC and the SAT. Speech delivered at the 83rd Annual Meeting of the American Council on Education Washington, D.C. Retrieved from UC and the SAT web site: http://www.ucop.edu/ucophome/conmserv/sat /speech.html Baldassare, M., Dean Bonner, D., Kordus, D., & Lopes, L. (2017, April). PPIC Statewide Survey: Californians and Education. San Francisco, CA: Public Policy Institute of California Bandeira de Mello, V., Bohrnstedt, G., Blankenship, C., & Sherman, D. (2015). Mapping state proficiency standards onto NAEP scales: Results from the 2013 NAEP reading and mathematics assessments (NCES 2015-046). U.S. Department of Education, Washington, DC: National Center for Education Statistics.

Berliner, D., & Biddle, B.(1995). The manufactured crisis: Myths, fraud, and the attack on America's public schools. New York City, NY: Basic Books.

Board of Admissions and Relations with Schools. (2002). The use of admissions test by the University of California. Berkeley, CA: Author.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

69

Bowdoin College. (2018). Bowdoin at a glance. Retrieved from https://www.bowdoin.edu/admissions/at-a-glance/index.html British Broadcasting Corporation News online. (2002, November 20). Reading “can bring social change” [Online article]. Retrieved from http://news.bbc.co.uk/1/hi/education/2494637.stm Broome, E. C. (1963). A historical and critical discussion of college admission requirements.

Princeton, NJ: College Entrance Examination Board.

Brown, R. S., & Conley, D. T. (2007). Comparing state high school assessments to standards for success in entry-level university courses. Educational Assessment, 12, 137–160.

Cabrera, A. F., & Burkum, K. R. (2001). College admission criteria in the United States: An overview. Madison, WI: University of Wisconsin at Madison.

Caldwell, O., & Courtis, S. (1925). Then and now in education 1845-1923: A message of encouragement from the past to the present. Yonkers-on-Hudson, NY: World Book.

California Department of Education. (2018). California Assessment of Student Performance and Progress (CAASPP) system. Retrieved from https://www.cde.ca.gov/ta/tg/ca/ California Postsecondary Education Commission. (2001). Assessment and change: The role of student testing in California education. Sacramento, CA: Author.

Camara, W. J. (2013). Defining and measuring college and career readiness: A validation framework. Educational measurement: Issues and practice, 32(4), 16-27.

Card, D., & Krueger, A. B. (2005). Would the elimination of affirmative action affect highly qualified minority applicants? Evidence from California and Texas. ILR Review, 58(3), 416–434. https://dx.doi.org/10.1177/001979390505800306

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

70

Chapman, P. (1988). Schools as sorters: Lewis M. Terman, applied psychology, and the intelligence testing movement, 1890-1930. New York City, NY: New York University Press.

Chou, T., Looney, A., & Watson, T. (2017). Measuring loan outcomes at postsecondary institutions: Cohort repayment rates as an indicator of student success and institutional accountability (No. w23118). Cambridge, MA: National Bureau of Economic Research.

Cimetta, A. D. (2012). Understanding the connection between high school exit exams and college performance. Tucson, AZ: The University of Arizona.

Cimetta, A. D., d'Agostino, J. V., & Levin, J. R. (2010). Can high school achievement tests serve to select college students? Educational Measurement: Issues and Practice, 29(2), 3-12.

Civil Rights Act of 1964, Pub.L. 88-352, 78 Stat. 241 (1964).

Clarke, M. M., Madaus, G. F., Horn, C. L., & Ramos, M. A. (2000). Retrospective on educational testing and assessment in the 20th century. Journal of Curriculum Studies, 32(2), 159-181.

Coelen, S. P., & Berger, J. B. (2006). First steps: An evaluation of the success of Connecticut students beyond high school. Quincy, MA: Nellie Mae Education Foundation.

College Board (2009a). The official SAT study guide (2nd ed.). New York, NY: Macmillan.

College Board (2018a). SAT Suite of Assessments PSAT/NMSQT and PSAT 10 scores. Retrieved from https://collegereadiness.collegeboard.org/psat-nmsqt-psat-10/scores College Board. (2018b). SAT suite of assessments SAT scores. Retrieved from https://collegereadiness.collegeboard.org/sat/scores

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

71

College Board (2018c). The SAT suite of assessments. Retrieved from the College Board website: https://collegereadiness.collegeboard.org/pdf/sat-suite-of-assessmentsoverview-brochure.pdf College Board (2019a). SAT suite of assessments PSAT/NMSQT and PSAT 10 taking the tests.

Retrieved from https://collegereadiness.collegeboard.org/psat-nmsqt-psat-10/taking-thetests College Board (2019b). SAT suite of assessments SAT fees. Retrieved from https://collegereadiness.collegeboard.org/sat/register/fees Colvin, R. (1997, October 1). Q & A: Should UC do away with the SAT? Los Angeles Times, p.

B-2.

Conley, D. T. (2007). Redefining college readiness [Report]. Retrieved from http://epiconline.org/files/pdf/RedefiningCollegeReadiness.pdf Cremin, L. A. (1964). The transformation of the school. New York, NY: Vintage.

Cremin, L. A. (2019). Horace Mann American educator. Retrieved from https://www.britannica.com/biography/Horace-Mann Crouse, J., & Trusheim, D. (1988). The case against the SAT. Chicago, IL: The University of Chicago Press. dâ&#x20AC;&#x2122;Agostino, J. V., & Bonner, S. M. (2009). High school exit exam scores and university performance. Educational Assessment, 14, 25-37.

Essex N. L. (2015). School law and the public schools: A practical guide for educational leaders (6th ed.). New York, NY: Pearson Education.

Every Student Succeeds Act. (2015). Pub. L. 114-95. Washington, DC: United States Department of Education.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

72

FairTest. (2018). More than 1000 accredited colleges and universities that do not use ACT/SAT scores to admit substantial numbers of students into bachelor-degree programs.

Retrieved from http://www.fairtest.org/university/optional Fina, A. D., Welch, C. J., & Dunbar, S. B. (2017) Establishing empirical links between high school assessments and college outcomes. ITP Research Series, 2017(3).

Friedman, Z. (2018). Student loan debt statistics in 2018: A $1.5 trillion crisis. Forbes. https://www. forbes. com/sites/zackfriedman/2018/06/13/student-loan-debt-statistics2018.

Fuess, C. (1950). The College Board: Its first fifty years. New York, NY: Columbia University.

Furgol, K., Fina, A. D., & Welch, C. (2011, April). Establishing validity evidence to assess college readiness through a vertical scale. Paper presented at the National Council on.

Measurement in Education, New Orleans, Louisiana.

Gewertz, C. (2017). Which states are using PARCC or Smarter Balanced? Education Week.

Retrieved from https://www.edweek.org/ew/section/multimedia/states-using-parcc-orsmarter-balanced.html Giambo, D. A. (2017). â&#x20AC;&#x153;I will study more . . . and prayâ&#x20AC;?: Metacognition about high-stakes test preparation among culturally & linguistically diverse students. Multicultural Education, 24(3), 26-34.

Gillespie, N. (2013, October 10). Folk High Schools, Adult Education, and the Philosophy of Nikolaj Grundtvig. Retrieved from https://youtu.be/cy2f4RnStlg Giordano, G. (2005). How testing came to dominate American schools: The history of educational assessment. New York City, NY: Peter Lang.

Gould, S. J. (1982). A nation of morons. New Scientists, 6, 349-352.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

73

Gose, B., & Selingo, J. (2001, October 26). The SATâ&#x20AC;&#x2122;s greatest test: Social, legal and demographic forces threatening to dethrone the most widely used college entrance exam.

The Chronicle of Higher Education, A10-A15.

Grove, A. (2010). Are my SAT scores good enough? Retrieved from About.com website: http://collegeapps.about.com/od/sat/f/goodsatscore.htm Haney, W. (1984). Testing reasoning and reasoning about testing. Review of Educational Research, 54(4), 597-654.

Healy, D. (1999). Conflicting Interests in Toronto. Retrieved from https://pdfs.semanticscholar.org/44a4/441a8c37f236d9e7fbac87237aa122ab1860.pdf Heller, N. (2017). Estonia, the digital republic. The New Yorker, 18. Retrieved from http://governance40.com/wp-content/uploads/2018/12/Estonia-the-Digital-Republic-The-NewYorker.pdf

Hiss, W. C. (1993, Summer). The emperorâ&#x20AC;&#x2122;s new clothes? The Journal of College Admissions, 136, 6-9.

Holland, R. (1999). Implementing Initiative 200: Keeping faith with the voters (Washington Policy Center, Policy Notes, 98-06). Retrieved from https://www.washingtonpolicy.org/library/doclib/PN99-06.pdf Hopwood v. Texas, 78 F. 3d 932 (5th Cir. 1996), certiorari denied, 116 S. Ct. 2582 (1996).

Hunt, M. (1993). The story of psychology. New York, NY: Random House.

Japan Cabinet (2019). Society 5.0. Retrieved from https://www8.cao.go.jp/cstp/english/society5_0/index.html Johnson, H. (2017). Improving college pathways in California. Retrieved from the Public Policy Institute of California website: https://www.ppic.org/publication/improving-collegepathways-in-california/

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

74

Johnson, H., Cook, K., & Mejia, M. C. (2017). Meeting Californiaâ&#x20AC;&#x2122;s need for college graduates.

Retrieved from the Public Policy Institute of California website: https://www.ppic.org/publication/meeting-californias-need-for-college-graduates-aregional-perspective/ Kelly, F. (1916). The Kansas Silent Reading Tests. Journal of Educational Psychology, 7(3), 63-80.

Kim, J. S., & Sunderman, G. L. (2005). Measuring academic proficiency under the No Child Left Behind Act: Implications for educational equity. Educational Researcher, 34(8), 3-13. https://dx.doi.org/10.3102/0013189X034008003 King, J. (2011). Implementing the common core state standards: An action agenda for higher education. Retrieved from the American Council on Education website: http://www.acenet.edu/AM/Template.cfm?Section=Home& CONTENTID=39580&TEMPLATE=/CM/ContentDisplay.cfm Kingston, N. M., & Anderson, G. (2013). Using state assessments for predicting student success in dual-enrollment college classes. Educational Measurement: Issues and Practice, 32(3), 3-10.

Kirst, M. W., & Bracco, K. R. (2004). How the K-12 and postsecondary split hurts students, and what can be done about it. Retrieved from https://www.researchgate.net/publication/242188083_how_the_k12_and_postsecondary_split_hurts_students_and_what_can_be_done_about_it Kirst, M. W., & Venezia, A. (2006). Improving college readiness and success for all students: A joint responsibility between K-12 and postsecondary education [Issue paper]. Retrieved

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

75

from the United States Department of Education website: http://www2.ed.gov/about/bdscomm/list/hiedfuture/reports/kirst-venezia.pdf Knekta, E., & EklĂśf, H. (2015). Modeling the test-taking motivation construct through investigation of psychometric properties of an expectancy-value-based questionnaire. Journal of Psychoeducational Assessment, 33(7), 662-673.

Koretz, D., Yu, C., Mbekeani, P. P., Langi, M., Dhaliwal, T., & Braslow, D. (2016). Predicting freshman grade point average from college admissions test scores and state high school test scores. AERA Open, 2(4).

Koretz, D. (2008). Measuring up: What educational testing really tells us. Cambridge, MA: Harvard University Press.

Koretz, D. (2016, April 4). Measuring postsecondary competencies: Lessons from large-scale k12 assessments. Presented at the KoKoHs II International Conference, Berlin.

Koretz, D. (2017). The testing charade: Pretending to make schools better. Chicago: University of Chicago Press.

LazarĂ­n, M. (2014). Testing overload in America's schools. Retrieved from the Center for American Progress website: https://www.americanprogress.org/issues/education-k12/reports/2014/10/16/99073/testing-overload-in-americas-schools/ Lemann, N. (1999). The big test: The secret history of the American meritocracy. New York City, NY: Farrar, Straus and Giroux.

Linn, R. L. (1993). Linking results of distinct assessments. Applied Measurement in Education, 6, 83-102. http://dx.doi.org/10.1207/s15324818ame0601_5 Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. (2016). Intelligence unleashed: An argument for AI in education. Retrieved from

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

76

https://www.pearson.com/content/dam/corporate/global/pearson-dotcom/files/innovation/Intelligence-Unleashed-Publication.pdf Marks, D. (1989). Statewide achievement testing: A brief history. Educational Research Quarterly, 13(3), 36-43.

McGhee, S. E. (2003). The relationship between WASL scores and performance in the first year of university. Seattle, WA: University of Washington, Office of Educational Assessment.

McWilliams, F. (2016). SAT and ACT: Credit by examination. Retrieved from https://info.methodtestprep.com/blog/sat-and-act-credit-by-examination Mertler, C. A., & Reinhart, R. V. (2016). Advanced and multivariate statistical methods: Practical application and interpretation. London: Routledge.

Miller, D. W. (2001, March 2). Scholars say high-stakes tests deserve a failing grade: Studies suggest students and educators are judged by faulty yardsticks. The Chronicle of Higher Education, 47(25), A14-A16.

Miyazaki, I. (1976). China's examination hell: The civil service examinations of Imperial China (C. Schirokauer, Trans.) New York City, NY: Weatherhill.

Moe, R. (2015). The brief & expansive history (and future) of the MOOC: Why two divergent models share the same name. Current issues in emerging elearning, 2(1), 2.

Moon, T. R., Brighton, C. M., Callahan, C. M. (2002). State standardized testing programs: Friend or foe of gifted education?. Retrieved from https://eric.ed.gov/?id=ED463300 Morgan, H. (2016). Relying on high-stakes standardized tests to evaluate schools and teachers: A bad idea. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 89(2), 67-72, http://dx.doi/org/10.1080/00098655.2016.1156628

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

77

Moses, M. S. (2001). Affirmative action and the creation of more favorable contexts of choice.

American Educational Research Journal, 38(1), 3-36.

Muijs, D. (2011). Doing quantitative research in education with SPSS (2nd ed.). Thousand Oaks, CA: Sage.

Mulvenon, S. W., Stegman, C. E., & Ritter, G. (2005). Test anxiety: A multifaceted study on the perceptions of teachers, principals, counselors, students, and parents. International Journal of Testing, 5(1), 37-61.

National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Washington, DC: Author.

National Research Council. (1999). Myths and tradeoffs: The role of tests in undergraduate admissions. Washington, DC: National Academy Press.

Nelson, H. (2013) Testing more, teaching less: What America’s obsession with student testing costs in money and lost instructional time. Retrieved from https://www.csaionline.org/resources/testing-more-teaching-less-what-america’s-obsession-studenttesting-costs-money-and-lost Nichols-Barrer, I., Place, K., Dillon, E., & Gill, B. (2015). Predictive validity of MCAS and PARCC: Comparing 10th grade MCAS tests to PARCC Integrated Math II, Algebra II, and 10th grade English language arts tests. Retrieved from Mathematica Policy Research website: https://www.mathematica-mpr.com/our-publications-andfindings/publications/predictive-validity-of-mcas-and-parcc-comparing-10th-grade-mcastests-to-parcc-integrated-math-ii No Child Left Behind Act of 2001, P.L. 107-110, 20 U.S.C. § 6319 (2002).

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

78

Olivas, M. A. (1997). Constitutional criteria: The social science and common law of admissions decisions in higher education. U. Colo. L. Rev., 68, 1065.

Pandya, J. Z. (2011). Overtested: How high-stakes accountability fails English language learners. New York: Teachers College Press.

PARCC Partnership for Assessment of Readiness for College and Careers. (2019). Retrieved from https://parcc-assessment.org/about Partnership for Assessment of Readiness for College and Careers website. (2012). Retrieved from http://parrconline.org/ Pearson, P., & Stallman, A.(1994). Resistance, complacency, and reform in reading assessment.

In F. Lehr & J. Osborn (Eds.), Reading, language, and literacy: Instruction for the twenty-first century (pp. 239-251). Hillsdale, NJ: Lawrence Erlbaum.

Perna, L. W., Finney, J. E., & Callan, P. M. (2014). Renewing the promise: State policies to improve higher education performance. Philadelphia, PA: University of Pennsylvania, Graduate School of Education. Retrieved from http://repository.upenn.edu/gse_pubs/326 Pope, J. (2012, September 24). SAT scores edge down; ACT now more popular exam. Associated Press. Retrieved from https://news.yahoo.com/sat-scores-edge-down-act-now-morepopular-150110664.html Prisecaru, P. (2016). Challenges of the fourth industrial revolution. Knowledge Horizons.

Economics, 8(1), 57. Retrieved from http://orizonturi.ucdc.ro/arhiva/khe-vol8-nr12016/09.%20Petre%20Prisecaru.pdf Purdum, T. S. (1995, July 20). President shows fervent support for goals of affirmative action.

The New York Times, p. A1.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

79

Ravitch, D. (2016). The death and life of the great American school system: How testing and choice are undermining education. New York: Basic Books.

Readence, J., & Moore, D. (1983). Why questions? A historical perspective on standardized reading comprehension tests. Journal of Reading, 26(4), 306-313 Resnick, D. (1982). History of educational testing. In A. K. Wigdor & W. R. Gardner (Eds.), Ability testing: Uses, consequences, and controversies (Vol. 2, pp. 173-194).

Washington, DC: National Academy Press.

Rothman, R. (1995). Measuring up: Standards, assessment, and school reform. San Francisco, CA: Jossey-Bass.

Ryan, K., Ryan, A., Arbuthnot, K., Samuels, M. (2007). Students’ motivation for standardized math exams. Educational Researcher, 36. 5-13.

Sacks, P. (1999). Standardized minds: The high price of America’s testing culture and what we can do to change it. New York, NY: Da Capo Press.

Samelson, F. (1987). Was early mental testing (a) racist inspired, (b) objective science, (c) a technology for democracy, (d) the origin of multiple-choice exams, (e) none of the above? In M. M. Sokal (Ed.), Psychological testing and American society 1890-1930 (pp.

113-127). London, England: Rutgers University Press.

Saracino-Lowe, M. (2015). 10 colleges that don’t require SAT or ACT scores. Retrieved from https://www.unigo.com/get-to-college/college-search/test-optional-colleges-10-collegesthat-dont-require-sat-or-act-scores Sawyer, R. (2013). Beyond correlations: Usefulness of high school GPA and test scores in making college admission decisions. Applied Measurement in Education, 26(2), 89- 112.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

80

Seidman, A. (2019). Minority student retention: The Best of the “Journal of College Student Retention: Research, Theory & Practice.” New York, NY: Routledge.

Segool, N. K., Carlson, J. S., Goforth, A. N., Von der Embse, N., & Barterian, J. A. (2013).

Heightened test anxiety among young children: Elementary school students’ anxious responses to high-stakes testing. Psychology in the Schools. 50, 489-499.

Sharp, L. A. (2016). ESEA reauthorization: An overview of the Every Student Succeeds Act. Texas Journal of Literacy Education, 4(1), 9-13.

Smarter Balanced Assessment Consortia. (2012). How will Smarter Balanced validate its college and career readiness benchmarks? Retrieved from http://www.smarterbalanced.org/resources-events/faqs/ Smith, N. (2002). American reading instruction (Special ed.). Newark, DE: International Reading Association. (Original work published 1934) Sovern, M. I. (1966). Legal restraints on racial discrimination in employment. New York: The Twentieth Century Fund.

Starch, D. (1915). The measurement of efficiency in reading. Journal of Educational Psychology, 6(1), 1-24.

Stewart, S., Lim, D., & Kim, J. (2015). Factors influencing college persistence for first-time students. Journal of Developmental Education, 38(3), 12-20. Retrieved from http://www.jstor.org/stable/24614019 Sulkunen, S. (2017). PISA reading literacy trends and literacy policy: Estonia & Finland [Powerpoint slides]. Retrieved from http://www.elinet.eu/fileadmin/ELINET/Redaktion/user_upload/Madrid_PISA_Symposium_Sulkunen. pdf

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

81

Texas Education Agency. (2013). State of Texas assessments of academic readiness standard setting technical report. Retrieved from http://www.tea.state.tx.us/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=257698 Theaker, R., & Johnson, C. (2011). College readiness linking study: A study of the alignment of the RIT Scales on NWEA’s MAP assessments with the College Readiness Benchmarks of EXPLORE, PLAN and ACT. Portland, OR: Northwest Evaluation Association.

The College Board. (2016, May 9). Concordance tables. Retrieved from https://collegereadiness.collegeboard.org/pdf/higher-ed-brief-sat-concordance.pdf The College Board. (2018). Guide to the 2018 ACT® /SAT® concordance. Retrieved from https://collegereadiness.collegeboard.org/pdf/guide-2018-act-sat-concordance.pdf The Princeton Review. (2013). The SAT vs. the ACT. Retrieved from the Princeton Review website: http://www.princetonreview.com/sat-act.aspx The Princeton Review. (2018). SAT vs. ACT: Which test is right for you? Retrieved from the Princeton Review website: http://www.princetonreview.com/sat-act.aspex Thorndike, E. (1914). The measurement of ability in reading. Teachers College Record, 15(4), 207-277.

Thut, I. N. (1957). The story of education: Philosophical and historical foundations. New York, NY: McGraw-Hill.

Ujifusa, A. (2012). Standardized testing costs states $1.7 billion a year, study says. Education Week, 32, 13.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

82

Ujifusa, A. (2015). Amid cries of overtesting, a crazy quilt of state responses. Education Week.

Retrieved from https://www.edweek.org/ew/articles/2015/07/08/amid-cries-ofovertesting-a-crazy-quilt.html Weiss, K. (1998, May 21). Fewer Blacks and Latinos enroll at UC. Los Angeles Times, pp. A-3, A-24.

White, E. (1886). The elements of pedagogy. New York City, NY: American Book.

White, E. (1888). Examinations and promotions. Education, 8(1), 517-522.

The White House, Office of the Press Secretary. (2015). White House Report: The Every Student Succeeds Act [Press Release]. Retrieved from https://www.whitehouse.gov/thepressoffice/2015/12/10/white-house-report-everystudent-succeeds-act Wigdor, A., & Gardner, W. (Eds.) (1982). Ability testing: Uses, consequences, and controversies (Vol. 1). Washington, DC: National Academy Press.

Wirtz, W. (1977). On further examination: Report of the advisory panel on the Scholastic Aptitude Test score decline. New York City, NY: College Entrance Examination Board.

Wise, A. (1979). Legislated learning: The bureaucratization of the American classroom.

Berkeley, CA: University of California Press.

Wolf, T. (1973). Alfred Binet. Chicago, IL: University of Chicago Press.

Wood, S., G., Hart, S. A., Little, C. W., & Phillips, B. M. (2016, July). Test anxiety and a high stakes standardized reading comprehension test: A behavioral genetics perspective.

Wayne State University Press, 62(3), 233-251.

Yerkes, R. M. (Ed.) (1921). Psychological examining in the United States Army. Washington, DC: U.S. Government Printing Office.

STANDARDIZED TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

83

Zwick, R. (1999). Eliminating standardized tests in college admissions: The new affirmative action? Phi Delta Kappan, 81(4), 320.

Zwick, R. (2017). Who gets in?: Strategies for fair and effective college admissions. Cambridge, MA: Harvard University Press.