CA Smarter Balanced Assessments and College Admission Tests as Predictors of College Persistence

1

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

Janie Dam, JD, EdD Alliant International University, San Diego California School of Education 16 April 2020

2 Dr. Chris Pilkington • Dissertation Chairperson

COMMITTEE MEMBERS

Dr. Shirley Baker • Dissertation Subject Matter Expert:

Multi-Cultural Education

Dr. James Pulido • Dissertation Research Methodology Expert

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

3 BACKGROUND OF PROBLEM

HIGHER LEARNING IS VITAL TO 21ST CENTURY WORKFORCE

Perna, Finney, & Callan, 2014

EVEN GREATER DEMAND AMONG HISTORICALLY UNDERREPRESENTED POPULATIONS

Perna et al, 2014

QUALIFIED WORKER SHORTAGES FUEL COLLEGE AND CAREER READINESS TESTING

Every Student Succeeds Act, 2015

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

4 BACKGROUND OF PROBLEM

2002

2005

2016

Alarmingly high student stress levels associated with testing

Mulvenon, Stegman & Ritter

Wood, Hart, Little & Phillips

Moon, Brighton & Callahan Ryan, K., Ryan, A., Arbuthnot & Samuels

2004

Segool, Carlson, Goforth,Van der Embse & Barterian

2013

39 states now looking to alleviate overtesting Ujifusa

2019

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

5 STATEMENT

OF PROBLEM How may states and local educational agencies avoid unhealthy overtesting while collecting the objective data needed to track progress toward a diverse higher-educated workforce?

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

6 RESEARCH

QUESTION How do the PSAT, SAT, ACT and California Smarter Balanced Summative Assessments compare as predictors of college persistence?

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

7 LITERATURE

REVIEW

Standardized Testing and College and Career Readiness

Empirical links between high school assessments and college outcomes essential to college readiness interpretations.

Furgol, Fina & Welch, 2011 Theaker & Johnson, 2011 Nichols-Barrer, Place, Dillon & Gill, 2015 Fina, Welch & Dunbar, 2017

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

8 LITERATURE

REVIEW

Postsecondary Assessment Culture

✓ Develop instruments to measure specific competencies. ✓ Match tests to specific uses and inferences to avoid “function creep.”

Koretz, 2016

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

9 METHODOLOGY

Research Question:

How do the California Smarter Balanced Summative Assessments, PSAT, SAT, and ACT compare as predictors of college persistence?

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

10 METHODOLOGY

Research Assumption:

Success in higher learning correlates positively with college persistence (length of enrollment in a college or university)

METHODOLOGY MANOVA Multivariate analysis of variance is a statistical test that simultaneously analyzes the relationship between a common set of predictor variables and several response variables.

In this study, the set of predictors is comprised of students’ test scores on the PSAT, SAT, ACT and California Smarter Balanced Summative Assessments. The response variables are three levels of college persistence.

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

11

METHODOLOGY MLR Multiple linear regression is a statistical technique that uses several explanatory variables to predict the outcome of a response variable.

The goal of MLR is to model the linear or straight-line relationship between the explanatory variables and response variable. In this case, the explanatory variables are scores on the PSAT, SAT, ACT and California Smarter Balanced, and the response variable is college persistence.

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

12

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

13 METHODOLOGY Participants: 142 students in the graduating class of 2016 who took the California Smarter Balanced Assessments as juniors during the Spring semester of 2015 and the PSAT, SAT and ACT some time during their 4 years of high school.

14

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

Major findings MANOVA Difference in predictive power between standardized tests studied was not found to be consistently significant across the levels of college persistence.

DISCUSSION

MLR A significant linear relationship was observed between ACT English scale score and college persistence, and between SAT Math scale score and college persistence.

There was a significant but reverse linear relationship between PSAT English scale score and college persistence.

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

15 DISCUSSION

Implications of findings • No test is clearly superior above others as predictor of college persistence.

• Lack of a single predictive test for both English and math in terms of college persistence.

16

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

Conclusions Since all tests were found equally not sufficiently predictive, best approach post-affirmative action is to: 1. Eliminate all but the one test that is

DISCUSSION

(a) most equitably accessible to all demographic subgroups, (b) most aligned to state’s local economy and workforce demand,

(c) least instructionally disruptive, (d) least costly.

2. Work further on the design of the selected test.

17

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

Conclusions California Smarter Balanced is best choice to streamline and re-align standardized testing:

DISCUSSION

▪ CAASPP assesses >95% of high school juniors, without cost to students and families.

▪ Interim Assessment system and data dashboard support instructional programs.

▪ Tied to state priorities and state academic standards.

▪ Tied to state community college and university systems.

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

Recommendation 1 Expand research: 18 DISCUSSION

Longitudinal study disaggregated by demographic subgroups.

Survey students’ attitude toward state testing.

Address both College-bound and Vocational/ Trade School tracks.

Examine college persistence as measure of higher learning success.

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

Recommendation 1I Upgrade test design:

19 DISCUSSION

Focus on Career Readiness Authentic Assessment Lifelong Learning

No Opt Outs

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION TESTS AS PREDICTORS OF COLLEGE PERSISTENCE

Student wellness

20 DISCUSSION IMPACT OF STUDY

Improved test design

Instructional time savings

Cost savings

CALIFORNIA SMARTER BALANCED ASSESSMENTS AND COLLEGE ADMISSION EXAMS AS PREDICTORS OF COLLEGE PERSISTENCE

21 AUTHENTIC ASSESSMENT

MICROCREDENTIALS

DISTRIBUTED LEDGER (DLT) SCORE REPORTING

REPORT CARD TOKENS & DIGITAL WALLET

MENTORING APP

DISCUSSION FUTURE APPLICATIONS