OKLAHOMA SCHOOL REPORT CARD A-F RESTART: Considerations, Adjustments, and Bold Next Steps

By Lesa Rohrer, Executive Director of Data Literacy, Oklahoma State Department of Education

When my son was in eighth grade, he brought home his final report card and it was evident that his math grade had been altered. As you can imagine, this alteration stimulated much conversation and provided an opportunity for us to assess where my son was at a pivotal time in his education. A report card tells us what happened, but it cannot tell us why something happened. In my son’s case, his grade (the unaltered version) reflected his lack of responsibility to complete assignments—which may have been related to his engagement with the curriculum, methods of instruction, and/or configuration of classes. The report card itself led to deeper conversations.

Likewise, the Oklahoma School Report Cards found at oklaschools.com can provide important information across a variety of measures to stimulate conversations about school successes and areas in need of improvement.

The A-F indicators, last published in SY 2018–2019, are getting a restart for SY 2021–2022. School sites have had access to aggregate Report Card-related data through the Accountability Reporting application in Single Sign-On since November to assess where they are at this pivotal time.

Taking Stock of Where We Are

It will take time to fully understand the extent and impact of COVID disruptions on school performance. Data from a national survey found that, “When compared to 2018–2019, 35% of states reported lower statewide attendance rates in 2019–2020, and 44% reported lower rates in 2020–2021” as shown in the graphs (Pinsonneault, 2022). Additionally, researchers from Attendance Works suggest that rates of students who are chronically absent may have doubled nationwide (Chang, 2022). When students have regular attendance, positive conditions for learning are more likely to occur. For several years, real-time attendance data have been available in Oklahoma’s Accountability Reporting application in Single Sign-On. These data have assisted school leaders in identifying students who are at risk of becoming chronically absent—and losing vital instructional time—as such data allows for timely and targeted supports.

Aside from being required by state and federal law, data from SY 2021–2022 will play a critical role in understanding the pandemic’s impact as we restart accountability. However, caution will be needed because context matters: students have had different learning experiences. Because of this, local data (e.g., levels of engagement, opportunity to learn, mode of learning, access to gradelevel content, teacher professional development) should be connected when using report card data to support continuous improvement efforts. The OSDE regularly uses data to engage in self-reflective cycles of continuous improvement.

Minor Adjustments to School Accountability System

In response to pandemic related disruptions, most states reported that they are making changes to their accountability systems for 2021-22 (Pinsonnault, 2022). Prior to the pandemic, the OSDE was already engaging in cycles of self reflected continuous improvement. Through an examination of three year of data, conversations with stakeholders, and consultation with our State Technical Advisory Committee, OSDE has made minor adjustments to its school accountability system in three areas:

1. the calculation of the Academic Achievement indicator

2. the minimum n-size

3. the definition for Targeted Support and Improvement (TSI) school identification

The goal of these minor adjustments is to provide a clearer and more sensitive signal of student performance and school improvement. These minor adjustments are in effect for the SY 2021–2022 Oklahoma School Report Cards.

Equally weighting the Academic Achievement Indicator

To better reward schools who show improvement in supporting students in being ready for the next grade, course, or level, the Academic Achievement indicator is being equally weighted across two components: Improvement Toward Expectations (ITE) and Performance Level Snapshot (PLS) as shown in the figure.

School sites will continue to earn credit for students that meet or exceed their interim achievement targets by priority student group through the ITE component. The PLS component awards credit when students score at the basic, proficient, or advanced level on state summative tests in ELA, math, and science. Students scoring basic have foundational knowledge and skills while students scoring proficient or advanced are meeting the grade-level expectations outlined in our Oklahoma Academic Standards (OAS).

Expanding the PLS component to include basic and equally weighting the ITE and PLS components will provide a clearer understanding of how schools are accelerating the improvement of Priority Student Groups and will more clearly recognize the observed performance of schools on state summative assessments. In addition, because of COVID-related disruptions in SY 2019–2020 and 2020–2021, interim targets for each priority student group were reset using a baseline from available and appropriate data from spring 2018, 2019, and 2021 assessment administrations. Approved interim targets are published in Appendix B of our consolidated plan starting on page 8, which can be found at https://sde.ok.gov/essa.

Adjusting the Minimum N-Size

The OSDE is increasing the minimum number of students (n-size) necessary for a school’s indicator calculations from 10 to 25 students. Doing so will eliminate the volatility of differentiating schools with small student populations from year-to-year. Because the OSDE is also committed to including as many schools as possible in accountability calculations, we will continue to utilize the multiple-year model. The multi-year model pools data across the last three years for schools that do not meet the minimum n-size threshold (≥ 25 FAY students) in a single year as shown in the example below.

The increase in the minimum number of students to 25 and the use of this multi-year model maintains the integrity of the accountability system while also improving the consistency of identification and reporting efforts.

Adjusting School Identification

Lastly, to ensure that schools designated for Targeted Support and Improvement (TSI) have the support and time they need, the OSDE adjusted its definition to, “Those schools where any student subgroup within the school falls below the 10th percentile for that group, across three consecutive years.” This designation will serve as an “early warning” signal to schools. Every three years, the OSDE will then utilize this list of TSI schools to identify schools for “Additional Targeted Support and Improvement” (ATSI) status. A school is ATSI status when any student subgroup at the school scores at the 5th percentile or lower, among all schools in the state. Designations for TSI occur annually, and designations for ATSI occur every three years.

Bold Next Steps

Across the country, the COVID pandemic has caused disruptions to student learning. To better understand the impact and maintain the integrity of our accountability system, OSDE has made some minor adjustments to our accountability system. Therefore, a school’s overall grade and that for Academic Achievement for SY 2021–2022 should not be compared with those from prior years. However, SY 2021–2022 data can serve as a baseline to help us understand where we are so that we can plan bold next steps. As you review your data, consider your local context; ask questions about what is working, celebrate successes, identify where improvement can be made, and take those bold next steps. ■

If you have questions, please reach out to lesa.rohrer@sde.ok.gov

References:

Chang, H. (2022, September 27). Pandemic causes alarming increase in chronic absence and reveals need for better data. Attendance Works. https://www.attendanceworks.org/pandemic-causes-alarming-increase-in-chronic-absence-and-reveals-need-for-better-data/ Chang, H, Osher, D., Schanfield, M., Sundius, J., Bauer, L. (2019). Using chronic absence data to improve conditions for learning, Attendance Works and American Institutes for Research (AIR). https://www.attendanceworks.org/wp-content/uploads/2019/06/Attendance_Works_Using_ Chronic_Absence__091619.pdf

Pinsonneault, L. (2022, November 23). The pandemic has changed the accountability landscape. The center newsletter. https://www.nciea.org/ blog/the-pandemic-has-changed-the-accountability-landscape/

Pinsonneault, L. & Domaleski, C. (2022). School Accountability in Flux: Results from a National Survey [PowerPoint slides]. National Center https://www.nciea.org/wp-content/uploads/2022/11/Pinsonneault_Acct_Landscape_2022.pdf