What Are the Critical Steps in the Analysis of Large-Scale Assessment Data?
What Are Some Basic Analyses to Determine Which Factors Affect Student Achievement? Various analyses can be performed to determine which factors affect student achievement. EXPLORING RELATIONSHIPS USING CORRELATIONS
A national large-scale assessment should inform discussions about how to improve learning outcomes. Analyses that find relationships between student achievement and specific factors covered in the background questionnaires can be particularly useful in this regard. For example, policy makers and other stakeholders may be interested in knowing whether there is a relationship between student attendance at school and mathematics achievement, or between time spent doing homework and literacy skills. Different relationships might suggest different policies to improve learning outcomes (Shiel and Cartwright 2015). Correlations are a common way to characterize the direction and strength of the relationship between test scores and other variables. For example, if we observe a strong positive correlation between school attendance and math scores, we may conclude that the more students attend school, the more likely they are to have better mathematics performance. In parallel, if we observe a negative correlation between school absenteeism and mathematics achievement, we may assume that the more days of absence from school, the more likely a student is to demonstrate poor performance in math. However, such correlations do not guarantee causation. For example, a high negative correlation between absenteeism and math scores does not necessarily mean that absenteeism causes students to have lower mathematics proficiency. The distinction between correlation and causation is discussed more in the next section. Correlations can also be used to understand relationships between student performance on different assessments of the same domain, competency, or skill. Table 6.2 shows correlations between Swedish students’ mathematics proficiency, as measured on the Trends in International Mathematics and Science Study (TIMSS) grade 8 assessment, and four alternative measures of mathematics performance: student mathematics proficiency scores on grades 6 and 9 national large-scale assessments and grades earned upon completing the grades 6 and 9 math curricula (Wiberg 2019). All correlation coefficients are positive, indicating that children demonstrating higher achievement on the TIMSS mathematics assessment also tend to perform well in school and are likely to earn higher scores on Sweden’s national tests. This is an expected result, given that there is a high degree of alignment, particularly between the grade 9 mathematics curriculum and what the TIMSS assessment measures. Results like these would be particularly relevant to a national assessment team interested in providing evidence of the validity of the national large-scale assessment in terms of being able to measure student performance in core areas of mathematics.
77
