Pieces of the Puzzle
Part 2. Potential Factors Behind Subscale Math Trends To help us further understand the math results, we explored two hypotheses concerning student NAEP math performance overall and at the subscale levels. First, we examined the alignment of state and/or district math standards with the NAEP math specifications by subscale. Second, the research team conducted site visits to the four selected districts to see what they were doing instructionally that would help explain the NAEP math scale scores. The methodology for both parts of this chapter is described in chapter 3 and in appendices C and D. Alignment of State and District Standards to NAEP Mathematics Specifications The purpose of this part of the analysis was to determine how well each state’s or district’s math content standards were aligned with the NAEP specifications and to see if there was any connection between the degree of alignment and how well a district did on NAEP. This work was done using the math specifications found in (1) the Mathematics Framework for the 2005 National Assessment of Educational Progress published by National Assessment Governing Board,15 (2) the state math standards, and (3) in the case of Boston and Cleveland, the district math standards in place during the 2006–2007 school year. Degree of Content Match Fourth-grade Mathematics Our analysis on grade four math showed that between 66 percent and 72 percent of NAEP specifications were either completely or partially matched by the local/state standards in the four jurisdictions. The highest overall matches appeared to be in Boston. These results are shown in table 4b.12 and figure 4b.3. The details follow in the bullets below. (Districts in bold are the main comparison districts in math.) There were 65 NAEP specifications in fourth-grade math. All jurisdictions showed similar patterns of overall matching.
15
Atlanta/Georgia standards matched 44 (68 percent) of the 65 NAEP specifications with 25 complete and 19 partial matches. Therefore, some 38 percent of the 65 NAEP specifications were completely aligned with the Atlanta/Georgia standards.
Boston, which had slightly different standards than its state, matched 47 (72 percent) of the 65 NAEP specifications, with 25 complete and 22 partial matches. Therefore, 38 percent of the 65 NAEP specifications were completely aligned with the Boston standards. The state’s degree of complete match was 19 percentage points higher, at 57 percent.
Charlotte/North Carolina’s standards matched 43 (66 percent) of the 65 specifications, with 30 complete and 13 partial matches. Therefore, 46 percent of the 65 NAEP specifications were completely aligned with the Charlotte/North Carolina standards.
Cleveland standards matched 43 (66 percent) of the 65 NAEP specifications, with 26 complete and 17 partial matches. Therefore, some 40 percent of the 65 NAEP specifications
Available at http://www.nagb.org/publications/frameworks/m_framework_05/toc.html
100 Council of the Great City Schools and the American Institutes for Research
Council of the Great City Schools • American Institutes for Research • Fall 2011
119