4c. Science
SCIENCE
4c
Pieces of the Puzzle
Part 1. District Performance on NAEP Science Subscales
4c. Science Content
The framework and specifications used to guide NAEP’s 2005 science assessments included content from Part 1. District Performance on NAEP Science Subscales three broad fields of science: earth science, physical science, and life science. The NAEP specifications present the science content in outlines that include grade-level objectives (at grades four, eight, and 12) along with ideas for exercises aligned with these objectives. Table 4c.1 shows how the item pool is Content distributed across the three fields in science (approximated as the proportion of the total amount of time that would be required if the entire pool were administered to a single individual). The framework and specifications used to guide NAEP’s 2005 science assessments included content from three fields of science: science, physical andlevel, life science. Tablebroad 4c.1 Percentage of itemsearth by science content areascience, and grade 2005 The NAEP specifications present the science content in outlines that include grade-level objectives (at grades four, eight, and 12) Field of Science Gradeobjectives. 4 Gradethe 8 item pool is along with ideas for exercises aligned with these Table 4c.1 shows how Earth Science 33% 30% distributed across the three fields in science (approximated as the proportion of the total amount of time Physical 33% 30% that wouldScience be required if the entire pool were administered to a single individual). Life Science
33%
40%
Table 4c.1 Percentage of items by science content area and grade level, 2005 In 2005, the grade four science assessment consisted of 157 specifications across three science subscales, Field eight of Science 4 8 and the grade assessment consisted of 222Grade specifications across the same three Grade subscales. In addition 30% structure that toEarth the Science specifications across fields of science,33% NAEP employs a cognitive-demand Physical Science 33%items. The assessments at both grade 30% levels contain incorporates low-, moderate-, and high-complexity Life Science 33% multiple-choice, short constructed-response, and extended constructed-response items.40% In 2005, the grade four science assessment consisted of 157 specifications three science subscales, Also included are performance exercises that allow students to manipulateacross physical objects and to draw and the grade eight assessment consisted of 222 specifications across the same three subscales. In addition scientific understandings from those manipulations. The full set of specifications that governed the NAEP to the assessments specificationsin 2005 acrosscanfields of science, employs cognitive-demand that science be found in ScienceNAEP Assessment and aExercise Specificationsstructure for the 1994 16 incorporates low-, moderate-, and high-complexity items. The assessments at both grade levels contain National Assessment of Educational Progress. multiple-choice, short constructed-response, and extended constructed-response items.
Composite, Subscale, and Item analyses – Strengths and Weaknesses in Science
Also included are performance exercises that allow students to manipulate physical objects and to draw scientific understandings fromand those manipulations. The full set of specifications that of governed NAEP In this section, we compare contrast the academic strengths and weaknesses the fourtheselected science in three 2005 science can be found Assessment and Exercise Specifications for theNAEP 1994 districts assessments in each of the fields in orScience subscales. As described in chapter 3 (Methodology), 16 National Educational Progress. subscalesAssessment are not all ofreported on the same metric, so the average subscale scores or gains in average subscale scores are not directly comparable from one subscale to another. So in order to estimate district Composite, Subscale, Itemsubscale analyses – Strengths and Weaknesses strengths and weaknesses, weand examine and item-level performance in two ways. in Science In thiswesection, andrankings contrastfor theeach academic and weaknesses of the four First, providewethecompare percentile selectedstrengths TUDA district on the distribution of selected average districts in each of the three science fields or subscales. As described in chapter 3 (Methodology), NAEP subscale scores on the national public school sample, after adjusting for student background subscales are not all reported on the same metric, so the average subscale scores or gains in average characteristics. Second, we provide item-level information about omission rates and the percentage of subscaleitems scores directly comparable from one subscale to another. So in order to estimate district correct by are itemnot type for each TUDA district in the three fields of science. strengths and weaknesses, we examine subscale and item-level performance in two ways. First, we provide the percentile rankings for each selected TUDA district on the distribution of average subscale scores on the national public school sample, after adjusting for student background 16 characteristics. Second, provide Governing item-levelBoard. information about omission rates and the percentage of Published by the Nationalwe Assessment correct items by item type for each TUDA district in the three fields of science. American Institutes for Research 119 Council of the Great City Schools and the
16
Published by the National Assessment Governing Board.
119 Council of the Great City Schools and the American Institutes for Research Council of the Great City Schools • American Institutes for Research • Fall 2011
137