Pieces of the Puzzle Decision Process: Step 1: For grade 4, examine the NAEP content objectives in column A and the state standards in columns C and D. For grade 8, examine the NAEP content objectives in column A and the state standards in column C. For every row: -
If the state column cell is empty, then code N for not matched. If the state column cells for grades 3 and 4 or for grades 7 and 8 contain an entry, then code P for partial or C for complete on the basis of the following decision rule: If the NAEP and the state content address the same mathematical expectations such that a reasonable person with reasonably strong content knowledge would say that the entries refer to essentially the same content and skills, then code C; otherwise, code P.
Step 2: For grade 4, examine the NAEP content objectives in column A and the district standards in columns H and I. For grade 8, examine the NAEP content objectives in column A and the district standards in column G. For every row: -
If the district column cell is empty, then code N for not matched. If the district column cells for grades 3 and 4 or for grades 7 and 8 contain an entry, then code P for partial or C for complete on the basis of the following decision rule: If the NAEP and the district content address the same mathematical expectations such that a reasonable person with reasonably strong content knowledge would say that the entries refer to essentially the same content and skills, then code C; otherwise, code P.
Assigning Cognitive Demand Codes Purpose: These codes refer to the cognitive demands of the NAEP content objectives and the strategies or skills implicitly or explicitly referenced in the state and district standards. Working independently, the content lead and two coders applied the cognitive demand coding guidelines to assign a cognitive demand code. Codes: For each content area, two coders and the content lead independently assigned one of the following codes.
Code L (for low): The task the student must perform is relatively simple for a student with gradelevel skills and appropriate background knowledge and experiences. The task places low cognitive demand on the student. Examples include recalling factual information, locating a piece of text that explicitly states the answer to a question, or performing a simple task. Code M (for medium): The task requires some cognitive engagement and mental processing beyond recalling or reproducing information. The task may involve making some decisions about solving a problem or drawing inferences by looking across several sections of text. Code H (for high): The task requires a greater depth of cognitive processing, including planning, using evidence, and applying demanding cognitive reasoning. The task may involve justifying a response, explaining the procedures followed, substantiating one’s thinking, and thinking abstractly.
D-22
Council of the Great City Schools and the American Institutes for Research Council of the Great City Schools • American Institutes for Research • Fall 2011
285