Understanding the SMARTER Balanced Assessment Design: ELA Specifications and Claims
1
Learning Targets for Assessment Design I can explain the basic design principles of the SMARTER Balanced Assessment framework. • Evidence-Based Design • Depths of Knowledge • Learning Goals organized by Claims • I can learn about how SMARTER Balanced proposes to assess student learning for each of the five claims through a jigsaw activity.
2
Curriculum-Instruction-Assessment Connections What should students know and to be ale to do? What should students learn? What should students be taught?
Curriculum
Students Norming (5 min.)
Instruction
Assessment What have students learned? What haven’t students learned?
What are student being taught? How are students being taught?
3
Evidence Based Design Framework Observation A set of specifications for assessment tasks that will elicit illuminating responses from students
Interpretation
“Assessment Triangle�
Cognition Beliefs about how humans represent information and develop competence in a particular academic domain
4
The methods and analytic tools used to make sense of and reason from the assessment observations/evidence
Think-Pair-Share
How do you think this model is different from how the CST measures student learning?
5
Models of Cognition Evaluation Synthesis Analysis Application Knowledge Comprehension Knowledge 6
Shortcomings of Generic Taxonomies • Capture a single point in time; do not look at a developmental process or learning progressions. • A one-size-fits-all model for all learning domains that doesn’t take into consideration what varies by content. • A focus on the verb and not the context of learning: the content or the task.
7
Webb’s Depth of Knowledge • A different way of looking at the cognition point of the “Assessment Triangle” • Adapted from a model developed by Norman Webb from the University of Wisconsin to align assessments to standards
8
Depths of Knowledge (DOK)
Webb’s Depth of Knowledge levels: Recall and Reproduction: Level 1 Basic Application of Skills & Concepts:
Level 2
Strategic Thinking:
Level 3
Extended Thinking:
Level 4
9
BLOOM’S REVISED TAXONOMY
WEBB’S DEPTH OF KNOWLEDGE
REMEMBER Retrieving relevant knowledge from long-term memory (e.g., recognizing, recalling) UNDERSTAND Determining the meaning of instructional messages, including oral, written, and graphic communication (e.g., interpreting, exemplifying, classifying, summarizing, inferring, comparing, explaining)
APPLYING Carrying out or using a procedure in a given situation (e.g., executing, implementing)
ANALYZING Breaking material into its constituent parts and detecting how the parts relate to one another and to an overall structure on purpose (e.g., differentiating, organizing, attributing) EVALUATE Making judgments based on criteria and standards (e.g., checking, critiquing)
CREATING Putting elements together to form a novel, coherent whole or make an original product (e.g., generating, planning, producing)
Recall – Recall of a fact, information, or procedure (e.g., What are the Red Cross Emergency Action steps [check, call, care]?)
Basic Application of Skill/Concept – Use of information, conceptual knowledge, procedures, two or more steps, etc. (e.g., Given a emergency scenario, students determine the care needed for a victim, and explain the reason for their actions).
Strategic Thinking – Requires reasoning, developing a plan or sequence of steps; has some complexity; more than one possible answer; generally takes less than 10 minutes to do (e.g., Module 363 –ER – Stressed due to parents’ divorce; Crunched for time; Signs of stress – ways to relieve stress – why managing stress is important to health.)
Extended Thinking – Requires an investigation; time to think and process multiple conditions of the problem or task; and more than 10 minutes to do non-routine manipulations (e.g., Task 608 – Welcome 10 to Health High – Create fact sheet/brochure from research activity) 10
Cognitive Rigor Matrix Depth + thinking
Level 1 Recall & Reproduction
Remember
- Recall, locate basic facts, details, events
Understand
Level 2 Skills & Concepts
Level 3
- Select appropriate words to use when intended meaning is clearly evident
- Specify, explain relationships - summarize – identify main ideas
- Explain, generalize, or connect ideas using supporting evidence (quote, example…)
- Explain how concepts or ideas specifically relate to other content domains or concepts
Apply
- Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning
– Use context to identify meaning of word - Obtain and interpret information using text features
- Use concepts to solve non-routine problems
- Devise an approach among many alternatives to research a novel problem
Analyze
- Identify whether information is contained in a graph, table, etc.
– Compare literary elements, terms, facts, events – analyze format, organization, & text structures
- Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text
– Analyze multiple sources - Analyze complex/abstract themes
– Cite evidence and develop a logical argument for conjectures
- Evaluate relevancy, accuracy, & completeness of information
- Synthesize information within one source or text
- Synthesize information across multiple sources or texts
Evaluate Create
- Brainstorm ideas about a topic
©2009 Karin K. Hess
- Generate conjectures based on observations or prior knowledge
Strategic Thinking/ Reasoning
Level 4 Extended Thinking
Depths of Knowledge (DOK) • Describes the assessment task, not the student • Not the same as difficulty, focuses on the level of cognition (thinking) required by a task • The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required 12
Depths of Knowledge (DOK) • DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (requires deep understanding of rock cycle and a determination of how best to represent it)
• DOK 2- Describe the difference between metamorphic and igneous rocks. (requires cognitive processing to determine the differences in the two rock types)
• DOK 1- Describe three characteristics of metamorphic rocks. (simple recall) Same verb, three levels of DOK 13
Checking for Understanding • At your table, think of a skill or concept that you might teach to students. • What would you ask students to do to reach each of the four levels of Depths of Knowledge?
14
Depths of Knowledge (DOK) SMARTER Balanced assesses at a variety of
DOK levels so we have opportunities to understand what students do & don’t know, and where they are in the novice- expert spectrum
Performance assessments can offer varying
levels of DOK embedded in a larger, more complex task
Planned formative assessments intentionally
focus on lower DOK levels so we can see where the student is in relation to the standard 15
Evidence Based Design Framework Observation A set of specifications for assessment tasks that will elicit illuminating responses from students
Interpretation
“Assessment Triangle�
Cognition Beliefs about how humans represent information and develop competence in a particular academic domain
16
The methods and analytic tools used to make sense of and reason from the assessment observations/evidence
Claims
• SBAC organizes the standards into 5 “claims”. These claims… • Frame a manageable number of learning goals around which instruction can be organized. • Guide the specification of appropriate evidence. • Provides a basis for meaningful reporting to different interested audiences. 17
Summary of Claims & ELA and Literacy CCSS Claim #1 – Students can read closely & critically to comprehend a range of increasingly complex literary and informational texts. •Literature: RL 1-3, 5-7, & 9 •Informational text: RI 1-3, 5-9 •Read/ Literacy: 1-3, 5-9 (gr 6–11)
Claim #2 – Students can produce effective writing for a range of purposes and audience. •Narrative (W1), Informational (W2); Opinion / Argument (W3) •Writing Literacy 1-2 (gr. 6-11) •Plan / Organize / Edit (W4); Language – Edit (L1, L2) •Writing – Gather evidence (W8) and Draw evidence (W0) (gr 4-11) 18
Summary of Claims & ELA and Literacy CCSS Claim #3 – Students can employ effective speaking and listening skills for a range of purposes and audiences. •Listening (SL – 1d, 2,3) •Speaking (SL – 4,5,6)
Claim #4 – Students can engage appropriately in collaborative and independent inquiry to investigate / research topics, pose questions, and gather and present information. •Writing (W7, W8) & draw evidence: W9 (gr 4-11) •Reading Literacy: 1-3, 5-9 (gr 6-11) •Writing Literacy: 1-2 (gr 6-11) 19
Summary of Claims & ELA and Literacy CCSS
Claim #5 – Students can use oral and written language skillfully across range of literacy tasks. •Read Literature (RL4); Info (RI4); Read /Literacy: 4 (6-11) •Language Use: 3,4,5 •Speaking / Listening: LS1d,2,3 This claim is “bundled with the other claims” in the assessment design, i.e. it is not assessed in isolation but in the context of reading, writing, listening and speaking.
20
Content Specifications: Assessment Target Anatomy P. 23 of content specification
21
Sources Smarterbalanced.org Norman Webb’s Depth of Knowledge, www.education.ky.gov/users/jwyatt/.../DOK%2012-151.PPT Increasing Rigor by The Network of Dynamic Learning Communities, www.cfn107.org/.../cfn_cognitive_rigor_and_tweaking_tasks_2_5_1...
22
Jigsaw Your task: In small groups, you will learn about how one claim is mapped out in terms of how it is linked to the CCSS and probable methods of assessment. You will also look at sample tasks that serve as examples of how student learning will be assessed. Note: These tasks are existing and public assessments that serve as examples, and are not designed by SMARTER Balanced.
23
Jigsaw • Four groups (one for claims #1-4) • 30 minutes to look at and discuss • Summative Assessment Targets (look at one grade level) • Sample Assessment Tasks • Chart main points and report back on: • A summary of what is assessed for each claim • How they are assessed • Examples of sample assessment tasks at one or two grade levels • 1-3 Implications for our work • 5 minutes for each group to report back + 2 minutes questions 24
Jigsaw: Expert groups
Claim #1: Suki, Erin, Sasha, Leah Claim #2: Mike, Sarah, Stan, Ann Claim #3: Brett, Jody, Nancy, Shelly Claim #4: Alexis, Elizabeth, Nicole, Nabila
25
LUNCH!
Please return at 1:15
26
Jigsaw part II
Each group has 7 minutes to report back
27
Independent Reading Time
Choose from the list of reading resources posted on the wiki or from your own list of CCSS readings. You have ONE hour to find a comfortable space to read. Be prepared to share some of the highlights and key passages. 28
Closing
29