Embedded Formative Assessment_Sample Page

formative assessment second edition

DYLAN WILIAM

© 2024 Ingrove Press. All rights reserved.

EMBEDDED

Copyright © 2024 Ingrove Press and Solution Tree Press. All rights reserved.

Published in Australia under license by Ingrove Press.

PO Box 3160 Mentone East, Victoria 3194, Australia Phone: (03) 8686 9077 Website: www.ingrovepress.com Email: orders@ingrovegroup.com No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the copyright holder.

Code: SOT1549 ISBN: 9780645931549 Published and printed in Australia Ingrove Press (formerly Grift Education) For permission to reproduce material from this publication, please contact orders@ ingrovepress.com

© 2024 Ingrove Press. All rights reserved.

First published in 2018 by Solution Tree Press.

Acknowledgments

Mentioning individuals is always invidious, but I owe particular debts of gratitude to Sarah Sharkey and Jerry Hardy (Christopher Wren School) and Dick Gmiterek (North Westminster Community School). When I joined the University of London, I lucked out again, being mentored by two great thinkers about education—first, Margaret Brown, and later, Paul Black. As well as being extraordinary academics, they are two of the nicest people one could ever hope to meet, and their generosity to younger academics is a model I have tried to emulate.

© 2024 Ingrove Press. All rights reserved.

The first edition of this book was the result of a thirty-five-year journey that took me from public school teacher to teacher educator to researcher and university administrator. Given the length of this journey, you will not be surprised that I have a lot of people to thank. First, the students at Christopher Wren School and North Westminster Community School in London taught me most of what I know about learning. The research I have read and carried out since then has done little more than help me make sense of what I learned there. Second are the teachers I have worked with in those two schools and in the extended networks of teachers of which I have been a part. As someone who went straight from a bachelor’s degree to the classroom, with no training or preparation, I was fortunate to start teaching in one of London’s toughest schools—fortunate because every teacher in that school struggled and was willing to talk about his or her struggles. It is only subsequently that I have realized how lucky I was, because that openness is not found everywhere.

Special thanks must go to Greg Ashman (@Greg_Ashman) for permission to use the graphic in chapter 3 from his blog Filling the Pail. Thanks also are due to Maureen Wellbery, at the American International School of Vilnius in Lithuania, for the story about all-student response systems in chapter 4. I am also grateful to the editorial staff at Solution Tree, and particularly Lesley Bolton (and Alissa Voss for the second edition), for all the work they have done in getting my manuscript to the Ingrove Press • 9780645931549 • SOT1549

iii

iv

E M B E D D E D F O R M AT I V E A S S E S S M E N T

finished book you have before you. Finally, and most important, my greatest thanks go to my partner, Siobhán Leahy. We first met at a mathematics teachers’ conference almost forty years ago, and for thirty-seven years, she has been my greatest inspiration and greatest support and has kept me grounded in the realities of daily school life. That is why the book is dedicated to her.

© 2024 Ingrove Press. All rights reserved.

Ingrove Press • 9780645931549 • SOT1549

Table of Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 chapter 1

Why Educational Achievement Matters. . . . . . . . . . . . . . . . . . . . . . . . . 5 The Increasing Importance of Educational Achievement. . . . . . . . . . . . . . . . . . . . . . . . . . . 5 The Difficulties of Raising Student Achievement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Three Generations of School Effectiveness Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 The Impact of Teacher Quality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Ways to Increase Teacher Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 chapter 2

The Case for Formative Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 The Importance of Professional Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 The Origins of Formative Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Definitions of Formative Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Strategies of Formative Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Assessment: The Bridge Between Teaching and Learning. . . . . . . . . . . . . . . . . . . . . . . . . 52 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

© 2024 Ingrove Press. All rights reserved.

About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

chapter 3

Clarifying, Sharing, and Understanding Learning Intentions and Success Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 The Importance of Learning Intentions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Ingrove Press • 9780645931549 • SOT1549

v

E M B E D D E D F O R M AT I V E A S S E S S M E N T

vi

When to Use Learning Intentions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Issues in Constructing Learning Intentions and Success Criteria . . . . . . . . . . . . . . . . . 67 The Case for and Against Rubrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Practical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 chapter 4

Eliciting Evidence of Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

chapter 5

Providing Feedback That Moves Learning Forward . . . 123 The Quality of Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Feedback as a Recipe for Future Action. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Practical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 chapter 6

Activating Learners as Instructional Resources for One Another. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Effective Cooperative Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Practical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 chapter 7

Activating Learners as Owners of Their Own Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Student Self-Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Self-Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Self-Regulated Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Practical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Ingrove Press • 9780645931549 • SOT1549

© 2024 Ingrove Press. All rights reserved.

Finding Out What Students Know. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Discovering Where Students’ Ideas Come From. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Practical Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Table of Contents

vii

Epilogue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Appendix: List of Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 References and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

© 2024 Ingrove Press. All rights reserved.

Ingrove Press • 9780645931549 • SOT1549

About the Author

During his early years of teaching in inner-city classrooms, Dr. Wiliam focused on physics and mathematics. He later joined the faculty of Chelsea College, University of London, which later became part of King’s College London. Here, he worked on developing innovative assessment schemes in mathematics before accepting leadership of the King’s College Mathematics Education Program.

For three years, Dr. Wiliam served as the academic coordinator of the Consortium for Assessment and Testing in Schools, which developed a variety of assessments for the national curriculum of England and Wales. He then returned to King’s College to serve as dean of the School of Education before being promoted to assistant principal of the College.

© 2024 Ingrove Press. All rights reserved.

Dylan Wiliam, PhD, is a consultant who works with educators in North America, the United Kingdom, and many other countries to develop effective, research-based formative assessment practices. He is former deputy director of the Institute of Education at the University of London. From teaching in urban public schools to directing a large-scale testing program to serving in university administration, his professional path has led to a variety of positions at the forefront of education.

Dr. Wiliam was also, from 2003 to 2006, senior research director at the Educational Testing Service in Princeton, New Jersey.

In 1998, he coauthored a major review of research evidence on formative assessment with Paul Black, and he has worked with many teachers in the United Kingdom and the United States on developing formative assessment practices to support learning.

Ingrove Press • 9780645931549 • SOT1549

viii

ix

E M B E D D E D F O R M AT I V E A S S E S S M E N T

In addition to holding a doctor of education degree, Dr. Wiliam has numerous degrees in mathematics and mathematics education. You can keep up with what he is writing at www.dylanwiliam.net and at www.dylanwiliamcenter.com, and you can follow him on Twitter @dylanwiliam.

© 2024 Ingrove Press. All rights reserved.

Ingrove Press • 9780645931549 • SOT1549

Introduction

The book has two main purposes. The first is to provide simple, practical ideas about changes that every teacher can make in the classroom to develop his or her practice of teaching. The second is to provide the evidence that these changes will result in improved outcomes for learners.

In chapter 1, I show why educational achievement is so important and why raising educational achievement needs to be a priority. With higher levels of educational achievement, people are healthier, live longer, contribute more to society, and earn more money. For society, the benefits include reduced criminal justice costs, reduced health care costs, and increased economic growth.

I also outline briefly how previous attempts at reform—including changes to the structure of schooling, to the governance of schools, and to the curriculum, and an increased role for digital technology—have been largely ineffective. These reform efforts have failed to take into account three crucial points.

1.

The quality of teachers is the single most important factor in the education system.

2.

Teacher quality is highly variable.

3.

Teacher quality has a greater impact on some students than others.

© 2024 Ingrove Press. All rights reserved.

In 1984, I left my job teaching mathematics and science in an urban public school in London to join a research project at the University of London that was exploring the potential of formative assessment to improve student learning. Over a quarter of a century later, this book is the result of that journey.

In chapter 1, I also show that attempts at improving the quality of entrants into teaching and removing the least effective teachers will result in small effects that will take a generation to materialize. The fundamental argument of chapter 1, therefore, Ingrove Press • 9780645931549 • SOT1549

1

E M B E D D E D F O R M AT I V E A S S E S S M E N T

2

is that to secure our future economic prosperity, we need to help the teachers who are already serving in our schools improve.

In chapter 2, I explore some of the ways that teachers might develop, and estimate how big an impact these changes would have on student outcomes. I show that some popular initiatives, such as learning styles, have no discernible impact on student achievement at all, while others, such as increasing teacher content knowledge, do improve student learning but by much less than people generally assume.

Chapters 3, 4, 5, 6, and 7 deal in turn with each of the five key strategies of formative assessment.

1.

Clarifying, sharing, and understanding learning intentions and success criteria

2.

Eliciting evidence of learning

3.

Providing feedback that moves learning forward

4.

Activating learners as instructional resources for one another

5.

Activating learners as owners of their own learning

In each of these five chapters, I present a summary of the research evidence that shows the impact of the strategy, and I offer a number of practical techniques that teachers have used—some in the literature, and others in classrooms that I have observed—to incorporate the strategy into their regular classroom practice. Although there is a definite progression throughout the chapters, I have made each chapter as self-contained as possible.

In all, these five chapters describe over seventy practical techniques for classroom formative assessment. Most of these techniques are not new. What is new is the framework for formative assessment presented in chapter 2, which shows how these disparate techniques fit together, and the research evidence that shows that these techniques are powerful ways to increase student engagement and help teachers become more responsive to their students’ needs.

As I have encountered many hundreds of teachers over the course of my career investigating formative assessment, I have not been able to recall the specific names

Ingrove Press • 9780645931549 • SOT1549

© 2024 Ingrove Press. All rights reserved.

In the second part of chapter 2, I summarize the research on classroom formative assessment practices and show that these practices appear to have a much greater impact on educational achievement than most other reforms. Chapter 2 concludes by outlining what formative assessment is, and what it is not, and presents the five key strategies of formative assessment.

Introduction

3

of teachers and the dates on which I observed their classrooms in every instance in this book. Additionally, other observations occurred as part of research studies in which we granted the teachers anonymity for their contributions. As a result, many of the examples and techniques throughout this book are not cited directly, but rather presented through the lens of my personal observations.

Additionally in this second edition, I have obviously updated many of the research studies, particularly those related to the changing nature of the working environment. I have also updated the research evidence on the various ideas that have been proposed for improving schools—and while the evidence has changed, the conclusions have not. First, this updated evidence continues to indicate that most current methods of school improvement won’t help improve schools much. Second, it indicates that the power of classroom formative assessment to improve students’ learning is incredibly strong, with evidence from all over the world showing that not only does classroom formative assessment work, but it is manageable in ordinary classrooms, without extra resources. I hope that this book convinces every reader about the impact that formative assessment can have on student achievement and provides some guidance about how best to begin the difficult, challenging, but worthwhile task of applying research to practice.

Ingrove Press • 9780645931549 • SOT1549

© 2024 Ingrove Press. All rights reserved.

In preparing this second edition, I show how the strategies and techniques presented in this book are just as relevant in colleges and universities as they are in K–12 settings. While the five strategies are equally applicable to college-aged students (and, indeed, to adult learning), the ways in which they are implemented do need to be adapted, and in this second edition, I provide a number of examples of practical techniques for doing formative assessment in higher education, and especially in lecture settings.

c h a p t e r

1

Why Educational Achievement Matters

The Increasing Importance of Educational Achievement

© 2024 Ingrove Press. All rights reserved.

E

ducational achievement matters—more now than at any time in the past. It matters for individuals, and it matters for society. For individuals, higher levels of education mean higher earnings, better health, and increased life span.

For society, higher levels of education mean lower health care costs, lower criminal justice costs, and increased economic growth. In this chapter, we will explore why education and educational achievement are vital to the prosperity of every nation and why the vast majority of attempts by policymakers to improve the achievement of school students have failed. We will then discuss three generations of school effectiveness research, the impact of teacher quality, and research-proven ways to increase teacher quality.

Education has always been important, but it has never been as important as it is now. In 1979, the median salary of those with bachelor’s degrees was $30,000 higher than the median salary of those with a high school diploma or GED (in constant 2012 dollars). By 2012, the annual earnings gap had widened to over $58,000 (Autor, 2014).

Higher levels of education are also associated with better health; people with more education are less susceptible to a whole range of diseases, including cancer, and are less likely to have a significant period of disability toward the end of their lives (Jagger et al., 2007). No doubt this is partly due to lifestyle choices, such as Ingrove Press • 9780645931549 • SOT1549

4

E M B E D D E D F O R M AT I V E A S S E S S M E N T

5

smoking, but it is also due in part to the kinds of work that are available to those with limited education. According to the Organisation for Economic Co-operation and Development (OECD, 2010), approximately 75 percent of American adults who did not complete high school say they are in good health, compared with 95 percent of those with college degrees.

Educational achievement also matters for society. Henry Levin and his colleagues at Columbia University estimate that preventing one high school dropout produces a net benefit to society of $209,000 (Levin, Belfield, Muennig, & Rouse, 2007). The main components of this total are: •

$139,000 in extra taxes the individual would pay because he or she would be earning more money

•

$40,500 in reduced health care costs, partly because the individual would be healthier, as noted previously, but also partly because he or she would be more likely to get health benefits from an employer and, therefore, be less dependent on public assistance

•

$26,600 in reduced criminal justice costs (largely because the individual would be less likely to be incarcerated)

With higher levels of education, the U.S. economy will also grow faster. Using data from OECD’s Programme for International Student Assessment (PISA) and a variety of other sources, Eric Hanushek and Ludger Woessmann (2015) examine the impact of increased achievement on economic growth. They estimate that if educators could raise the scores of American fifteen-year-olds on the triennial PISA tests by twentyfive points—the improvement that Poland made over a period of ten years—then by 2095, the U.S. economy would be 30 percent larger than it would otherwise be.

And if American fifteen-year-olds could achieve a level of reading and mathematics that allowed them to participate effectively in modern society (defined by a score of 420 on PISA as compared with the international average of 500), the U.S. economy would grow by an extra $30 trillion (Hanushek & Woessmann, 2015).

Ingrove Press • 9780645931549 • SOT1549

© 2024 Ingrove Press. All rights reserved.

Perhaps more surprisingly, people with more education live longer. Between 1915 and 1939, at least thirty states changed their child labor laws and periods of compulsory schooling. As a result, a number of students were required to attend school for one more year than children in other states. By looking at the life spans of those who had been required to attend an extra year of school, Adriana Lleras-Muney (2005) estimates that each additional year of schooling adds 1.7 years to one’s life. These are high stakes indeed.

Why Educational Achievement Matters

One common reaction to such changes in the working world is the fear that we are going to run out of jobs. Many people believe that there are only a certain number of jobs to go around, and if some of these jobs are destroyed, then there will not be enough work for everyone. This is an incorrect, albeit common, belief. Despite the millions of jobs that were lost in manufacturing, according to the Bureau of Labor Statistics, there are more people working in the United States than at any time in its history—160 million as of May 2017 (U.S. Bureau of Labor Statistics, 2017).

Interestingly, many of the new jobs being created do not demand much in the way of educational qualifications. In 2013, the U.S. Bureau of Labor Statistics projected that between 2012 and 2022, the U.S. economy will create just over four million new jobs for those with college degrees, but it will also create three million that require some education beyond high school but not a college degree, four million that require only a high school diploma, and another four million jobs that do not even require a high school diploma (U.S. Bureau of Labor Statistics, 2013). As far as can be seen, there will be jobs for people whatever their level of education in the United States. It is therefore probably not accurate to say to young people that they need to get a good education to get a job, but it does seem that education will be important to getting a good job.

Ingrove Press • 9780645931549 • SOT1549

© 2024 Ingrove Press. All rights reserved.

The reason that higher levels of education are so valuable is because employers’ educational demands are increasing steadily, nowhere more so than in manufacturing.

According to the U.S. Bureau of Labor Statistics (2016), seventeen million Americans worked in manufacturing in 2000. Ten years later, the figure was less than twelve million. This means that in the first decade of the 21st century, the U.S. economy lost 2,700 manufacturing jobs every day. It is common to hear people say, “We don’t make stuff in America anymore,” but the sentiment is wrong. It turns out that more goods were manufactured in the United States in 2016 than at any other point in its history, surpassing the previous peak reached in 2008 (Federal Reserve Bank of St. Louis, n.d.). The United States makes more stuff than it has ever made before. It just doesn’t use so many humans to do it—and that’s a good thing. Across all U.S. manufacturing, between 2002 and 2015, manufacturing output per hour of labor went up by 47 percent (Levinson, 2016). Compared to the heyday of American manufacturing, the average American worker employed in manufacturing in 2016 is more than six times as productive per hour as a worker in 1950. The reason that American workers are so much more productive is because they can work with more sophisticated technology, but this means that modern workers need higher levels of skill. Almost half of the manufacturing jobs that did not require a high school diploma in 2000 were gone by 2015, while the number of manufacturing jobs that required at least a master’s degree rose by 32 percent (Levinson, 2016).

6