Evaluating and Improving a System of Evaluation

District Leader’s Guide

Evaluating and Improving a System of Evaluation

Continuously improving the system through feedback, data collection and analysis is critically important. Š Copyright 2012 Colorado Legacy Foundation. All rights reserved.

District Leader’s Guide

Evaluating and Improving a System of Evaluation Implementing Evaluation Systems: Learning from Pioneering Districts In an effort to help districts implement local educator evaluation systems in line with the requirements of SB10-191, the Colorado Legacy Foundation is providing a suite of resources to district and school leaders. CLF worked with three Colorado districts who recently revised their evaluation process. Although these districts implemented their new systems prior to passage of SB191, the process that each district went through is informative. Highlighting these districts and the lessons they learned along the way will help other districts leverage their success and avoid re-living their most difficult challenges.

Three case studies provide the foundation for this work. District and school leaders can read the case studies to learn how different districts have approached similar goals. A series of District Leader’s Guides builds on the case studies and provides more direct and specific guidance to district leaders as they move forward with implementation.

1

Brighton Case Study - Brighton educators and administration agreed that their evaluation system was a “dog and pony show.” In 2009 they revised their system, with terrific buy-in from the union, to more meaningfully support teachers.

2

Eagle Case Study - Eagle has spent nearly a decade developing their evaluation system and aligning it to instruction, assessment, and professional development.

3

Harrison Case Study - In 2007, Harrison hired a new superintendent who instituted a new evaluation system along with rigorous instructional supports, interim assessments, and a pay-for-performance system.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Who should use this Guide?

This guide is developed for district leaders who: • Have read Preparing Your District to Implement a New Educator Evaluation System, Communicating Effectively with Stakeholders, Selecting and Using Multiple Measures to Evaluate Educators, Determining Levels of Performance, Training and Selecting Evaluators, Ensuring Data Integrity, Transparency, and Using Evaluation Results.

• Have established a district evaluation implementation team which is looking for examples, lessons learned and implementation tips from Colorado school districts; • Are prepared to explore, with the district implementation team and technology professionals in the district, the data infrastructure needed to effectively evaluate educators and provide them with targeted support.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Evaluating and Improving a System of Evaluation

Introduction

The State Council for Educator Effectiveness (SCEE) set out five key priorities for district and state leaders in developing evaluation systems. One key priority is that the implementation and assessment of any evaluation system must embody continuous improvement. In this spirit, districts should consider “systematically evaluating the performance of the evaluation system in terms of its goals and results, and modifying its structure, processes, or format to assure system efficacy and sustainability.” i The rules that govern SB10-191 embrace these recommendations and require the Department of Education to monitor school district and BOCES implementation of evaluation systems. Specifically, the Department will be focused on: 1. Increasing the effectiveness of all Educators.

2. Analyzing the correlation between student performance outcomes and the assignment of educators to Performance Evaluation Ratings.

3. Analyzing the equitable distribution of effective and highly effective educators.

4. Analyzing the extent to which Principals and Teachers understand how they are being evaluated, what they need to do to improve, and how to access resources they need to support their professional development.

The rules also propose how each of the above objectives may be evaluated and the data that may be used. More information from CDE on how districts will demonstrate compliance with the law will be available prior to July 2013, the date by which districts will need to decide if they are using their own system or adopting the state model system.

Districts may also want to consider measuring the following outcomes, as proposed by the State Council for Educator Effectiveness.

Student Achievement and Equity Outcomes: • Student outcomes are improving.

• Educators are increasing their effectiveness over time.

• Districts are retaining highly effective and effective educators at a greater rate than ineffective educators.

• Districts are increasing the number and percentage of highly effective and effective educators in high needs schools and subject areas over time.

• Districts are attracting, developing and retaining more highly effective educators over time.

• Effectiveness ratings of teachers and principals are consistent with overall student and school performance.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Introduction (cont.)

Educator Support Outcomes • Districts are able to provide more individualized and useful feedback to educators over time.

• Districts are using educator evaluations to facilitate meaningful, growth-producing dialogue with and among educators.

• Educators are reporting that the process is professionally meaningful and assists them in the improvement of their practice.

• Education professionals collaborate around improving student outcomes.

• Educators are using data and feedback to improve teaching and learning through an ongoing process of planning, assessment and improvement.

• The pre-service training and ongoing professional development of educators are aligned with each other and with the quality standards that inform evaluations of educator performance.

• Districts are experiencing greater alignment between district, school and teacher goals.

• The quality and quantity of valid and reliable measures of educator and student performance improve for all subjects and areas.

• Districts, schools and teachers have more access to examples of best practices, including exemplars of instructional practice and student work over time.

• Resources are being used to continuously improve the value of the evaluation system.

Educators are reporting that the process is professionally meaningful and assists them in the improvement of their practice.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Guiding Questions

Questions for the district implementation team are:

What One Pioneering District Did

• How will the district put in place a monitoring and reporting system to collect data on the outcomes listed above?

An External Evaluation of the Teacher Evaluation System – Brighton 27J

• What should be done if educators are not increasing their effectiveness over time?

• What should be done if districts are not retaining highly effective teachers?

• To what extent are educator effectiveness ratings correlated with student achievement results? What will district and school leaders do when they are not correlated?

• How often will the district administer a climate survey to collect teacher perception data on the evaluation system?

• How are educators using data and feedback to improve teaching and learning through an ongoing process of planning, professional learning, assessment and improvement?

• How can the district establish explicit benchmarks, targets or indicators to determine the effectiveness of the evaluation system?

• Are effective teachers equally distributed across schools within the district? If not, what action should district leaders take?

Brighton 27J used a Title IIA teacher recruitment and retention grant to hire a highly regarded Denver consulting firm to evaluate their new teacher evaluation system. According to the evaluation report, the new evaluation presents a significant potential shift in policy and philosophy with regard to the role of school principals as instructional leaders, and the role of the evaluation process in not only assessing teacher talent, but in developing teachers as professionals.

The evaluation sought to capture principals’ and teachers’ initial perceptions at the start of school year 2009-10 so the district would be in a better position to make changes along the way and to target specific areas of the new evaluation process as it was rolled out in the first year. The study was designed to help the district learn from the pilot year and make any needed changes or improvements to keep the program moving forward to achieve its desired outcomes.

Overall, principals and teachers in Brighton like many of the new elements of the evaluation system, specifically the emphasis on coaching and feedback on teacher instructional practices.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Implementation Tips

Pioneering districts suggested the following implementation tips: • Be open to – and seek—feedback on the evaluation system, especially from teachers and principals.

Make adjustments to the evaluation system based on constructive feedback.

• Focus on continuous improvement. Every year there should be adaptive changes to the evaluation system based on feedback and other data. Always ask: How is the evaluation system meeting its goals?

• Track student outcomes. Are they improving? If not, what other data can inform this? It can take a year or more for teachers to improve their instructional practices, which means student achievement gains are not instantly identifiable.

• Evaluate district programs as well as the evaluation system. District leaders should identify what programs are working for what students and what programs aren’t working. How can principals and teachers be held accountable for student achievement when district programs aren’t working for their students?

• Track new hires, retention and promotion. What is the rate of promotion and retention? What is the rating of teachers that are leaving the system? The teachers that are staying? The novice teachers? What is the district’s retention rate?

• Identify what professional learning opportunities are supporting teachers’ instructional skills. Frequently solicit teachers’ feedback on what helps them be better teachers.

• Administer a “climate survey.” Ask teachers and principals their opinions of the evaluation system.

Report these results. Act on them where appropriate.

• Promote collaboration. Even in high stakes evaluation systems, teachers want to share their practices and learn from other teachers.

• Hire an external evaluator. Because they are anonymous and external to the district, external evaluators can collect authentic data and report it to district leadership.

• Ensure principals are instructional leaders. All three pioneering districts had a high priority on improving principals’ skills as instructional leaders. Principals are not only the evaluators of teachers, they are also coaches and managers of instruction. Principals must therefore use the quality standards for teachers and evaluation results to improve their schools.

• Identify what high performing teachers in the district do. Share these practices across the district.

• Identify more measures that can be used to evaluate teachers and students. Pioneering districts were either searching for new measures or developing their own — or doing both.

© Copyright 2012 Colorado Legacy Foundation. All rights reserved.

Conclusion

Continuously improving the system through feedback, data collection and analysis is critically important. No system will be perfect, but weighing different options for different purposes and outcomes should drive improvements to the evaluation system. Collecting information on different aspects of the system will provide a broad view of how well it is working and how different aspects can be aligned for both efficiency and productivity.

Acknowledgements Author: Jane Armstrong, JM Armstrong and Associates Research Support and Editing: Heather Chikoore, Colorado Legacy Foundation Research Support for the Case Studies: Ulcca Hansen, Colorado Legacy Foundation and Kristen Davidson, University of Colorado at Boulder Central office staff, school board members, principals, teachers, parents and community members participated in interviews that informed the development of this guide. Brighton, Eagle and Harrison School Districts opened their doors to researchers to describe what they were doing, how they were doing it, the challenges they faced and lessons they learned. Interviewees were both gracious and candid in their interviews. Without them, this knowledge could not be captured and shared. i Laura Goe, Lynn Holdhelde and Tricia Miller. A Practical Guide to Designing Comprehensive Teacher Evaluation Systems. (Washington, DC: National Comprehensive Center for Teacher Quality). May 2011.

Š Copyright 2012 Colorado Legacy Foundation. All rights reserved.