Pioneering Districts: Harrison School District Two

OCTOBER 2011

Pioneering Districts

Harrison School District Two Implementing Educator Effectiveness Systems

1 6 6 0 L i n c o l n S t r e e t , S u i t e 2 7 2 0 , D e n v e r, C o l o r a d o 8 0 2 6 4 • t e l e p h o n e : 3 0 3 - 7 3 6 - 6 4 7 7 - • w w w. c o l e g a c y. o r g

© Copyright Colorado Legacy Foundation 2011. All rights reserved.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

2

Table of Contents Introduction

4

Setting the Stage for Reform

5

Basing Teacher Compensation on Effectiveness

6

Implementing the Effectiveness & Results (E&R) Plan

7

Teacher Evaluations

9

Teacher Evaluations ‒ The Teacher s Perspective

11

Principal Evaluations

11

Teacher and Principal Evaluation System ‒ The Principal s Perspective

13

Recruiting and Hiring Process

14

Attrition and Dismissal

17

District Instructional Program

18

Assessments and Data System

19

Budgets and Action Plans

21

Enabling Factors

22

Inhibiting Factors

24

Lessons Learned

24

Outcomes

25

Next Steps

26

Acknowledgements

27

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

3

Harrison Educator Effectiveness Case Study Introduction

District Demographics

Mike Miles, superintendent of Harrison District Two (HSD2), is calling all heroes, because he believes every teacher in his district can be one.

Says Miles, They are the ones who hold the success of our students in their hands. Teachers and principals are expected to lead the charge in improving instruction and creating greater opportunities for students. Student

Lo catio n HSD2 is located just south of Colorado Springs and north of the Fountain-Ft. Carson Army base.

Scho ol Stati st ic s HSD2 is home to 14 elementary schools, one K-8 school, three middle schools, two high schools, one alternative school, four charter schools and a home school.

Te ac her S tati st ic s

achievement is going up.

HSD2 employs 860 teachers.

Five years ago, the district was one of the lowest

St udent St atist ic s HSD2 educates 10,472 students.

performing districts in Colorado. The school

Race and Ethnicity

board was seeking a new superintendent. Prior

• Black: 20%

to being interviewed for the superintendent posi-

• Hispanic: 39%

tion, Miles did his homework on the district. He

• White: 30%

knew there were way too many disconnected

• American Indian, Pacific Islander, other: 11%

programs that weren t creating results. He came Free and Reduced Lunch Population

with a systemic vision to improve teaching and learning in the district: change the ways teach-

1999

2005

2011

ers and principals are evaluated while supporting their professional learning in instruction. The

80

board was looking for a superintendent who 60

would change the dynamics of the district by focusing on the needs of the students and not

40

the adults in the system ‒ and someone not afraid to make hard decisions.

20

Trained at West Point, Miles served as an Army ranger and later as a special assistant to the U.S. Ambassador to Russia. He was also the associate superintendent for curriculum in a neighboring district and participated in the national Broad C o l o r a d o L e g a c y F o u n d a t i o n!

0

% Students who qualify for free or reduced price lunch

In 2006 HSD2 was deemed one of the worst performing districts in Colorado̶both low achieving and under-performing.

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

4

The Harrison School Board uses

Superintendents program. He was used to making de-

a streamlined governance proc-

cisions and thinking about organizations systemically.

ess called coherent governance.

Said Miles, If you change one part of the system, there

Board policies have been re-

are implications for the rest of the system. Everything

duced from 300 to 29. The 29

has to be aligned to get the most benefit and outcomes.

policies cover governance culture, board/superintendent rela-

Setting the Stage for Reform

tionships, operational expecta-

When Miles took the reins in 2006, HSD2 was on aca-

tions and results. In general, the

demic watch. The board gave him wide latitude to im-

superintendent is held respon-

plement his vision.

sible for operations and the board is responsible for policies.

Miles first operational decision was reconstituting the collaborative decision-making team (CDMT) that required 100% consensus to make a decision. Few deci-

The CDMT with the superinten-

sions were made and certainly not quickly. The CDMT

dent governs the development

was trumping decisions that the school board should

of the Agreement of Trust and

have been making. Within 24 hours there was a new

Understanding (ATU). The

policy charge for the team and a chance to elect new

school board is responsible for

members. While the new team would need a 75% quo-

the approval of the ATU.

rum for decisions, it had an incentive to advance decisions that broke with the status quo since failure to do so would mean the implementation of new policies that

Miles and the Board established.

Miles first instructional decision was to require all teachers to open their classroom doors.

This was the beginning of creating a new district culture of collaboration, transparency and accountability. There was teacher push back and angst but Miles held to this first symbolic action. Teachers were also to write their lesson objective on the board every day to focus students attention on what they were learning.

He also jettisoned a program called docked days where teachers could call in, say they are not teaching that day and dock my salary.

Over 100 teachers took more than 25 docked

days. Miles found this practice unconscionable and said, No wonder achievement was low.

He added, It was crazy and bad for kids!

Miles next action was putting in place a set of district core beliefs. Says Miles, If you haven t identified your core beliefs, you are not going to be successful.

C o l o r a d o L e g a c y F o u n d a t i o n!

Miles developed these with

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

5

input from district staff. Within six weeks they were adopted by the board.

These are widely understood by all board members, district staff and teachers. The Harrison Core Beliefs are: • Our main purpose is to improve student academic achievement.

• Effective instruction makes the most difference in student academic performance.

Figure 1: The relationship of systemic factors in ensuring teacher effectiveness in

• There is no excuse for poor quality instruction.

HSD2

• With our help, at-risk students will achieve at the same rate as non-at risk students.

• Staff members must have a commitment to children and a commitment to the pursuit of excellence.

These values are deeply embedded in the thoughts and actions of staff. A teacher reiterated the values when she said:

It s our job to make sure all kids are successful and to believe that

all kids can learn. The only way to do this is to improve our instruction. We have to rely on outside views to do this and have our instruction critiqued through professional learning communities and our ongoing evaluations.

Basing Teacher Compensation on Effectiveness Harrison s Effectiveness and Results (E&R) plan is based on two premises: 1) that research has identified what effective teachers do and these actions and behaviors can be measured; and, 2) students of effective teachers achieve at higher levels and this, too can be measured.

Three years of data for the Harrison students show that the quality of instruction greatly determines the amount of student academic growth.

HSD2 had three main purposes in mind when creating the new evaluation system. These were to: • Raise student achievement • Provide effective signals to teachers to improve the quality of instruction

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

6

• Differentiate teacher evaluations in order to make better decisions in promotion, retention and professional development HSD2 leaders knew that to do this fairly and accurately, it would take continual observation of teachers and would include feedback, coaching and professional development. Principals would need to be instructional leaders and the evaluation system would have to align instructional feedback with academic priorities, spot observations, the evaluation instrument, resources and professional development.

Over the last several years, Miles has trained administrators, teachers and board members to be systemic thinkers. The diagram below has been part of the training and shows how the pieces of the system are connected. Says Miles, We started by doing a few things very well ‒ purposeful, aligned instruction, multiple response strategies and helping principals develop action plans.

I also trained principals in walk-throughs, how to evaluate good instruction, using lesson objectives and how to manage change.

You can t do one thing at a time if you want to trans-

form it̶but it can t be crazy either, said Miles. In his mind was a progression to help schools implement demonstrations of learning (DOLs), formative assessments, and effective instructional strategies. Scaffolding and differentiation would follow.

We also made sure there was an assistant principal in every school, said Miles. This was to help with the day to day operations to free up princiFigure 2: The nine HSD2 pay grades and related annual salary

pals and learn to evaluate and coach teachers.

Implementing the Effectiveness & Results (E&R) Plan In October, 2009 the Harrison School Board approved a plan to change the teacher salary schedule to a compensation system based on teacher performance and student academic

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

7

outcomes. At the end of five years, the Superintendent, School Board, CDMT, and teacher committee will assess whether the E&R initiative should be dropped or continued.

The E&R plan has nine levels or grades for teachers. Teachers move to the next grade if they meet the criteria for performance and student achievement results. Each succeeding level requires a higher degree of mastery and demonstrated results. For example, a first year novice teacher need only receive a satisfactory evaluation to advance to the next grade (Progressing I). On average, teachers have earned a $3,9771 increase under the E&R plan.

Advancement to all other grades requires the teacher's students to achieve at a certain level (detail on student achievement found in Teacher Evaluation section on pages 9-10). These achievement results become more rigorous as the teacher attains proficiency and then mastery over the art of teaching. Teachers in non-core subjects must also achieve progress monitoring targets or other performance measurements.

Compensation is significantly higher at each succeeding grade. Years of service play no role in the E&R compensation system. College or continuing education credits play only a small role. They are considered as evidence of "life-long learning," which is part of the criteria for becoming a "Proficient II" or higher-grade teacher. Base salaries for each grade may be adjusted for inflation by the board of education. Not counting first year teachers (who advance automatically), about 25% of teachers move to the next effectiveness level each year.

There are no stipends for additional duties. As a teacher moves up the pay scale, more is expected of them as part of their compensation. (These savings are invested in the pay-forperformance system.)

Teachers still achieve "non-probationary" status after three years of teaching. Probationary teachers who do not meet the district's standards will be non-renewed. Teachers who are not performing at the proficient level according to the district's evaluation rubric will receive targeted professional development. In some cases the teacher will undergo remediation and, if necessary, removal from the district.

Teachers received a summative performance evaluation in April 2010 and were placed in one of the nine levels in consultation with the teacher s principal and district administrators. The new salary system began in September 2010. Salaries remain the same until the teacher moves into the next level. It is possible for teachers to reach $48,000 a year after three years of teaching in the district instead of 12 years on the typical salary system.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

8

After initial placement, teachers can advance from one grade to the next without skipping a grade. However, there is no minimum number of years that a person must remain at a certain level. The district makes grade movement determinations twice a year. Teachers can also move down a level if they don t meet the performance criteria of their current grade two years in a row.

Currently, teachers receive at least one summative evaluation annually, a formal evaluation, and 4 spot evaluations each semester. Spot evaluations are relatively short drop-in observations by the principal who observes and comments on classroom instructional practices. Summative evaluations include teachers scores on the evaluation rubric and student achievement results.

Teacher Evaluations HSD2 used existing research to develop rubrics based on proven practices and strategies that have a positive impact on student achievement. Standards were developed with teacher and principal input to create a clear rubric. Teacher evaluation rubric categories are: • Preparation for instruction • Use of data to inform instruction • Delivers quality instruction • Interventions to meet diverse needs • Classroom environment • Leadership • Professional responsibilities Teachers are rated on each standard as unsatisfactory (0), progressing (1), proficient (2) or exemplary (3). HSD2 says they have recently mapped their teacher evaluation rubric with the

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

9

State Council on Educator Effectiveness state quality standards for teachers and there is good alignment.

The same criteria are used in spot observations, formal evaluations and summative evaluations.

Principals are in every classroom every week doing 10-15 minute spot observations, providing immediate feedback to teachers. Schools may develop their own feedback forms but they usually include the daily lesson objective, the demonstration of learning, check boxes related to 1) preparation for instruction, 2) delivers quality instruction and 3) classroom environment. There is space for giving tips,

things well done,

reflective questions and next

steps. If there are concerns about a teacher then that teacher receives more spot observations from the principal and AP. Observations are kept by teachers as well as in a data binder

At Harrison we took very large

in the principal s office. Spot observations are

leaps where others across the coun-

reviewed by the principal every week to identify

try were taking very small steps.

professional learning needs for individual teach-

-Principal, Harrison School District 2

ers or for the school, in general.

Student achievement data counts for half of a

teacher s evaluation. By combining the rubric evaluation score with an index score on student growth measures, teachers are placed on the E&R scale shown in Figure 2. After teachers reach proficiency level I and before being advanced to a higher level, they must receive an evaluation and distinguished review by central office staff to ensure consistency in evaluations across the district and to prevent inflation of evaluations.

The most recent data are weighted the most heavily. Achievement data (both proficiency and growth) could include (but are not limited to): • CSAP (Colorado state assessment, including both proficiency and growth measurements) • Common assessments and progress monitoring assessments • Curriculum based measurement (CBM) data • Timed student constructed response results for writing once a quarter The use of progress monitoring data to improve instruction is central to professional learning community (PLC) meetings and has been part of the district and school action plans since 2006.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

10

At the end of each year teachers and their evaluators set goals for their professional growth plan. These include the goal, plan of action, how it will be measured and then a rubric rating on goal achievement.

Teacher Evaluations ‒ The Teacher s Perspective We had a lot of say in the performance rubric from the beginning ‒ it was a great opportunity to share our ideas, said one teacher.

We even had input on the weight of each per-

formance category.

Every teacher gets a large notebook that explains the evaluation system. Called the Professional Educator Evaluation System, it includes the rubrics, feedback forms, improvement and remediation plan guidelines and forms, professional growth plans and goals and data collection and reflection tools.

Because they are unannounced, most teachers value the spot observations the most.

I get

very specific feedback and have an opportunity to follow-up with an administrator to get new instructional ideas.

Formal evaluations are also valued. These are scheduled and evaluators come in to observe an entire lesson plan. One teacher said, I get great ideas for how to improve different areas of my instruction. I also get a short concise report including strengths and areas for improvement.

Peer coaching is another opportunity for feedback in many of the HSD2 schools. Teachers can select the teachers they want to observe and the principal sets up a schedule so teachers can use their planning time for a peer observation. Observations are used to set up potential collaborations, see their students reading in other classrooms or see what content other grades are teaching.

Principal Evaluations First, HSD2 principals are held accountable for being instructional leaders and improving instruction. As one principal said, If we aren t instructional leaders we won t have jobs!

The principal and assistant principal performance rubric rates principals as Unsatisfactory (0), Progressing (1), Proficient (2), and Exemplary (3). The rubric performance dimensions are (and some of these are double weighted):

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

11

P R I N C I PA L P E R F O R M A N C E R U B R I C

SCORE (0-3)

Leadership • Establishes a shared vision of success • Leads change • Maximizes human potential • Demonstrates other leadership skills The Instructional Program • Maintains a comprehensive program of instruction • Improves the quality of instruction • Provides for assessment of instruction Staff Development • Develops staff • Facilitates individual growth • Hires quality staff Effective Management • Manages resources effectively • Time Management School Climate • Professional Responsibilities • Maintains positive relations with district and community • Grows professionally Principals receive a mid-year review and a summative review, often by the superintendent.

Evaluators ask how they can better support the principal and if the principal has specific areas of need. Although HSD2 rolled the teacher evaluation rubric out first, significant professional learning was being provided to principals through monthly principal s meetings and onsite coaching.

Superintendent Miles and central office staff provide enormous support to principals. First, they shield principals from distractions such as unnecessary phone calls or data requests C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

12

(now handled by the central office). Every school has an assistant principal to handle some of the school operations and learn instructional coaching strategies. Said one principal, Miles knows you can t insist on higher student achievement if you don t provide good instructional feedback to teachers.

To lead their schools in raising student achievement, principals and assistant principals receive almost 10 days of professional development per year. This includes principal academies, leadership retreats, and bi-monthly leadership team meetings.

Principals are also well-trained in curriculum alignment, walkthroughs and effective instructional feedback, engagement strategies, effective teacher evaluations, systems thinking, action planning, expanding leadership capacity and other areas related to instruction or organizational effectiveness. In many cases the superintendent conducts these trainings.

Teacher and Principal Evaluation System ‒ The Principal s Perspective You start with a very narrow focus on core values and make sure everything you do addresses them, said one principal. The new system was rolled out to principals by the district s core team. Principals had a chance to add to the performance standards.

My first year, there was a lot of staff fear and emotion in making this transformation to a new performance system. It was almost too much information coming to them at once.

Teachers were questioning if they could do it. These were difficult conversations because teachers were being held accountable.

This same principal emphasized the need to understand the rubrics. Both principals and teachers need to know what it looks like to perform at each level.

Principals are clear about their roles. Said one, I know I am the instructional leader. I am the sense maker. I know I need to be in classrooms every week talking to teachers and coaching them. I always tie my coaching back to the rubrics so teachers become more familiar with them. . . We know we have to help students achieve 1.3 years of growth if we are to close the achievement gap.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

13

Evaluation System Costs The percentage of General Fund used for teacher compensation is stable prior to and after implementation, fluctuating by less than 1% each year, but the average teacher salary has increased from $40,729 in 2009-10 to an anticipated $45,312 in 2012-2013.

It s not costing more money, we are using money differently, said Miles. Only about 25% of the teachers get a raise each year but across the board salaries are higher.

Savings come from: • Eliminating automatic yearly raises in the traditional salary schedule.

• Eliminating salary increases due to experience.

• Eliminating salary increases for education credits.

• Reducing stipends by almost 1/3.

Costs are redirected to paying teachers based on effectiveness and advancement through the new E&R Scale.

Costs for professional development and communications have increased during the initial start-up. The curriculum and assessment department has increased its staff to create the curriculum-based measures and to help teachers and principals learn how to track and use data.

Title II is used for instructional coordinators to work in the trenches with the teachers.

Proficient II and higher teachers may be transferred to schools that require more skilled teachers. "Master" teachers must first receive national board certification or spend one or two semesters in a Harrison-designated exchange program with a rural district.

There is no "quota" for the percentage of teachers who can be at each level.

HSD2 successfully sought external funding to develop the curriculum based assessments (see Assessments and Data System section on page 19).

Recruiting and Hiring Process Potential teachers and principals know exactly what district expectations are because they get this letter immediately from Superintendent Miles after their application is received:

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

14

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

15

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

16

The district screens applicants based on the district s core beliefs and their type and level of experience. Principals make the final hiring decision. Candidates are interviewed in person or over Skype™. Candidates must conduct a performance interview by teaching a lesson with students often part of the interview team. References are checked and, if hired, there is a new teacher orientation that familiarizes them with the E&R system. The district has a growing relationship with Teach for America.

Attrition and Dismissal Attrition runs about 20% in HSD2 but district leaders aren t worried because those leaving are not among the most proficient teachers. Also, teachers who are leaving the district have a greater number of years of experience than those that are staying. It s likely they can go to a district with a traditional pay scale and be paid more.

However, there is a concern about the third of novice teachers who leave at the end of their first year. Much is expected of new teachers beginning the first week of school. But Superintendent Miles says, We are really clear about our expectations up front. We know it is harder to work here than anywhere else.

A plan of improvement may be initiated at any time by an evaluator based on problems or concerns in performance from spot, informal or formal observations or lack of growth on professional growth goals. These plans include the specific problem, the desired level of performance, additional education or professional development needed and then evidence of improvement. At the end of a realistic time period for improvement, the teacher may continue with the improvement plan, be placed on a remediation plan, or meet the professional standards and return to the current evaluation cycle.

Remediation plans are developed when a teacher s performance is rated as unsatisfactory on the summative evaluation or after an improvement plan. The teacher is given a written notice of deficiencies and a list of resources and assistance available to improve performance in a reasonable amount of time. If successful, they return to the regular evaluation cycle. If not the district may reinstate the remediation plan or recommend dismissal. The district is open to providing a second evaluation by another evaluation team.

Because of the district s core belief that all students deserve good instruction, Superintendent Miles wanted to dismiss teachers who were depriving students of a good education.

Miles invited members of the teachers association to come observe the classrooms of struggling teachers. He challenged them, If this is a place you are willing to put your kid, then I will C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

17

keep this teacher another year. But otherwise don t say it is discrimination and that you are protecting them to raise the quality of the profession. The teacher association representatives don t even come and watch when I invite them.

In the beginning, the teachers association defended these teachers. But the evaluation process was deemed fair and the rubrics and student achievement measures as accurate. As the teachers association lost more cases in court, fewer challenges against dismissal were made.

District Instructional Program Action plans (school improvement plans) are created in June at the school, district and district department levels. They begin by reviewing the prior year s plan and what has been accomplished.

Data are used to create a current needs assessment. Then goals and priorities are set and tracked with indicators of success. Specific actions are identified to achieve the goals, along with the support the district will provide to help achieve them. Finally, evidence of systems thinking is part of the plan, as well as how staff development, budget and action plans are tightly aligned.

School plans must also be aligned with district goals. Every year, as goals are met the expectations are set higher. For example, if 70% of students meet literacy proficiency by the end of the year, then the goal may be set to 75% for the following year. Over the summer district leaders review the school plans, organize supports for schools, and provide school budgets.

In November and April the superintendent or an executive director meets with the principal to review the action plan, look at school climate data and observe how the principal is meeting instructional priorities.

Curriculum maps identify lesson objectives and units of study. An instructional calendar serves as a pacing guide so teachers know the timing of teaching the curriculum which is geared to months or quarters. Curricula are aligned to state standards although some curricula are in transition between the old and new state standards.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

18

Literacy is a high priority in the district. In some schools exploratory teachers participate in a morning literacy block and work with students on literacy, including writing. Social studies is embedded in the reading curriculum.

Assessments and Data System District-wide assessments are clearly identified and principals and teachers know when they will be administered. Four types of assessment formats are used, including multiple choice, short answer, short-constructed response and extended-constructed response (writing).

During the 2010-2011 school year, teachers were brought together for two weeks in the summer to create test bank questions. They were paid stipends from a grant, trained in item construction and then given curriculum maps and scope and sequence charts to create items. Some were given passages that would be on the reading assessment and teachers were asked to create questions that they weighted by the depth of knowledge required. Central office staff reviewed the test items and selected many for the curriculum based measures and common assessments. In the spring items were reviewed.

Common assessments are given in October and February, curriculum based measures are administered in December and May, the CSAP in March and elective CSAPs (called performance assessments in exploratory areas) are administered in April. These include art, music and PE. Elective CSAPs are scored by outside teams using rubrics. Some CBM questions come from a commercial publisher but after a large development effort of having HSD2 teachers write assessment items, the commercial publisher will no longer be used.

Teachers and district leaders develop rubrics for constructed response items and calibrate them across teachers. Rubrics are frequently refined based on feedback from scorers.

Some of these items require scripts that a teacher reads or they begin with a question that the student answers. For example, a first grade art assessment asked students to look at a piece of art and then describe how they would recreate it with water colors. Students painted the water color the next day which was also scored.

Items are scrubbed (a review that ensures item validity and reliability) by teachers and student results on questions are analyzed to make sure the question is asking what they think it is. A reliability check comes when the district-developed CBM measures are correlated to CSAP results̶a correlation of almost .8 in 2010 - which is strong.

District-developed or selected items are color-coded in relationship to their frequency on the state assessment. Red items are most frequently tested on CSAP, blue is next tested and C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

19

black is least tested on CSAP. This gives teachers the opportunity to focus more instruction on what is being assessed̶especially if students are missing those questions.

Teachers score constructed response items (but not their own students) and results go into the district management system within a few days. Results are linked to individual teachers evaluations in the management system.

In the first year of linking student test results to teacher evaluations, teachers were allowed to drop one of the tests and not count it in their evaluation. The first assessment establishes a baseline and then the next assessment can serve to establish growth. Scores are also reported in median growth percentiles for more challenged students. Other teachers may use proficiency scores.

H A R R I S O N S S T R AT E GY Create a college-going and career-ready culture (one option is paying for up to $5,000 for the first year of college) End social promotion in five years (in grades 3, 8, and 10) Engage parents and the community (to expand academic privilege, raise money for scholarships and expand volunteer opportunities) Recruit and retain the best and brightest teachers (expand recruitment, work with Teach for America, perfect performance reviews, hire a pool of permanent substitutes and hire a strong teacher reserve)

The board is considering raising graduation requirements.

Depending on the school, data can be posted in elaborate ways on the walls ‒ allowing visible tracking of student progress. Class averages or student numbers (no names) are posted.

Students write a data driven goal and then reflect on their scores. This helps students and teachers understand some of the affective issues that identify why students score better at certain times than others. Said one teacher, Kids know exactly what they re working on. Everyone is in alignment.

In one school a principal said she met every Thursday with grade level teachers and they would have a dialogue about data. In this school they administered DIBELS (a reading progress inventory), AIMS (mathematics) and CSAP. Every child had a magnet with their name on it so teachers can see immediately how they are doing on multiple measures. These data are used to move children into different flexible groups and alert teachers when to start the response to intervention (RTI) process. Interventions with students are starting and stopping all the time as needed.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

20

The principal gives teachers a chart on how their students are doing in meeting proficiency targets and how other teachers in the districts are doing.

Professional learning communities were started five years ago in every school. It s up to schools how they use the time but it is typically used to review student data and for the principal and AP to help teachers with new instructional strategies such as multiple engagement strategies, differentiation and demonstrations of learning. Superintendent Miles does a lot of training monthly with principals and they go back to their school and turn-key this training to teachers.

PLC time can come from early release days, late start days, planning before or after school, common planning time with grade level teams or professional development days. Schools can decide how to use their professional development time.

As an example, a recent initiative in the district is the use of demonstrations of learning (also sometimes called exit slips. ) They take 5-10 minutes to administer and test students understanding of the lesson just taught in class. Teachers look at them after-school and decide if they need to re-teach the lesson or change their lesson. Sometimes teachers bring DOLs to PLCs to seek ideas for new strategies to teach the lesson or adapt the DOL to become an item on the district s common assessments.

Budgets and Action Plans Miles brought a new budgeting and planning process to the district. Instead of adding or subtracting resources from the prior year s budget, he started from scratch̶reviewing how expenditures ‒ and the programs they supported-- impacted student achievement. Everything the district does is now evaluated on merit.

Efficiencies were also made in operations, for example, consolidating bus routes and using technology to share information with schools.

School action plans identify how each school will spend its budget and how expenditures will result in increased student achievement. Resources not aligned to the district action can be taken away during a mid-year budget review. Plans can also be changed during the year.

Because principals are responsible for increasing achievement in their schools, the district role is to support principals in achieving their action plan. Since principals are held accountable they have resources and support from the district office. They are protected from chasing down rabbit trails and their management tasks are limited by having an AP in every C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

21

school. Two thirds of principals time is expected to be spent on improving instruction and one-third on administrative tasks.

Enabling Factors Harrison School District Two has undergone transformative change in the past five years.

Here are the factors that helped them do it.

• Having an urgency to improve student achievement and hire a superintendent who had the vision and leadership to do this. Harrison was on academic watch when Superintendent Miles was hired. He proposed the clearest vision of system reform among the candidates to the school board when he was hired.

• Streamlining decision-making and making decisions quickly. The community s sense of urgency gave Superintendent Miles a lot of latitude in making decisions. The core beliefs were written and adopted in six weeks. The board streamlined its policies from 300 to 29 and eventually most members became strong supporters of the evaluation system. Some instructional products were adopted from other districts. Committees that were delaying important decisions were re-formed.

• Having core beliefs that everyone knew, believed in and acted upon̶and that guided the work of the district. They included raising student achievement, ensuring effective instruction, holding high expectations for at-risk students and requiring the commitment of staff to children and the pursuit of excellence. The letter to applicants for district teaching positions spells these out very clearly!

• Having a systemic approach to educator effectiveness and improving student achievement. The superintendent believed you couldn t change one part of the system without it having impact on other parts of the system. HSD2 began with leadership at the core but also included using data, teacher and principal evaluation, aligned professional learning, and curriculum aligned to standards.

• Having a focus on core mission and freeing up people to achieve it. Responsibilities in the district office were aligned to the district action plan. In some cases department roles were combined and in other cases split apart. Some previous roles were dropped. In all cases the stupid stuff was eliminated. Principals were protected from outside interferences and given an AP to handle many of the management responsibilities of the district so they could be in the classroom.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

22

• Having access to and responsiveness from district leadership. As the evaluation system was being rolled out, questions arose and were quickly dealt with by the district leaders.

Answers to questions were put into an FAQ newsletter and sent to schools, teachers and board members in the district. District leaders were very open to making changes suggested by teachers if it strengthened the system.

• Looking for heroes. Advertisements placed in Education Week for principals have a banner at the top that says Only Leaders and Heroes Need Apply.

2

• Providing extensive training, supervision and professional learning opportunities to district staff. The superintendent provided training in systems thinking, action plans, walkthroughs, curriculum alignment, aligned instruction, and multiple response strategies.

Monthly principal meetings are used as professional learning opportunities as are the professional learning communities that meet weekly in all schools.

• Having aligned curriculum and assessments. The district has used its own talent ‒ teachers ‒ to develop curricula and assessments aligned to the state standards. Lesson objectives help focus students attention on what they are learning and their own learning goals. These tools, plus coaching on instructional strategies, were at the core of the instructional program in HSD2 designed to improve student achievement.

• Setting and funding priorities. Resources have been reallocated to fund system accountability and improvements. Salaries under the pay-for-performance plan are the same percentage of the general fund as under the traditional compensation system. Stipends for mentor teachers or department chairs do not exist because they are built into the teachers E&R compensation level. Grant resources have been used to pay teacher stipends to write test items, eliminating the need for an outside contractor.

• Using a do-it-yourself approach where possible. By using their own staff to develop products and processes, not only has time and money been saved, but staff capacity has been built. Involving teachers in the development of curricula and test items has accelerated the process and given credibility to the tests used in the pay-for-performance system.

• Having a data system that tracks information for students and teachers. HSD2 developed a very sophisticated data management system tailored to its E&R program. This data system is critical to the pay-for-performance plan. The District will also be implementing a commercially available data system called Infinite Campus™.

C o l o r a d o L e g a c y F o u n d a t i o n!

This is a Web-based

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

23

system that serves as a district-wide data warehouse that allows student data to be entered once and used across the district for data-informed decision-making.

Inhibiting Factors Harrison Two has managed to overcome many of the inhibiting factors that sometimes plague other districts. Just a few were mentioned: Declining resources for education. One district leader said Implementing the pay-forperformance plan would have been a cake walk in earlier years when more resources were available.

Teachers association protection of poorly performing teachers. Although this was more prevalent in the early implementation of the system, the rigor and fairness of the teacher evaluation process has since reduced law suits. Remediation and legal fees were high when Miles first began to transform the district. Now most unsatisfactory teachers have chosen to resign.

Requests from the Colorado Department of Education. In one example the CDE requested updated unified school improvement plans. But the superintendent sent the school action plans with a cover letter explaining these were driving every day decision-making in schools̶and holding principals and schools accountable for results.

Lessons Learned • Know your vision and purposes for an educator effectiveness system. The district had three purposes for the evaluation system: 1) raise student achievement, 2) provide effective signals to teachers to improve the quality of instruction; and 3) differentiate teacher evaluations in order to make better decisions in promotion, retention and professional development. The design of and supports for the system flowed from these three purposes.

• Use SB-191 to create the best learning environment for kids ‒ and principals, teachers, school board members and central office staff. Although SB-191 wasn t in effect when HSD2 designed their system, they urge other districts to make the most of the legislation by including a strong professional learning component as part of the evaluation system and other school improvement strategies.

• Create your own messages and media. Sometimes negative press that fueled opposition came from teachers in other districts commenting on the system. HSD2 worked quietly at first, but when the system had to be widely communicated they got ahead of the press with C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

24

carefully constructed messages. Simple and understandable language was used to communicate the intent of the evaluation system.

• Be strong and relentless in supporting teachers and dismissing those that don t meet performance criteria after efforts at remediation. HSD2 uses the evaluation system for one of its intended purposes ‒ to raise student achievement. When teachers couldn t do this with supports and coaching, they went through the dismissal process.

• This is a huge system and paradigm shift. Be ready to coach teachers and staff through a large-scale change process. Involving teachers in particular design aspects of the system (particularly rubrics) helped Harrison teachers buy in to the system as well as shape it. Each building had an E&R liaison that went to monthly meetings and reported back to their school.

• Leadership should be accessible and available to answer questions, keep a pulse on the district and help deepen understanding of the new system and how it works.

When issues arose, the superintendent was available to answer questions and deal with them. As common issues arose, monthly FAQ newsletters went out to teachers. District leadership also made adjustments to the system based on teacher input.

• New hires should know exactly what they are signing up for! Make sure expectations are clear from the start. A corollary: Teachers: Be Ready! Or Get Out! A teacher commented, Teachers should be ready to meet 100% of their students needs or consider a career change.

• You won t be successful unless there is passion attached to it. Teachers that bought into the system for the sake of raising student achievement had a different passion for teaching than those hoping to move up the pay scale quickly.

Outcomes Already HSD2 has raised student achievement. The total district median growth percentile has gone from 45 to 51 in mathematics between 2008 and 2010 (based on CSAP scores).

Writing scores are slowly improving. The biggest gains being made are the students who are catching up.

Over the past three years, about 5% more students are catching up. While the

achievement scores of this high-poverty district are below the state average, the Harrison District has significantly improved achievement, narrowing the gap with the state. In elementary reading for example, the District experienced a 12% increase in proficiency from 2007 to 2011. The State saw only a 2% increase during the same period. In middle school math, the C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

25

District witnessed an 18% increase in math; the State s increase was 7% during the same time period (2007 to 2011).

A majority of Harrison teachers support the E&R plan. Some results from the 2010 survey: Licensed professionals should be compensated based on performance and student achievement results My principal will accurately place me at the right level on the E&R plan The district assesses standards and benchmarks important to improving quality of instruction

: 80 75 70 65 60 % of Teachers That Agree or Strongly Agree (based on 58% response rate)

Next Steps Currently data are being collected on the implementation and outcomes of the E&R program.

It is part of a systemic transformation of the Harrison School District. It is difficult to assess the success of the program in isolation of the other reform initiatives. Nevertheless, the initiative will be shown to be successful, if, after five years: • The district's students score at or above the state average (for proficient and advanced scores) in at least half of the 27 CSAP exams.

• The district surpasses the state's median longitudinal growth measure (50) on at least 16 of the 21 exams for which longitudinal growth is calculated.

• The district improves retention of high quality staff as measured by a 20% decrease in the number of proficient teachers resigning from positions in the District.

• The district improves teacher performance as measured by a 20% increase in the number of teachers receiving a proficient evaluation.

Progress on these indicators will be monitored every year.

A new five-year plan was released in Harrison in January 2011 calling on the community̶and supported by the district-- to raise student expectations and create a college-going mindset.

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

26

Believing that college is within the reach of our students, the following goals̶to be achieved by 2016̶are set: • 90% of students will graduate from high school • 70% will be college and career ready • 70% will enter a post-secondary institution or college, or the military, directly from high school Superintendent Miles sums it up this way: It takes a lot of commitment to work in a district likes ours. But it s that hard work that is truly making the difference in our district. We are looking for those who can make that kind of commitment and sacrifice to meet our goals. And no, it is not easy. But it is and can be extremely rewarding.

Acknowledgements Central office staff, school board members, principals and teachers participated in interviews.

Harrison School District opened its doors to researchers to describe what they were doing, how they were doing it, the challenges they faced and lessons they learned. Interviewees were both gracious and candid in their interviews. Without them, this knowledge could not be captured and shared.

Author: Jane Armstrong, JM Armstrong & Associates Research Assistance provided by: Ulcca Hansen, Associate Director of Educator Effectiveness at Colorado Legacy Foundation; Heather Chikoore, Promising Practices Manager at the Colorado Legacy Foundation; and Kristen Davidson, University of Colorado, Boulder

Endnotes 1 2

Harrison School District Two: Calling All Heroes. Education Week, June 11, 2011 Education Week, January 5, 2011

C o l o r a d o L e g a c y F o u n d a t i o n!

H a r r i s o n S c h o o l D i s t r i c t 2 Te a c h e r E f f e c t i v e n e s s C a s e S t u d y

27