TPS Planning Guide by The Colorado Education Initiative

TEACHER PERCEPTION SURVEY PLANNING GUIDE

Copyright 2014 by The Colorado Education Initiative. All rights reserved. The Colorado Education Initiative is pleased to have organizations or individuals share its materials with others for non-commercial purposes. To request permission to excerpt or share this publication, either in print or electronically, please contact publications@coloradoedinitiative.org.

Overview of Colorado’s Teacher Perception Survey Teacher Perception Surveys provide a unique form of actionable feedback that districts and principals can use to inform practice. The Colorado Legacy Foundation (CEI) has engaged in a rigorous development and pilot process to create a Teacher Perception Survey that is specifically designed with input from Colorado districts and principals. This survey is a tool for collecting teacher input that can inform an individual principal’s annual evaluation and/or a principal’s professional growth efforts.

HOW DOES THIS SURVEY DIFFER FROM TELL? Colorado statute requires the Colorado Department of Education (CDE) to administer a biennial survey of principals and teachers about school and district working conditions. CDE uses a survey for this purpose called “Teaching, Empowering, Leading and Learning” (TELL). It is an anonymous statewide survey of school-based educators to assess teaching conditions at the school, district and state level. The TELL survey is used in many districts across the country. TELL survey results are intended to support school and district improvement planning and to inform policy decisions. See www.tellcolorado.org for more information. Colorado’s Teacher Perception Survey differs from the TELL survey in several key ways: • Colorado’s survey asks teachers about observable practices related to the Principal Quality Standards, not general questions about teaching conditions. • Colorado’s survey was created specifically for Colorado and is aligned to the Colorado Principal Quality Standards. • The TELL survey is biennial. Colorado’s Teacher Perception Survey can be administered annually or even more frequently if districts choose. • The TELL survey measures teaching conditions at the school-level whereas the Colorado’s survey measures practices attributed to the building principal.

HOW TO USE THIS GUIDE The Colorado Education Initiative worked closely with 16 partner districts during the pilot process to identify best practices regarding survey planning, administration, and reporting. We have compiled those best practices into a comprehensive guide with everything that districts need to effectively use Colorado’s Teacher Perception Survey. This guide will walk you through the entire process of planning for and executing a successful teacher perception survey administration. Planning timeline The planning timeline at the beginning of this guide will give you a big-picture view of what the planning, administration, and reporting process will look like. The subsequent sections of the guide will elaborate and provide examples for each component of the process. Other resources Throughout this guide we will reference other materials such as editable templates, checklists, and PowerPoint presentations. You can find all of these resources on our website (www.coloradoedinitiative.org/resource/teachersurvey-toolkit). Copyright 2014 by The Colorado Education Initiative. All rights reserved. The Colorado Education Initiative is pleased to have organizations or individuals share its materials with others for non-commercial purposes. To request permission to excerpt or share this publication, either in print or electronically, please contact publications@coloradoedinitiative.org.

Table of Contents 1. Planning a) Planning timeline b) District-level planning i. Key decisions ii. Data collection prior to survey administration iii. Informing stakeholders 2. Administration a) Process 3. Results and reflection a) How to prepare reports and analysis b) Planning for the distribution of results

Planning

PLANNING TIMELINE Category

Person Responsible

Planning Planning

Superintendent

Task 3 Months Before Administration Assign a district survey coordinator Make key decisions: Use a vendor or manage administration in-house? How will results be used? When will the survey be administered? Start pulling needed data

Superintendent and survey coordinator

Planning

Survey coordinator

Communication Planning

Superintendent Superintendent and survey coordinator Survey coordinator

2 Months Before Administration

Planning

Inform staff of the purpose and process Plan for distribution of results Verify teacher email lists with principals

1 Month Before Administration

Communication

Survey coordinator

Administration

Survey coordinator or vendor

Remind teachers when the survey is coming

Administration Window (2-3 weeks)

Communication Communication Planning

Survey coordinator Survey coordinator Survey coordinator

Communication

Superintendent and survey coordinator

Send the survey links to teachers Send reminder 2 weeks after initial email Send final reminder two days before survey closes

Immediately After Administration Debrief with staff on the survey administration process Review the timeline for reports and plans for engaging with results Prepare data for reports

1-2 Months After Administration Distribute reports and engage with staff on results

PLANNING KEY DECISIONS

Assign a district survey coordinator It is important to have one point person at the district level who can manage the survey process across all schools in the district. This person should have access to all schools in the district and a working knowledge of district data systems and processes. Depending on the structure and capacity of your district, it may make sense for this person to only manage logistics while the superintendent or another senior administrator is in charge of communication with principals. Use a vendor or manage administration internally? Making the decision to use a report vendor or to manage the entire process internally will depend on your district’s budget and internal capacity. While using a vendor may cost more initially, doing so may ultimately result in a smoother survey process unless there is sufficient district capacity to manage all aspects of planning, administration, and reporting. As you decide the best strategy for your district, we recommend that you read through this guide and consider all aspects of coordinating the administration logistics and reporting to decide if there is enough internal capacity to manage the survey process alone. How will results be used? It is important to be clear with principals from the very beginning on how Teacher Perception Survey results will be used to inform goal-setting and evaluations. See the last section of this document, “Planning for the Distribution of Survey Results” for more information and resources on this topic.

BEST PRACTICES • Assign one person to be in charge of surveys for the district • Be clear with principals on how results will be used • Consider holidays, testing and in-service days when choosing an administration window

When will the survey be administered? • The Colorado Teacher Perception Survey will give principals feedback that they can use to inform their practice. To maximize this potential, administer the survey during a window that gives principals time to reflect on their results and apply these insights during the current school year. • Choose an administration window of about 2-3 weeks. Take into account testing schedules, in-service days, and other school holidays during planning. • Ideally you should choose a window between November and March. To ensure fairness and avoid potential conflicts of interest, it is recommended that teachers complete the survey before they receive their final evaluation results from their principals.

DATA COLLECTION PRIOR TO SURVEY ADMINISTRATION

Data quality is extremely important in ensuring a smooth administration process and building confidence in the accuracy of results. Sampling Criteria • Teachers: Most schools choose to survey only licensed teachers, but districts or schools could also choose to include other non-licensed staff members, such as paraprofessionals or other administrative staff members. • Administrators: Colorado’s Teacher Perception Survey is currently only designed for school principals, not other administrative staff. Relevant data • Certain data elements are imperative for accurate administration and reporting: Copyright 2014 by The Colorado Education Initiative. All rights reserved. The Colorado Education Initiative is pleased to have organizations or individuals share its materials with others for non-commercial purposes. To request permission to excerpt or share this publication, either in print or electronically, please contact publications@coloradoedinitiative.org.

•

o School assignments o Teacher names and/or ID numbers o Principal names and/or ID numbers o Teacher email addresses Your district may also choose to include additional data so that results can be disaggregated by specific subgroups, such as: o Teacher years of experience o Probationary status o Teacher effectiveness ratings

Data verification • After the data listed above has been collected and aggregated at the district level it is important to have school administrators review it before it is used to assign surveys.

INFORMING STAKEHOLDERS • •

•

It is important to inform teachers and principals of the survey purpose and process early. Districts can utilize the following resources on our website to help with communication: o The survey instrument o Introduction email for principals o Introduction presentation for principals o Introduction email for teachers o Principal FAQ o Crosswalk of survey questions to Principal Quality Standards School board members and other community stakeholders should also be informed about the survey purpose and process. o Districts can use our overview of the Teacher Perception Surveys to communicate with these stakeholders

Administration

PROCESS • •

• •

Survey links should be sent to teachers on the first day of the administration window with a clear reminder of the purpose and process of administration. Districts can use our sample email to teachers. Reminder emails should go out to teachers at least once during the window to encourage participation. It is important for as many teachers to complete the survey as possible to ensure a representative sample size for principal reports. Send final reminder two days before survey closes. The survey administration window should be 2-3 weeks.

BEST PRACTICES • Encourage teachers to complete the survey during an in-service day • Provide school-level incentives for teachers to complete the survey (e.g., a teacher appreciation lunch for the school with the highest response rate)

Results and Reflection

HOW TO PREPARE REPORTS AND ANALYSIS There are three things to consider when preparing results and conducting analyses related to the Teacher Perception Survey: • How to aggregate data to the principal level • How to ensure data quality (e.g., cleaning and preparing data for analysis) • How to organize reports for interpretability

BEST PRACTICES • Decisions about how data will be used should guide the development of reports and analysis of data • Principals should be consulted and included in decisions about report organization • Data should be thoroughly cleaned and reviewed to ensure accurate, actionable results

Aggregating data • In general, there are two ways we recommend presenting principallevel results: o Mean score – this can be calculated by item, element, and overall. o Percent of responses in the top two categories, referred to as “percent favorable” – this can also be calculated by item, element, and overall. • The decision to use either method is defensible and should be made based on the needs of your principals.

Ensuring data quality • The full survey instrument asks teachers to enter their school and principal names to ensure accurate assignment of teachers to principals. o Analysts should compare this self-reported data to the original district data to ensure it matches. In instances where the teacher-entered data do not match the assigned data, that response should be invalidated. • Analysts should also remove duplicate teacher responses. In the pilot administration, these most often occurred when teachers had been kicked-out of the survey due to web connectivity issues; in these instances, it is recommended that you include completed surveys in analysis, and remove any incomplete responses. • In order to ensure data quality and protect teacher confidentiality, principals with fewer than 10 responses (or with especially low response rates) should not be given principal-level results. o Analysts should also ensure that there is a sufficient sample and response rate at the district-level to justify the inclusion of district-results. • District personnel and analysts should always consider potential unintentional violations of confidentiality when reporting on the data from the Teacher Perception Survey. For example, in a small district, providing comparison data for other schools could potentially yield information about the scores and trends in a specific building. Organizing reports The way you choose to organize your reports should align with the way you want to use the results. For example, if you want to include an aggregate score as one measure in a principal’s final evaluation, your reporting should include some aggregate measure. At a minimum, principals should always be provided with the same information that is shared with their district or evaluator. • Colorado’s Teacher Perception Survey is organized by the following eight elements: o Distributive Leadership Copyright 2014 by The Colorado Education Initiative. All rights reserved. The Colorado Education Initiative is pleased to have organizations or individuals share its materials with others for non-commercial purposes. To request permission to excerpt or share this publication, either in print or electronically, please contact publications@coloradoedinitiative.org.

•

o Professional Growth o Student Learning & Expectations o Problem Solving, Conflict Management, and Disciplinary Leadership o Vision & Goal Setting o Instructional Leadership o School Community o School Culture & Teaching Conditions Reports can be organized by the elements, by individual question, or as an aggregate score of all questions.

PLANNING FOR THE DISTRIBUTION OF SURVEY RESULTS

Planning for the distribution of survey results should take place during the initial planning stages. Using the results • Districts and schools should have plans in place for professional BEST PRACTICES development around survey results. Some suggestions include: o Group discussions about district-level trends • Start planning the process for o School-level discussions about specific trends report distribution early o Strategic pairing with peers to match strengths and • Ensure that principals are aware weaknesses of the timeline • Districts can use the following resources on our website to assist with the use of results: o A crosswalk of all Teacher Perception Survey Items to the Principal Quality Standards o Sample guidance for using survey results for principals and district staff o Guidance for districts on using results for Unified Improvement Planning