6. Continuous Quality Improvement

6

Continuous Quality Improvement

This coaching guide is all about CQI: supporting programs in an ongoing, formal approach to creating and sustaining a high-quality learning organization.

The competencies in this category delve into how to support others in their CQI process, covering change theory, stakeholder involvement, use of data, and action planning. This chapter gives you tools to support programs in building their capacity for continuous improvement.

1. Change Theory Understands that improvement is continuous and takes time

CORE COMPETENCY

“People don’t resist change. They resist being changed.”

Senge (2006)

Recall the CQI process from Chapter 1.

Change does not happen overnight. Realistic expectations and continuous, incremental steps are key to successful quality improvement. Tips for supporting change related to CQI principles are presented in the table below.

Continuous Improvement Process

CQI PRINCIPLE

Make gradual improvements involving people who have a stake in the change

Build on strengths, interests, and talents

Help people feel the potential of the change

Build buy-in and address resistance

COACHING TIPS

• Coordinate change efforts so teams don’t feel swamped by changes. People have difficulty making changes when they are tired or distracted or when they have too many changes to focus on. • Make sure that the program leader communicates key ideas and directions throughout the program, including to organizational leadership. If you are the program leader, this communication is your responsibility. • Help individuals and teams identify ways to build on their strengths to reach their goal. • Highlight moments of effective practice when you observe them. Help the team see the first small signs of change. These bright spots can illuminate the way forward (Jablon et al., 2016). • Find out how the program leader or CQI team feels about the planned changes. Work to shift negative thinking and feelings to positive. • To help the program leader or CQI team envision the changes, take a field trip to a similar program that is already operating the way the CQI team is planning. • Find out where the resistance is coming from. Address any needs for more information, more support, or more time for implementation. • If people seem uneasy about talking about the proposed change, put out an informal survey. Bloom (2015) offers a sample change-reaction checklist.

People often talk about “low hanging fruit” and “early wins” when it comes to making change. If you want to work on improving peer relations in an OST setting, maybe there’s an easy change you can make in the arrival or opening activity. If you want to work on staff development, you can build in reflection time to every staff meeting.

2. Stakeholder Involvement and Shared CORE COMPETENCY Decision-Making Emphasizes continuous quality improvement in partnership with the program leader or staff team

“Action without the participation of those involved is destined to fail… How we talk to one another, how we listen, how we relate, may be the most important thing of all. If we do not get that right, our noble intentions will not work.” Intrator & Scribner (2007)

A stakeholder is a person or group with an interest in or concern about the issue at hand. For an afterschool program, key stakeholders are the program leader, the staff team, and the participants. Additional stakeholders could be family members and community and school partners. A CQI team may include representatives from each stakeholder group.

Involving stakeholders in decision-making increases buy-in. Change is hard enough when people want to change; the failure of most New Year’s resolutions is a case in point. When the decision to change comes from the top down, the change is even less likely to be successful or sustainable.

By contrast, when the people affected by the change have a say in the decision—when planners encourage participatory decision-making—stakeholders are more likely to support the action plan or policy change.

You can support shared decision-making by encouraging the program leader or staff team to:

• Create a shared vision. Involve as many stakeholders as possible in identifying program goals and outcomes. Start with funder requirements. Brainstorm with staff, or have the program leader do so, to see what the people who work daily with participants think is important. Then involve participants, their families, and community members in tailoring the vision to meet local needs. • Learn as a team about research-based best practices so that the team selects strategies that are guided by evidence. Staff turnover and the fact that youth • Collect data about participants age out of programs means that stakeand families, the local com- holders are likely to change frequently. munity, and current program Encourage your learning partners to quality. (See Core Competency plan proactively to preserve continuity 3 below, Data-Informed Practice despite these changes. and Assessment.)

Which stakeholders the organization chooses to involve depends on its readiness for CQI. If the level of readiness is not apparent, one good strategy is to start with a core staff team and gradually build a bigger CQI stakeholder team with a more comprehensive process. The table below shows a sample threephase plan for deepening CQI efforts. program evaluation report. engagement.

CQI PHASE

Initial

Intermediate

Comprehensive

STAKEHOLDERS

Program staff

Program staff, youth, and some family members program assessment results. fiscal strength.

Program staff, youth, families, external stakeholders

SAMPLE TASKS

Program leader facilitates weekly staff dialogue about the latest Staff receive training on a program quality self-assessment tool to gain experience in assessing program quality, clarifying the vision, and identifying areas for improvement. They can start by looking at the areas of observational quality that are most predictive of positive youth outcomes: adult-child interactions and participant CQI team conducts a program quality observational assessment to see what is working and what needs to improve. In addition, CQI team decides which stakeholders to survey (staff, families, participants) to get the most valuable data to add to the The program leader summarizes findings on strengths and areas for improvement, focusing on programmatic areas and structural quality issues such as access to professional development and and partners

CQI team collects additional data on, for example, afterschool attendance or outcomes, school achievement, youth risks or assets.

3. Data-Informed Practice and Assessment CORE COMPETENCY Supports the program leader or staff team in collecting and analyzing data to make informed decisions

Program stakeholders are likely to have a variety of opinions about program strengths and areas for improvement. Data-informed practice can help programs move beyond opinions and hunches to focus improvement efforts where they can make a difference.

Sources and uses of data Coaches can support program leaders and CQI teams in analyzing data on a regular basis to inform an action plan. The program may have specific data collection requirements. The required data plus information from a broad range of sources can give programs a 360-degree picture of what is working well and what needs to change in their strategies, structures, policies, and procedures (Bernhardt & Geise, 2014).

“Without a clear sense of how the people, structures and processes of a center interact to produce desired outcomes, directors can’t ensure their program will be as effective as possible. Assessing organizational needs can help you turn these vague, amorphous feelings into more precise data about what aspects of your program can be improved.” Bloom (2015)

As a starting point, you may want to recommend that the program adopt a valid and reliable self-assessment. Use of such a tool enables the staff team to think about program quality and to discuss what quality means.

NIOST’s A Program Assessment System (APAS; see www.NIOST.org) is a tested and nationally recognized suite of tools including an observation protocol, program staff and youth surveys, and a structural quality questionnaire. Together, the tools measure OST quality and youth outcomes.

In addition to a self-assessment, you may recommend that the team use surveys and interviews to gather qualitative data on consumer satisfaction. For example, if a few families have complained about hours of service, the CQI team can use a survey to find out how many people feel this way. Having this information will help the organization balance the need of families with the cost of expanded hours.

Sometimes data is a potent force in shifting routines and attitudes to bring about needed change. For example, staff may think that the children are satisfied with program activities. However, the youth surveys show that youth in grades 6 to 8 are much less happy than those in grades 4 and 5. A focus group with the middle schoolers could not only provide more data about the reasons for dissatisfac- Kids do say the darnedest things! tion but also begin the process of Even before formal data analysis of engaging youth in restructuring the program to better meet their needs. youth surveys, go ahead and share funny or heartwarming quotes. They can give staff a big boost! You can also encourage the CQI team to look for other sources of information on program quality such as program attendance data and school attendance and achievement records. Encourage your learning partners to go slowly, collecting only the data they know how to use.

Analyzing the data Once they have collected data, you can support program leaders and staff teams in analyzing the data. The purpose of data analysis is to create a

complete portrait of the program with clear indications of program strengths and areas for improvement.

Caution: People often want to start thinking of strategies and solutions before they have analyzed all the data. Encourage your learning partners to wait for the full picture from all the data before they set priorities, following these steps:

Step 1. Summarize data collected based on the ideas or constructs measured. Step 2. Consolidate the data collected using a spreadsheet or data analysis software; the team may need the help of a data specialist. Step 3. Identify program strengths and areas that need improvement based on the data.

Support for data-informed practice Coaches can support CQI efforts by encouraging the program leader or CQI team to adopt data-informed practices. In a review of more than 50 studies from the fields of educational leadership and management, health care management, nonprofit leadership and management, public management, and organizational learning and development, six elements were found to consistently support data-informed practice (Derrick-Mills et al., 2014):

1. Leadership. Leaders must set a clear expectation that use of data is part of daily practice. They must also involve the CQI team in shared decisionmaking. To be fully prepared to make decisions, the team needs information that can guide decisions. Involving stakeholders in gathering and analyzing data is critical to shared decision-making. 2. Analytic capacity. Leaders must be prepared to collect valid and reliable data and to help the team analyze it to decide how to improve program quality. If team members aren’t ready, how can readiness be increased? What technology is available to help with the collection and analysis of data? 3. Commitment of resources and professional development. Leaders must prioritize and commit time and resources to data collection, analysis, and use. They must identify who is responsible for data collection and make sure those staff members have time separate from other responsibilities to accomplish this task. 4. Culture of collaborative inquiry. Leaders must create a safe space where learning is emphasized over accountability. Otherwise, conversations about data may lead to defensiveness and disagreements. When focused on learning, the team can more freely learn from what the data reveals. 5. Cycle of continuous quality improvement. Data-informed practice starts with collecting data to inform the development of an action plan and

continues throughout implementation. Checking back on how the data is changing provides important feedback loops for improving practice. 6. Organizational characteristics and environment. Each organization is different. Data should be understood in the context of these differences.

4. Action Planning and Implementation CORE COMPETENCY Supports the program leader or staff team in developing and implementing an action plan for continuous improvement

The next steps are to help the program leader or staff team identify their priority areas, create a plan for how they will improve, and then to begin implementing this plan.

Identify priorities for improvement Once the program leader and staff team have analyzed the data and identified areas for improvement, they can set priorities for the action plan. Though the top priorities may be self-evident, more likely there will be more areas for improvement than the team can tackle at one time. If so, help the program leader or staff team to identify priorities. Here are some initial steps they can take.

• Start by talking generally about where the program is now. • Review the strengths and areas for improvement identified by the team. • Examine the extent to which these data-informed lists match what the CQI team sees in practice. Is anything important missing? Are there trends in the field that should be taken into consideration? • Once each quality construct is sorted into an area of strength or an area that needs improvement, prioritize the improvement items to create an action plan. These questions can help with the process of discerning what is most important (Hunter et al, 2015):

1. Need: What is the most important issue that requires change? 2. Capacity: Do you have the resources you need to make the change, including staff, skills, and funding? 3. Impact: What change will yield the biggest impact on program participants? 4. Evidence-based: What changes are evidence-based and aligned with promising practice?

Create an action plan Once priorities for improvement have been identified, you can support the program leader or staff team to create an action plan. Here are some tips you can share to help your learning partners create a plan.

5. Write clear SMART goals (Cornell Human Resources, n.d.). • Specific. A specific goal generally answers these questions: • Who is involved? • What will be accomplished? • Where will it happen? • When will it be completed? • Which requirements and constraints need to be considered? • What is the purpose of including this goal? • Measurable. The goal includes specific criteria for measuring progress, including how the team will know when the goal is accomplished. • Attainable. The goal considers schedules, workload, knowledge, and funding. • Realistic. The team is willing and able to work on this goal. If no one agrees to be responsible for completing the goal, perhaps there is a lack of buy-in. • Timed. There is a timeframe and dates for completion. 6. Generate possible strategies or action steps. • Create a list of evidence-based strategies for the issue at hand. • Select a strategy that fits the program and has a high likelihood of success. 7. Identify who is responsible. Assign tasks to those who know the most about the goal and have the motivation and time to implement the strategy. 8. Set a timetable. Be realistic about when the task can be completed. Consider other tasks When setting a timetable, OST programs or initiatives the program is al- need to consider their annual cycle, ready implementing. including their school-year and summer programs. Coach the program leader or 9. Identify resources. Think team to use the calendar to their advantage through what is needed to com- to develop realistic milestones. plete the task. This may be staff time, training, funds, or materials.

Implement the action plan The program leader is generally responsible for implementing the action plan. External coaches can support program leaders to take the steps below; if you are the program leader, use these steps as a guide.

1. Coordinate implementation. The program leader should: • Make sure that each person responsible for an objective or action step knows how to report back on progress or problems. • Check in frequently with people who are doing the work. Ask how their energy is for the work ahead and whether they need assistance. • Track the timetable. If the work is not getting done on time, find out whether the person responsible Regular staff meetings can be a great tool in is swamped with other tasks or does not have the the CQI process—the necessary knowledge or skills. If the timetable is too already-reserved time ambitious, work with the CQI team to modify the plan. can be structured to 2. Continue to build buy-in. The program leader needs to help the CQI team take time for ongoing communicainclude time for reflection or to work on an identified action step. tion and coordination. • Get buy-in from organization leadership before you begin. Make sure it will provide the time and money needed to implement the plan. • Share progress with the larger program community and organization leadership. • Continue to meet with program staff on the implementation of the plan. • Publicly celebrate efforts and successes. CQI adds a layer of work on top of everyday tasks. People need to feel their efforts are leading to progress. • Identify any problems or resistance. Maybe the person in charge of implementation needs help, or maybe some part of the program community is resisting the change. Start with the CQI team, and then broaden the discussion as necessary to identify the issue and develop a solution or compromise (Metz, 2007). 3. Incorporate feedback loops. Institute procedures for reflecting on progress as new practices are implemented. • Regular meetings enable the CQI team to reflect on progress. • Check to see whether staff members need training or technical assistance to implement proposed changes. • Check in with people affected by the change, either formally or informally, to see whether the goal has been accomplished as intended. 4. Once the action plan is complete, celebrate! Then talk about when to start the CQI cycle again.

FOR REFLECTION How is building staff and stakeholder buy-in different from addressing resistance? How are they the same? How can you help program leaders or your staff team see and address these similarities and differences?

In your experience, what kinds of stakeholders are most often difficult to engage? What advice can you give program leaders or your staff team about persisting to engage such stakeholders?

What are the biggest challenges programs face in becoming more data-informed? What can you do to help program leaders or your staff team minimize these challenges?

CQI References

Bernhardt, Victoria and Geise, Bradley (2014). Using Data for Continuous School Improvement. https://proposals. learningforward.org/handouts/Nashville2014/PC212/PC212%20Handouts.pdf Bloom, Paula Jorde (2015). Blueprint for Action: Leading Your Team in Continuous Quality Improvement. Third Edition. New Horizons. Cornell University Human Resources. (n.d.). Goal setting. https://hr.cornell.edu/professional-development/ career-management/career-management-resources/goal-setting Derrick-Mills, Teresa, Heather Sandstrom, Sarah Pettijohn, Saunji Fyffe, and Jeremy Koulish. (2014). Data Use for Continuous Quality Improvement: What the Head Start Field Can Learn from Other Disciplines, A Literature Review and Conceptual Framework. OPRE Report # 2014-77. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. U.S. Department of Health and Human Services. https://www.researchconnections.org/childcare/resources/28919/pdf Hunter, Sarah B., Patricia A. Ebener, Matthew Chinman, Allison J. Ober and Christina Y. Huang. (2015). Promoting Success: A Getting to Outcomes® Guide to Implementing Continuous Quality Improvement for Community Service Organizations. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/tools/TL179.html. Intrator, Sam and Scribner, Megan (2007). Leading from Within: Poetry That Sustains the Courage to Lead. Wiley. Metz, Allison J. R. (June 2007). A 10-Step Guide to Adopting and Sustaining Evidence-based Practices in Out-of-School Time Programs. Child Trends. https://www.childtrends.org/wp-content/uploads/2007/04/ child_trends-2007_06_04_rb_ebp2.pdf Senge, Peter M. (2006) The Fifth Discipline: The Art & Practice of the Learning Organization. New York: Doubleday.