DPP Operations Evaluation 2022-2023

Denver Preschool Program Year 1 Evaluation Report December 2023 APA Consulting Mariana Enríquez, Dale DeCesare, and Brianna McGagin

Executive Summary

2

I.

Introduction

3

II.

Participant Sample

4

1.

Providers

4

2.

Families

6

III.

Findings

14

Perception of DPP

14

Finding Out About DPP

14

The DPP Application Process and Awareness of Tuition Credits

16

Awareness of Quality Ratings

21

Outreach to families

23

DPP and UPK

27

1.

2.

Customer Service / Interaction with DPP

35

3.

Tuition Credit System

39

4.

Choosing Preschool

55

Affordability

56

Accessibility

56

Quality

61

Culturally Responsive Services

63

5.

Maximizing DPP Impacts and Access to Childcare

66

6.

Communicating DPP evaluation findings to multiple audiences

75

Summary of Findings

81

IV.

Highlights of Section 1: Perceptions of DPP

81

Highlights of Section 2: Customer Service / Interaction with DPP

83

Highlights of Section 3: The Tuition Credit System

84

Highlights of Section 4: Choosing Preschool

85

Highlights of Section 5: Maximizing DPP Impacts and Access to Childcare

87

Highlights of Section 6: Communicating DPP evaluation findings to multiple audiences 88 V.

Recommendations

89

Appendix A: Evaluation Advisory Group (EAG)

94

Appendix B: Evaluation Research Questions

95

Executive Summary To be completed after initial reviews and discussion with DPP.

2

I.

Introduction

Augenblick, Palaich and Associates (APA) was contracted by the Denver Preschool Program (DPP) to conduct a 3-year evaluation of its operations. This report covers the 2022-23 evaluation year, and presents findings in the following six major topic areas: 1. Perception of DPP and UPK 2. Customer service / Interaction with DPP 3. The Tuition credit system 4. Choosing preschool 5. Maximizing DPP impacts and access to childcare 6. Communicating DPP evaluation findings to multiple audiences.

These areas were selected to help address a set of evaluation research questions, which are listed in Appendix B. The APA evaluation team (Evaluation Team) convened an Expert Advisory Group (EAG) to help inform the design and execution of evaluation activities. The EAG offered ideas for outreach to providers and parents, provided input on priorities regarding DPP’s operations that should be addressed during each year of the evaluation, and offered feedback on the evaluation’s data collection instruments and on interpretation of the data collected (additional information on the EAG is provided in Appendix A). The evaluation team also sought and received input from DPP personnel on the design of data collection instruments and data collection activities. The evaluation team wishes to thank both the EAG and DPP personnel for their support.

Data were collected from families1 of 4-year-old children enrolled in DPP who received tuition credits during the 2022-23 school year as well as from early childcare providers active in DPP during this same school year. With the purpose of supporting DPP in promoting equity and ensuring equitable access to high-quality preschool, the data was analyzed not only based on families’ income tiers as identified by DPP, but also by the geographic area where they live. The evaluation team decided to split the city into quadrants (northeast, northwest, southeast and southwest) using Broadway and Alameda Avenue as the axes of this geographic division.

It is expected that this disaggregation of data will help DPP and the evaluation team understand families’ different lived experiences as they relate to accessing preschool for their children, as well as availability of services. Findings are presented in Section III of this report, broken down by the six major topic areas listed above. Key findings are highlighted at the end of each of these six sections and are presented in bulleted fashion as “Section Highlights.” The overall set of these highlights are also presented in Section IV, along with recommendations from the APA evaluation team in Section V. 1

Throughout this report, the term family will be used interchangeably with parents.

3

II.

Participant Sample

Quantitative and qualitative data were collected from childcare providers and families via a mix of online surveys and virtual focus groups. Data were collected in English and Spanish according to participants’ preference. This section describes both groups of participants and the process used to engage them in the evaluation.

1. Providers Directors of all 239 preschool centers active in the DPP program during the 2022-23 school year were invited to participate in virtual focus groups to share their perceptions of/interactions with DPP, DPP’s cost of care model, and UPK. A total of 150 providers from community- and homebased centers, and 89 school-based providers2 were sent the evaluation team’s email invitation.

The same group of providers was also invited to complete an online survey. Focus group data were used to fine-tune the survey questions that addressed the same general topics covered by the focus groups. Table 1 shows the distribution of providers invited to participate in the data collection, broken down by type of center and quadrant. Figure 1 presents the distribution of all invited providers broken down by quadrant. The northeast quadrant had the highest representation at 44%, followed by the northwest with 20%, the southeast with 19%, and the southwest with 17%. Community-based preschools were represented by 63% of providers, while the remaining 37% were school-based (mainly DPS) providers.

Table 1. Quadrant distribution of providers invited to participate in data collection Invitation sent to all active DPP providers, spring 2023: School-based providers (DPS & charter) (37%) Community- & home-based providers (63%) Total (community-, home-, & school-based)

NE

NW

SE

SW

Total

38 43% 68 45% 106 44%

20 22% 27 18% 47 20%

12 13% 33 22% 45 19%

19 21% 22 15% 41 17%

89 99% 150 100% 239 100%

Figure 1 shows these data graphically, again showing that there were similar proportions for the NW, SE, and SW quadrants, while the NE quadrant had more than double of any of the other three quadrants. One explanation for the higher number of current NE childcare providers may

2

For purposes of this report, community- and home-based providers will be referred to as community providers or as non-DPS providers, while school-based providers will be identified as school providers or DPS providers when referring to that subset of providers. School providers include DPS and charter school providers; during the 202223 school year, there were six charter school providers partnered with DPP.

4

be the significantly larger geographic area of the NE quadrant as compared to the other quadrants as shown in Figure 3.

Figure 1. Percentage of all providers invited to participate in the data collection, by quadrant (n=239) 17%

44% 19%

20% NE (n=106)

NW (n=47)

SE (n=45)

SW (n=41)

Providers’ overall response rate to the survey was 35%. Figures 1 and 2 combined show that, compared to their overall representation in DPP, survey participation rates were higher from providers in the NE (44% v 50%) and SE (19% v 25%) quadrants, while the NW representation was slightly lower (20% v 17%) and significantly lower for the SW (17% v 8%) quadrant.

Figure 2. Percentage of providers' survey responses received, by quadrant (n=52, 35% response rate) 8%

25% 50%

17% NE (n=26)

NW (n=9)

SE (n=13)

SW (n=4)

Figure 3 shows the geographic distribution of providers (GIS mapped by Denver neighborhood) who completed the survey. Closer analysis of these data shows that providers in the NE quadrant were highly concentrated in the Near-NE area (NNE) of Denver, while only three providers were 5

located in the Far-NE (FNE) located in Montbello and Gateway-Green Valley Ranch. Similarly, NW providers were more concentrated towards the south-center of that quadrant and only one provider from farther north (Chaffee Park) completed the survey. The SE quadrant shows a more even distribution of respondents, while the SW had minimum provider survey participation.

Figure 3. ECE Providers that responded to Evaluation survey, by neighborhood

Providers’ interest in participating in the evaluation team’s focus groups was low in spite of encouragement to participate from DPP personnel and the evaluation team. Eleven community and three school providers expressed interest in participating, although only six community providers representing the four Denver quadrants elected to participate in a one-hour virtual focus group conducted in English.

2. Families During the 2022-23 school year, DPP provided tuition credits to 4359 families3 (identified using existing data and removing duplicates). Some of these families received tuition credits for more than one child (e.g., more than one 4-year-old child, or one 4-year-old and one 3-year-old).

3

The number of families included in the data collection reflects one moment in time during spring of 2023 because the number of families fluctuated throughout the year as families enrolled or dropped out of DPP. Of the 4359 families receiving DPP tuition credits in the spring of 2023, more than half (51%) resided in the northeast (NE) quadrant of Denver, followed by 18% in southeast (SE), 17% in southwest (SW), and 14% in the northwest (NW) quadrant. In terms of income Tiers, Tier 1 had the highest representation at 31%, followed by Tier 2 with 23%, Tier 5 with 18%, Tiers 3 and 6 with 12% each, and Tier 4 with 4% representation.

6

Figure 4 shows the distribution of these families by income tier and city quadrant as of spring 2023. These data show that families from income Tiers 1 and 2 (>100 FPL and 100-185 FPL, respectively), followed by families from Tier 5 (<400 FPL) had the highest participation in DPP. Families from Tier 4 (300-400 FPL) had a significantly lower participation, followed by families from Tiers 3 (185-300 FPL) and 6 (no income reported).4 Data also show that NE quadrant families had the highest DPP proportional participation in each of the Tiers, while families in the SW quadrant had greater representation in Tiers 1 and 2. Families from the NW quadrant had a proportionally equal distribution across income tiers, while SE families showed high numbers at both ends of the income levels, slightly more pronounced in Tiers 1 and 5. The maps in Figures 5 and 6 show the geographic distribution of families by neighborhood and income tier respectively.

Figure 4. Number of unduplicated families receiving DPP tuition credits by tier and quadrant, spring 2023 (n=4359)

NUMBER OF UNDUPLICATED FAMILIES

NE

NW

SE

SW

307

186 253 200

48 133

213

116 678 485

114

94 80 70

108 25 42 20 83

258 TIER 1

TIER 2

22 126

TIER 3

TIER 4

423 275 TIER 5

TIER 6

The map in Figure 5 shows high concentrations of DPP families towards the outskirts of Denver, while families are more dispersed towards the center of the city. The NE quadrant shows the highest density of DPP families, primarily in Central Park (formerly Stapleton) and the Far-NE area. Figure 6 shows that high density DPP service areas are where low-income families tend to live, with the notable exception of the Central Park neighborhood in NE Denver, that shows high concentrations of high-income families, as well as several neighborhoods in NW Denver that show mid-level concentrations of middle to high income families. The SW quadrant and the 4

Since these data were collected, DPP revised the FPLs that define each income Tier.

7

north central neighborhoods showed the lowest income families with mid-level density of families receiving DPP tuition credits.

Figure 5. Geographic distribution of families receiving DPP tuition credits by neighborhood, Spring 2023

Figure 6. Geographic distribution of families receiving DPP tuition credits by income tier, Spring 2023

8

Figures 7 and 8 show the proportion of unduplicated families receiving DPP tuition credits by Tier and by quadrant, respectively. Data show that more than half of DPP’s families (54%) are in the lowest income (Tiers 1 and 2), while close to one fifth of families (18%) come from high income households (Tier 5).

Figure 7. Proportion of unduplicated families receiving DPP tuition credits, by Income Tier, Spring 2023 (n=4359) Tier 6 12% Tier 1 31% Tier 5 18%

Tier 4 4% Tier 3 12%

Tier 2 23%

Figure 8. Proportion of unduplicated families receiving DPP tuition credits, by city Quadrant, Spring 2023 (n=4359) SW 17%

NE 51%

SE 18%

NW 14%

Not all families were asked to complete the survey or participate in the evaluation’s focus groups. The Evaluation Team used the following criteria that determined that some families needed to be excluded from the data collection efforts:

9

•

•

•

In coordination with other DPP evaluation teams, families who had already participated in any data collection efforts with the other evaluation teams were excluded from the sample to minimize their burden.

Families who had more than one child receiving tuition credits were also excluded from the sample. This decision was made because several survey questions asked specifically about experiences with the child’s provider and it could be confusing for families to have to select and refer to only one of their providers to answer the questions. Similarly, families whose records indicated their child was enrolled in more than one school were also excluded. Exceptions to this situation were made when the child was enrolled in more than one type of program (e.g., full- and extended-day) in the same school.

Families where neither parent/caregiver had an identified email address. All outreach efforts for Year 1 of the evaluation were made via email communications, therefore, families without email addresses were not contacted. For Year 2, the evaluation team is exploring using different approaches to contact and engage families in the data collection.

Of note, some families fell into more than one of the excluding criteria. A total of 975 cases were excluded from the original 4359 families, leaving 3384 who were invited to participate in the data collection efforts.

Family survey Families were sent an email, in English and Spanish, inviting them to complete the online survey. The email included links to both the English and Spanish versions of the survey. As an incentive, families were offered a $25 Walmart gift card if they were one of the first 100 parents completing the survey. In addition, they were informed that all parents completing the survey would be entered into a drawing to have a chance to win their choice of an electronic Tablet or a one-year family membership to the Denver Museum of Nature and Science. The survey was open for two weeks in June 2023 and parents received a reminder to complete the survey five days before the survey closed.

The DPP Families survey yielded 383 family participants. Nearly half (49%) of those families reached were located in the NE quadrant of Denver, followed by 23% from the southeast, 16% from the northwest, and 12% from the southwest quadrant. Representation by Tiers was 22% from Tier 1, 17% from Tier 2, 8% from Tier 3, 7% from Tier 4, 35% from Tier 5 and 12% from Tier 6. A total of 59% of families participating in the evaluation had their children enrolled in a DPS school, while the remaining 41% were attending community-based preschools. Respondents appeared to be balanced between lower income tiers (Tiers 1-2) and higher income tiers (Tiers 45). For example, 38% of respondents fell into either income tier 1 or 2, and 41% fell into tier 4 or 5, though tier 5 was the most represented, with nearly 35% of all respondents. Table 2 shows additional details on the family survey respondents.

10

Table 2. Number of family survey participants by income tier and quadrant Income tier & family quadrant 1 2 3 4 5 6 Total

Northeast

Northwest

Southeast

Southwest

Total

46 32 14 12 63 22 189 (49%)

15 10 5 1 21 9 61 (16%)

9 7 9 8 40 14 87 (23%)

15 14 4 4 8 1 46 (12%)

85 (22%) 63 (16%) 32 (8%) 25 (7%) 132 (34%) 46 (12%) 383

Table 3 shows that the children of more than half (59%) of all survey respondents attended a DPS childcare center. Interestingly, survey respondents from lower income families were much more likely to be served by a DPS provider than the higher income families, who were much more likely to be served by a non-DPS provider. Approximately 62% of higher-income tier family respondents (98 of all tier 4 and 5 families) sent their children to non-DPS childcare centers, while 75% of lower-income tier families’ children (111 of all tier 1 and 2 families) attended a DPS childcare center.

Table 3. Distribution of survey participants by income tier & school-based enrollment Income Tier 1 2 3 4 5 6 Total

DPS 62 49 24 15 44 31 225 (59%)

Non-DPS 23 14 8 10 88 15 158 (41%)

Total 85 (22%) 63 (16%) 32 (8%) 25 (7%) 132 (34%) 46 (12%) 383

Survey data found: • Most families (47%) surveyed had two children under 18 years old living in their household. The most children under 18 years old living in any surveyed family’s household was six children. The distribution of family size was similar across the four quadrants of the city.

• Most families (51%) surveyed had two people in their household working a paid job and 34% of surveyed families had one person in their household working a paid job. Almost two thirds (63%) of families working less than 2 paid jobs fell into income tiers 1 and 2.

11

•

•

On average, 61% of respondents stated that they worked 25 hours or more a week and 42% of these families are tier 5 families, while 39% of families working less than 25 hours a week are tier 1 families.

Eight percent of survey respondents completed the survey in Spanish, all of them fell into income tiers 1 and 2.

Family focus groups The online survey administered to families included a question asking if they would be interested in learning more about a series of small group conversations with parents/guardians about how to improve DPP and how to help Denver families with more affordable preschool. It was explained to them that, if selected to participate in one of the groups, they would receive a $40 stipend in appreciation for their time. English-speaking families who indicated interest in learning more about the conversations were sent a follow-up email with additional details about the conversations and asking for their availability from five options for different days and times of the day. A smaller number of Spanish speaking families responded with interest in participating, and this smaller group was offered three options of time and day to participate. Families received information about the day, time, and link to join the virtual conversations, and two reminders about the upcoming conversations.

Fifty-three English-speaking and 21 Spanish-speaking parents indicated interest in the conversations. Interestingly, 50% of each group lived in the NE quadrant of Denver. All Spanish speakers came from income tiers 1 (62%) and 2 (38%). For English speakers 36% came from tiers 1 and 2, while 38% came from tier 5 households.

Seven focus group sessions with a total of 24 participants took place. One of the sessions was conducted in Spanish with two participants. Participants were sorted by quadrant, and sessions had an average of 3.4 participants each. Forty-two percent of participants enrolled their children in DPS preschools. Half of all participants (50%) come from tier 5 households and the children of 75% of these families attended non-DPS schools. By contrast, 29% of participants came from tier 1 and 2 households, and 57% of their children attended DPS preschools. There were no participants from tier 4, while tiers 1, 2, 3 and 6 were represented by an average of 3 participants each. The NE quadrant had the highest representation (46%) in the sessions, NW and SE quadrants had 25% of participants each, and SW had 4%.

Other data from families that participated in the focus groups: • Participants had an average of 2.3 children under 18 years old living in their household, with a minimum of one child and a maximum of five.

• Almost two thirds of participants (63%) had two people in their household working a paid job. About 25% of participants had only one person working a paid job.

12

•

•

54% of focus group participants worked between 25 and 40 hours a week, with just over half of them (29% of the total) from tier 5. On the other hand, 29% of total participants did not work or worked fewer than 8 hours a week, including 17% of total participants from tier 5, and 12% from tiers 2 and 3.

Participants from tiers 1 and 2 represented 28% of all focus group participants, 8% did not work, another 8% worked 8-24 hours a week, and 12% worked 25-40 hours a week.

As indicated above, it is important to note that those who agreed to participate in focus group data collection activities were overrepresented by English-speaking families, families from the Northeast quadrant, and families in income tier 5.

The next section of this report provides findings from across all data collection activities.

13

III.

Findings

APA’s evaluation of DPP’s operations is organized by a set of research questions contained within six major focus areas, including: 1. Perception of DPP and UPK 2. Customer service / Interaction with DPP 3. The tuition credit system 4. Choosing preschool 5. Maximizing DPP impacts and access to childcare 6. Communicating DPP evaluation findings to multiple audiences The evaluation team’s discussion below of Year 1 (2022-23) findings is organized by these six major topic areas.

1. Perception of DPP Finding Out About DPP Families who participated in the evaluation team’s focus groups reported learning about DPP from a variety of sources, but primarily from other parents or their childcare providers, some of whom strongly encouraged them to apply regardless of their income and ability to pay for their children’s tuition. Several low- and high-income5 families shared that their providers required all families to complete the DPP application alongside their preschool enrollment paperwork.

Parents also learned about DPP from friends, family, and neighbors who told them that by applying they could get their tuition partially or fully covered, as this parent shared: “Una amiga me dijo que a veces si no calificabas pagaban o te ayudaban a pagar la mitad, o a veces, si calificabas no pagabas nada.” [A friend told me that sometimes, if you did not qualify they would pay or help you pay half [the tuition], or sometimes, if you qualify, you did not have to pay anything]. (Tier 2, DPS, NW)6

5

For purposes of this report, families/parents will be identified as low-income if they fall within DPP’s income tiers 1 (>100 FPL) or 2 (100-185 FPL); identified as middle-income if they fall within DPP’s income tier 3 (185-300 FPL); and as high-income if they fall within DPP’s income tiers 4 (300-400 FPL) or 5 (<400 FPL), effective for the 2022-23 school year.

6

Throughout the report, parents’ quotations will be identified by the family’s DPP income tier level (T1 to T6), whether or not their child attends a DPS preschool (DPS, non-DPS), and Denver’s geographic quadrant where they live (NE, NW, SE, SW). Childcare providers’ quotations will be identified only by the geographic quadrant where the center is located (NE, NW, SE, SW).

14

Others in focus groups reported finding out about DPP through the Women, Infants, and Children (WIC) office or while attending community festivals or events where they were strongly encouraged to apply. A few parents indicated that internet searches of preschools took them to the DPP website where they found information about Denver preschools as well as financial information.

Focus group data about how families tended to get information about DPP was slightly different from data collected through the online survey. As shown in Table 1.1, similar proportions of parents who took the survey reported that they found out about DPP by asking their preschool providers7 or visiting the DPP website. This finding suggests the DPP website is an important tool that parents use to find out about DPP. Internet searches and asking other parents show smaller proportions in the survey than what parents reported during the focus group conversations, although it is important to note the smaller size of the focus group sample versus the number of survey participants (24 parents participated in focus groups versus 383 families represented by the survey). Table 1.1 shows the percentages of survey responses for each of the sources where families indicated they found information about DPP. Table 1.1 Survey data on how families find information about DPP Source of information I ask teachers or administrators at my preschool I go to the DPP website I search the Internet I ask other parents

Proportion of Responses 24% 23% 16% 12%

Parents who completed the online survey selected what they thought were the best ways of reaching families to let them know about DPP. Figure 1.1 shows that 65% of responses indicated a preference for verbal or written communications directly from either their childcare centers or DPP personnel.

7

Survey data shows that preschool providers have a high degree of confidence in their understanding of the DPP enrollment process, the quality rating system, the tuition credit system, and the benefits of participating in DPP as well as their capacity to help parents enroll in DPP. Their confidence was not equally high in how effectively they could communicate these topics to parents for whom English is not their first language.

15

Figure 1.1. Best form of communication to reach families News forms 13%

Social media 22%

Written communication 32%

Verbal communication 33%

Verbal Communication was identified as either direct conversations or presentations with DPP employees, preschool employees, or families or friends already in DPP. Fifty-five percent of responses in this category indicated a strong preference for communicating verbally directly with their preschool provider personnel. Written Communication was identified as either flyers, emails, or text messages from either childcare centers or DPP, with no specific preference about who issues the communication. Social Media was identified as Facebook, Instagram, WhatsApp, TikTok, YouTube, and Twitter. Facebook was the highest-ranking social media platform (54%) across all respondents and income levels. News Forms were identified as Radio, TV, and newspapers, magazines, and community newsletters. While all these news forms received minimal responses, 41% of responses in this category indicated a preference for TV when communicating via news forms.

The DPP Application Process and Awareness of Tuition Credits Most families in the focus groups reported learning that the DPP application was available on the DPP website at the same time they learned about the program. Some providers gave hard copies of the application to the parents to fill in, while other providers helped families complete the application and asked parents to gather needed documents to be submitted. One Spanishspeaking parent recalled having applied by calling a phone number and answering questions to complete the application.

“Creo que hablé por teléfono y ya allí me hicieron preguntas y ya me dijeron que tenía que mandar los comprobantes de ingreso a un correo electrónico que ellos me mandaron y ya allí me iban a decir si estaba aprobada o no, si iba a recibir la beca completa o pagar algo y ya fue que me mandaron la carta, la muchacha me dijo ahí que estaba aprobada cuando me habló y ya después me mandaron la carta.” [I believe I made a 16

phone call where they asked me questions and they told me I had to send proof of income to an email they sent me, and they would tell me if I had been approved or not, if I would get full scholarship or if I needed to pay something, and then they sent me the letter, the gal called and told me I had been The majority of parents indicated that approved and they sent me the letter later] (Tier 2, DPS, NW) completing the DPP application was a very easy and straightforward process.

The majority of parents indicated that completing the DPP application was a very easy and straightforward process. However, some parents in the evaluation’s focus groups reported not knowing that they had applied to DPP at the same time they were completing their preschool enrollment application. This was true for families from different income levels, as shown in the following quotes: “I was not aware that I applied for DPP with both of my children. I honestly just went up to the school that I wanted my kids to go to and asked if they had any openings. They gave me a packet, I filled out the packet and turned it back in. I did not realize what I was applying for and I got the DPP credits, so I guess I need to be more aware of what I'm signing. I wasn't aware of it.” (Tier 1, non-DPS, NW) “I'm hearing that people didn't know [DPP] was a program, I felt the same way when I was applying I think through the DPS choice… for better or for worse, I guess it's maybe a seamless thing that people are getting the funds that they need and qualify for even if they don’t know about the program.” (Tier 5, DPS, NW) One focus group participant reported they did not know they had applied to DPP, and also did not know about DPP even after having received tuition credits. This parent reported first hearing about DPP when being invited to participate in the DPP evaluation via the online survey and focus group conversations: “I didn’t know that it [DPP] was a program. I also just tried to enroll my child, both of my kids into preschool, Some parents reported not cause I knew you could, but I didn’t know that there was knowing they had applied to DPP at the same time they such a thing as a program that could help you to pay for it. were completing their I just thought, cause, I’ve always been low income and preschool enrollment usually qualify for things, I’m in that income bracket. So I application. didn’t even know it existed until recently with this whole thing that we’re doing today.” (Tier 2, DPS, NW) Another parent wrote a similar comment in the DPP online evaluation survey, “La verdad no se nada de DPP quisiera saber de que se trata esto?” [The truth is that I know nothing about DPP, I would like to know what this is about.] (Tier 2, DPS, NE).

17

All parents in the focus groups mentioned that when they learned about DPP they also learned that it provides financial support to pay for children’s preschool, which motivated families to apply. However, focus group data indicate that many families had an incomplete and inconsistent understanding of DPP and in some cases of preschools overall. Two low-income parents shared that before learning of DPP they did not even know they had to pay for their children to attend preschool. A high-income parent mentioned that the only thing they knew about DPP before applying was that they needed to pick a school in the DPP program to get the tuition credits.

A few parents explored DPP’s tuition calculator where they realized that tuition credits were based on household size, income level, and the school’s Colorado Shines rating, but even with the calculator most families did not know how much financial support they could get. This was even more confusing for parents when trying to understand the different costs for half- or fullday services. One of the parents indicated never seeing any charts or eligibility criteria. The following quote illustrates some of the difficulties families faced when trying to learn how much in tuition credits they would receive. “[I]t also was a little hard to know how much you qualified for, like you can look at the income bracket chart [from Focus group data indicate many families DPP’s Tuition Credit Calculator] and see where you might fall in that chart, but I was like well, is that after DPS tells me had an incomplete and inconsistent how much I'm supposed to pay in tuition or is that before? understanding of DPP And I couldn't quite tell if the credit had been applied until I and in some cases of got my first bill. So that was a little of the confusion that I preschools overall. had, kind of not knowing when the program came in to help.”

(Tier 5, DPS, NW) Only one parent reported figuring out the exact amount of their tuition credit through the DPP calculator, “I actually checked the chart that they have of income versus what you qualify for. … I was surprised because I didn't think we would qualify for any help … but we did qualify for some help still … and honestly the number that I got was exactly what we did qualify for and it worked.” (Tier 5, DPS, NE) The evaluation team conducted GIS mapping of survey responses to the question “Do you know how much money DPP pays for your child’s preschool tuition?” As shown in Figure 1.2, the red, “no” responses to this question were spread across the four quadrants of the city. However, certain neighborhoods in the southwest quadrant, such as Harvey Park and Bear Valley were more consistently red, as were some neighborhoods in the northeast quadrant, such as Congress Park, Elyria Swansea, and portions of Gateway-Green Valley Ranch. The Central Park neighborhood was notably green, indicating families in this area were particularly more aware of the amount of tuition support they were receiving through DPP. Understanding the nuances 18

behind why some of these neighborhoods appear to have a better understanding than others (for instance the potential existence of stronger inter-community communication structures through social media, newsletters, or other community-level communication avenues) could reveal new or different strategies for outreach that DPP might explore to make sure more communities within the city have a higher awareness of the tuition support value they receive.

Figure 1.2 – Responses to Parent Survey Question: Do you know how much money DPP pays for your child’s preschool tuition?

Data gathered by the evaluation team indicates there is still a need for more communication to parents and families – prior to their applying to participate in DPP – of the message that every family who enrolls in DPP will Evaluation data indicate there is a need to receive a tuition credit of some kind, regardless of their income. communicate to parents For instance, several families from high income households were that every family who hesitant about applying because they thought they would not enrolls in DPP will receive a qualify for any financial support and were unsure whether the tuition credit of some kind, time and effort completing the application was worth any support regardless of their income. they could get (“I didn’t think we would qualify for anything, so I kind of felt it was going to be a waste of time and then gathering 19

all the documents felt kind of inundating” Tier 6, non-DPS, SE). A high-income parent commented that the DPP tuition credits they received was the first time ever they had qualified for any type of financial assistance. A few parents explained that if it had not been for the expected financial assistance, they would not have applied to DPP. One of these parents explained that the financial assistance was a deciding factor to apply, “if it wasn’t for the credit, we may have not applied” (Tier 3, DPS, NE). On the other hand, low-income families reported being used to apply and qualify for different types of assistance and did not hesitate to apply to DPP. Some parents participating in the focus groups indicated they were not aware of some information that would have been useful to have when applying to DPP, such as the qualifying criteria to receive tuition credits and a clear timeline for the approval process. The following list describes the parents’ interests for additional information: • A parent mentioned that there was a belief that the tuition credits are available to all ECE children in DPS without an understanding that the financial support comes from DPP and that DPP is an outside organization, independent from DPS, and its funding covers all students. This parent believed that it was important for parents to know where the funding comes from.

• A parent indicated not knowing what DPP’s criteria are to qualify for tuition credits.

• Parents would like to have had access to updated information about DPP preschools, such as a comprehensive list of Parents identified a providers, their location, and the number of spots available to number of topics where more enroll children. Parents complained that they do not know information about DPP where to look for providers, that there is not an easy way to was needed, including find them other than through DPP or the Colorado Parents a comprehensive list of Magazine (print and online formats), and that even the providers and the Colorado Shines website availability information is outdated. number of spots available to enroll Accessing outdated information was a source of frustration for children. parents. One of the parents suggested that providers should market themselves and offer relevant information in community newspapers such as the Glendale/Cherry Creek Chronicle and Front Porch8.

However, the evaluation team’s survey data indicated that community newspapers are not necessarily the most effective way to reach parents and let them know about DPP. • Parents want to know in a clear and timely manner, how much tuition support they would get and how quickly the tuition credit would take effect after submitting their application.

Several parents mentioned submitting their applications to DPP and not knowing what the next steps were.

8

These two community newspapers circulate primarily in the following NE quadrant neighborhoods: Central Park, Cherry Creek, East Colfax, Glendale, Lowry, Mayfair Montclair, and Park Hill.

20

•

One of the parents mentioned not knowing that their child could only get DPP tuition credits for one year. This was especially important because their child’s birthday fell exactly on DPP’s qualifying cutoff day, in a way that they could enroll their child when they were 3 or 4 years old. As this parent expressed, “It would have just been helpful to have understood, oh, if you decide to keep your child in preschool because their birthday is right on the cutoff you won't get funding again.” (T5, non-DPS, NE)

A few parents mentioned that they had all the information they needed at the time they applied to DPP or could have found it on the website or by calling DPP (“I think [DPP] did a really good job providing information.” T3, non-DPS, SE). Some of these parents also agreed that if their decision to enroll their child had depended on hearing about the approval of tuition credits, they would have liked to receive the information in a clearer, more timely and streamlined manner, because they felt that the system of communicating approvals was “kind of strange.” A highincome parent recognized that for people who are not in a privileged financial position, it would be ideal that DPP provides them with better and more timely communication about tuition credits up front because they may not have the financial flexibility to enroll in services they cannot afford without DPP support.

Awareness of Quality Ratings Preschools’ quality ratings were mentioned only a few times during the focus group conversations with parents, even though the ratings are one of the A little over 40% of factors affecting the amount of tuition credits that families received. survey respondents Survey data provide additional insight on this topic. As shown in indicated they did not Figure 1.3, while 41% of survey respondents indicated that they did not know DPP preschools know that DPP preschools receive a quality rating, another 44% receive a quality rating. indicated that they know this fact. Interestingly, almost 60% of highincome families reported knowing this information, while 25% of lowincome families did not answer the question.

21

Figure 1.3. Percentage of families who know that DPP preschools receive a quality rating Skipped 15%

Yes 44%

No 41%

Given that so many families do not know that preschools receive a quality rating, it is not surprising that, as Figure 1.4 shows, 57% of parents who completed the survey indicated they were 57% of parents who completed the unaware of the quality rating of the preschool their survey indicated they were child attended or that only 33% of high-income unaware of the quality rating of the preschool their child attended. families know the rating of their child’s preschool.

Interestingly, 20% of families that were aware of the quality ratings did not know the rating of their child’s preschool. The same families that did not answer the question of whether they knew that DPP preschools receive a rating also skipped answering the survey question asking them if they knew the quality rating of their child’s preschool.

Figure 1.4. Percentage of families who know the quality rating of their child's preschool Skipped 15%

Yes 28%

No 57%

22

Outreach to families Evaluation data from previous years has found that many Denver families with 4-year-old children do not enroll them in DPP. The reasons for not enrolling their children were explored through the evaluation team’s online survey by giving parents a list of 14 reasons and asking them to select those they thought were the most important for parents. Seventy-six percent of parents who completed the online survey selected five primary reasons why they believe families do not enroll their children in DPP, including: 1) they did not know that DPP exists; 2) they believed their income was too high to qualify; 3) they did not know that DPP reduces the cost of preschool by providing tuition credits; 4) they believed that only children enrolled in DPS qualify to receive tuition credits; and 5) that families do not know where to start the DPP enrollment process. As a self-identified Mexican American parent explained during the focus groups, “I don't think a lot of moms in my community actually know about any of this [DPP]” (T2, DPS, NW). Table 1.2 presents additional details on the top five reasons identified in the parent survey for not enrolling in DPP. Table 1.2. Reasons why families do not enroll their children in DPP Top five reasons They do not know that DPP exists They believe they do not qualify for DPP tuition credits because their income is too high They do not know that DPP provides tuition credits to reduce the cost of preschool They believe that DPP tuition credits are given only in Denver Public Schools preschools and not any other preschools They do not know where to start

Proportion of responses 22% 19% 17% 9% 9%

In the evaluation team’s online survey, childcare providers rated the importance of 11 different reasons that families may have for not enrolling their children in DPP. Similar to parents’ perceptions, providers gave the highest importance rating to families not knowing about DPP. An equally high importance rating was given to confusing DPP According to childcare with other financial support programs such as Head Start, which providers, the two most creates confusion with parents believing they are in DPP when important reasons why they are not. The next two importance ratings were given to families do not enroll their children in DPP are families misconceptions/misunderstandings about the benefits that families not knowing about DPP, get from enrolling in DPP and to the inability of families to cover and confusing DPP with the gap between tuition credits received and the total cost of other financial support tuition. Providers also mentioned that some families believe they programs. will not qualify for tuition credits, some because they believe their income is too high to qualify, others because of their immigration status.

23

One parent’s comment offered in the evaluation survey illustrates the misunderstanding about where the preschool funding is coming from, ”Having the help to pay for preschool was such a great advantage for my family. It allowed my daughter to attend preschool and learned so much. I’m very happy DPS offers these kind of credits to low income families because without it I would have never been able to afford it.” (T2, DPS, NE) Of note, two providers, one from NE and one from SW, explained that although the application is seemingly simple, many families are overwhelmed by life, some are experiencing houselessness or high mobility that makes DPP “just one more thing to worry about” (NE provider). This comment resonated with a low-income Spanish-speaking parent who because of her long working hours did not have time to complete the application; it was thanks to her child’s teacher who helped her to complete the application that she was able to enroll her child in DPP. In summary, parents’ and providers’ data collected in focus groups and surveys concur that many families do not know about DPP or that they have misunderstandings about DPP that prevent them from applying to participate in the program.

Parents offered their ideas about how to encourage other parents to enroll their children in DPP. Reflecting on their own experience, they explained that parents need to start thinking about preschools pretty soon after their child is born, and that DPP needs to get the preschool idea into people’s heads as early as possible, especially considering the long waitlists at many schools.

Several parents suggested using the Bright by Text9 program that sends cell phone texts to parents with timely information about parenting and children’s development from pregnancy through age 8. New parents usually register to this program in the hospital when the child is born. Parents suggested that DPP join Bright by Text to send text reminders about enrolling their children and also inserting printed information in the packages that parents receive when they register in Bright by Text. Underscoring the importance that parents need to think early about preschool, a parent suggested that DPP place posters in OBGYN offices or places where parents are getting ready for pregnancy.

To encourage parents to enroll their child in DPP, a parent suggested having someone from DPP, or a DPS partner, visit preschools to explain and walk parents through the DPP registration process. These presentations could take place in August, close to when the school starts, either in the school, a park, or another community place close to the school. Important messages to convey during these presentations would be the benefits to children of early education, the financial assistance available to all families in Denver, and helping parents find placement for the child in a preschool center.

9

Bright by Text is available in English and Spanish to anyone living in the United States https://brightbytext.org/

24

Several focus group participants recognized that even within their own families or close communities, there are parents who are not interested in registering their children in preschool arguing that preschool is useless, that the children are too young, that they will get tired of school because they will be in school their whole life, or that hearing about so many school shootings worries them about what may happen to their child. To overcome this reluctance, participants believed DPP should share more information about the benefits of the programs to children and their families. Benefits that are To overcome parent explained should include research findings about the short-, reluctance to have their medium-, and long-term benefits of early childhood education to child participate in preschool, focus group student outcomes. participants indicated DPP should share more information about research findings on the short- and long-term benefits of participating.

Presentations of information to prospective DPP parents should also include descriptions of some of the observed advantages and benefits to children and families identified by past parents who participated in DPP. This could include any of the benefits mentioned by focus group participants from current or past evaluations, including for example that: 1) the transition from preschool to kindergarten is made easier when a child participates in preschool; 2) preschool prepares the child to be a learner and to know how to be a student and follow the routines of a school and to be comfortable in a school building; and 3) that having a quality preschool experience can set children up for success throughout their school experiences. As one parent said, “the sooner he goes to school, the sooner he’ll adapt and also will learn lots of things.” (T2, DPS, NW). Other benefits shared include the academic learning that takes place in preschool, as well as the development of children’s social skills that were halted because of the isolation caused by the Covid pandemic and the concerns of getting sick. Two parents explained, “[M]y kids were so much more advanced than some other kids that didn't attend the program, just that learning piece … [Because of Covid] a lot of people are afraid to send their kids to school now. But for me, my kids needed that, they needed to go back, they needed that social part, they needed to be around other kids their age.” (T2, DPS, NE) “I know it is something good for him and he will learn … it is better that they are in school, because if they are at home, they only want to be with the tablet, they want to be on the phone, or they get more bored, while in school they learn and play.” (T2, DPS, NW) Focus group participants talked about the families’ financial benefits of enrolling children in DPP, which one parent described as a “huge benefit of having extra money in your pocket.”

They mentioned that all families who reside in Denver qualify for a tuition credit, and even if it’s a small amount, that small amount ends up being quite a lot, “everyone gets something, that’s important to call out” (T5, non-DPS, SE). Families who can afford their preschool and do not 25

need the money can still get DPP’s financial support that can be donated back to the school to support teachers and other families who need the assistance to send the children to preschool.

Another related and very important benefit mentioned is that instead of struggling with daycare or trying to find family members who can watch their child for them, children have access to a school with a teacher where learning takes place. In addition, they said, enrolling in DPP through DPS’ school choice is an easy online process.

One parent mentioned that DPP needs to explore what messages resonate with different communities because the terminology used may not be understood by people from different backgrounds. They explained, “I don't know if ‘tuition credit’ might mean something to some people, but it might not mean the same thing to other people. People just may not understand what that means or have dealt with that kind Some terminology of terminology before so maybe just something around the associated with DPP, such language that's used in there, in their website and their as ‘tuition credit’ might not be understood in the same flyers and things like that and emails would be helpful.” way by people from different (T5, DPS, NW) backgrounds.

Another parent talked about the importance of sharing research that shows the significance of preschool for children, and conveying the message of how Denver stands apart from other cities in the country where parents cannot send their children to preschool because it is so expensive, “just kind of recognizing how lucky we are to live in Denver and having the opportunity to qualify for something like this” (T3, DPS, NE).

Two parents reflected on what they would tell other parents to encourage them to enroll their children in DPP: “Give [the child] the opportunity, and what even better than to do it at an early age when they are open to learning.” (T2, non-DPS, NE) “I would tell them, give your child a chance to learn … Put them in school, give them a step up, let them learn instead of just putting them in a regular day care and let them play all day. [At school] they can play and learn at the same time. [Enrolling them in preschool] is not more work, it is easy … You’ll see what [the child] can do even if it's a half a day. … give them a chance to start in kindergarten, get some knowledge in their brain, a little bit more step up.” (T1, non-DPS, NW) Preschool providers also chose what they considered the most effective strategies to attract families to DPP. Out of a list of 13 strategies to choose from in the survey, interactions with families/friends already in the DPP program (e.g., word of mouth) was the strategy chosen by 26

most providers, followed in order by: face-to-face individual or group interactions with providers, social media (FB, Twitter, TikTok, Instagram), face-to-face interactions with DPP staff/representatives, and printed or electronic communications from DPP (e.g., fliers, emails, texts). Figure 1.5 shows the number of providers choosing each of these strategies broken down by the characteristics of families to be engaged in DPP. Figure 1.5. Number of providers choosing various strategies to attract families to DPP (n=52) From families & friends, word of mouth

Face to face interactions with providers

Social media

Face to face interactions with DPP staff

Printed/electronic DPP communications

0

5

10

15

20

25

30

Number of providers Low-income

Low English proficiency

Traditionally underserved

It is interesting to note the slight difference in effectiveness of these strategies depending on the families to be attracted to DPP, for example, except for face-to-face interactions with DPP staff, all strategies are regarded as highly effective to attract low-income families, with word of mouth having the highest consideration. Face-to-face interactions with providers are also considered a very good strategy to attract the three groups of families. Social media is regarded as slightly less effective with low English proficiency families, as compared to low-income and traditionally underserved families. Printed/electronic DPP communications are considered much more effective with low-income than with low English proficiency and traditionally underserved families.

DPP and UPK Providers’ registration to Colorado's Universal Preschool Program (UPK) started in the fall of 2022, while preschoolers’ applications were open in early 2023. As UPK’s Local Coordinating 27

Organization (LCO) for Denver, DPP’s operations and financial assistance for families were to be coordinated with UPK in support of preschoolers and their families. By the time that evaluation data collection took place, providers had an opportunity to sign the agreement to become UPK providers. On the other hand, although applications also had been open for families, the children who were enrolled in DPP in 2022-23 would not qualify for the financial support because they had completed preschool and were entering kindergarten during the first year of UPK operations. Nonetheless, families shared questions and opinions about UPK during the focus group conversations.

Not surprising for a new program, parents heard multiple and sometimes mixed messages from other families and from the media and had significant questions about what exactly UPK was; how it worked; what type of financial assistance was to be provided, who was eligible and under what conditions; whether families could qualify for DPP and UPK Parents heard at the same time and whether one or two applications would be multiple and required; the relationship between UPK and DPP, whether DPP sometimes mixed was ending or going away, or whether UPK was to replace DPP. messages and had significant questions about what exactly UPK was.

Only two parents in the focus groups mentioned that they had actually applied to UPK and both reported finding the process very confusing. One of them shared that she needed to access too many websites to be able to register her child for the school she wanted. The other parent mentioned not understanding some of the questions in the application or what UPK was about, whether it was to provide additional hours of preschool, or what its overall purpose was. The experiences of these parents were reinforced by another parent’s comment explaining that the enrollment process had been “way more complicated and dreadful to deal with compared to our experience [with DPP one year earlier]… just the people I have talked to seemed to be a lot more stressed than I was about where their kid is gonna go … everything seems to be a lot more uncertain.”

(T5, non-DPS, SW). A parent reported hearing that the UPK roll out had been “a disaster” that had even delayed the school choice process, having a negative impact not only on the families, but also on the preschools that had hired extra teachers for programs that families would not qualify for anymore and maybe having to let go of those teachers.

Other parents mentioned that they had heard that UPK would not cover all the hours of tuition that had originally been promised, and one parent expressed being angry at the Governor for not keeping his promise to low-income families of covering full day preschool for their children. A parent talked about how changes to the original UPK plan made a few weeks before school started took away full daycare from many families who had planned on that, and these families then had a very hard time finding childcare for their children at the last minute. Another parent mentioned hearing that UPK’s initial rollout was difficult in many Colorado towns and she felt “very bad for my friends who don’t live in Denver because DPP just made it so easy to get that 28

assistance and to get the kids learning and ready to go to kindergarten” (T3, non-DPS, SE).

Parents also expressed their gratitude towards DPP saying that families in Denver were protected from UPK’s shortcomings and that DPP covered the gaps left by the state.

DPP providers were more informed than parents about UPK, however, they also had many questions about different aspects of UPK implementation. Overall, providers were pleased that UPK would provide financial support to all Colorado families (not only to Denver residents) which meant that now all preschool children in their centers could be supported by one or both programs. When compared to DPP, however, providers expressed feeling disconnected from UPK, which meant that they did not feel as comfortable providing Providers had specific the type of advice and support they typically offered to families areas of concern regarding during the preschool enrollment process.

whether UPK would require the use of a specific curriculum and whether providers would need to be rated using a different system than Colorado Shines.

Providers understand that DPP is the LCO for UPK, that they are two separate entities and that they could continue their partnership with DPP independently from UPK. Although they recognize that no one has the answers to all the questions, they expressed being confused about UPK while finding the informational meetings they attended with UPK unhelpful mainly because they were not getting answers to the many questions they had. Providers mentioned a couple of specific areas of concern, including: 1) not knowing whether UPK would require the use of a specific curriculum; and 2) whether providers would be asked to be rated with a rating system different from Colorado Shines, which was a daunting idea for providers.

One provider noticed that DPS preschools always have high quality ratings and that is not the case for community centers, and she believed community centers were not recognized as highly as DPS preschools. She was concerned that this difference would mean that UPK would give first matching/assignments of families to DPS preschools and that assignment of children to enroll in community centers would only take place when no slots remained open at DPS sites. In the end, she worried that community centers would not fare as well with UPK funding and while UPK might be good for DPS preschools, it would not be helpful for private community centers.

Providers worried that UPK The matching of families/children to preschools was a major could move children from their area of providers’ concerns. Providers felt strongly about current school and match them having good relationships with the families they served, with another preschool, usually meeting them during the enrollment process before assigning families and children the children started attending preschool. Without to providers where they had no previous relationship. understanding the UPK matching process, providers worried that UPK could move children from their current school and match them with another preschool, assigning families/children to providers where there was no

29

previous relationship. Providers wanted to make sure that the families already in their centers were matched accordingly by UPK. This apprehension was underlined by a provider that made 20 spots available to UPK expecting their own 20 returning families to take the 20 spots, however, 75 percent of the returning families were matched to the center and the rest of the spots were assigned to families unknown to the center; the provider was uncertain about what happened to the remaining families they were expecting to return.

Uncertainty about Uncertainty about how to manage availability of preschool slots was how to manage another problem area for providers with UPK because they indicated availability of that upon entering their agreements with UPK they had to decide on preschool slots was a problem area for the number of part- and full-time spots they would make available to providers with UPK. UPK. Providers did not know whether UPK could penalize the centers for declining matching to families they do not know and then ask for more families to fill in the spots. Only one provider reported having a positive experience with the UPK matching process where two of their non-Denver families insisted they would register with UPK and after making two spots available, the two families were matched to the center; this provider reported that the families’ enrollment and matching process was very quick and both provider and families were very happy with the outcome.

One of the providers shared that two of their own teachers with 4-year-old children decided to test UPK’s enrollment process for families. These teachers found the UPK website very difficult to navigate (e.g., uploading of documents) and the enrollment itself very complicated; these teachers were surprised to find that the enrollment completely depended on the income guidelines and risk factors to access additional preschool hours or days. As a result, they perceived that not having risk factors would be a barrier to UPK enrollment. In addition, these teachers enrolled their children in UPK in February 2023, but would not know what school their child would go to until several months later. A related concern was that because families have high mobility rates, there was no certainty about which school they would be able to attend.

Some providers indicated a concern that UPK’s cut-off dates for matching purposes would penalize homeless families and providers worried that these children would be left out of the system.

An important aspect of UPK implementation is the tuition that the state will reimburse to providers depending on the number of hours per week that children/families qualify for. The evaluation team’s online survey asked providers whether UPK’s annual reimbursement rates would be enough to cover the costs they incurred as providers. The questions were specific to the three different options established by UPK: 15 state-funded hours per week (half-day) for eligible 4-year-old children, 30 state-funded hours per week (full-day) for eligible 4-year-old children, and 10 state-funded hours per week (part-time) for eligible 3-year-old children. They were asked to respond using a scale from 1 = No, it’s not enough, to 5 = Yes, it is more than 30

enough. As shown in Table 1.3, providers that responded to this survey question believed that UPK’s reimbursement rates would not be enough to cover their costs for providing care.

Table 1.3. Providers’ ratings about UPK’s annual reimbursement rates being enough to cover the providers’ incurred cost (Scale: 1 = No, it’s not enough; 5 = Yes, it is more than enough)

Mean Median St.Dev. Minimum Maximum

$6,102 for 15 hrs/week for a 4-year-old 1.6 1 1.076 1 4

$10,749 for 30 hrs/week for a 4-year-old 1.9 1 1.330 1 5

$4,883 for 10 hrs/week for a 3-year-old 1.5 1 0.972 1 4

Providers participating in the focus group sessions agreed that the UPK tuition is not enough to cover their centers’ expenses because UPK has a top limit on the amount of tuition they will cover and, if the center’s tuition is higher, the center has to cover the difference. Providers recognized that the amount of annual tuition UPK will cover is a great benefit for families, but it was viewed as insufficient for providers. One provider explained that, as a large center, they would lose a quarter of a million dollars per year Many providers do if they had to cover the difference between UPK’s reimbursement and not believe that their actual costs. Focus group providers from small and large centers UPK reimbursement rates are sufficient agreed that they could not afford to cover the cost of care gap that UPK to cover their costs would leave uncovered. One provider claimed that UPK should not be for providing care. considered to be universal pre-K if the centers are expected to subsidize their actual costs because of the program. Providers explained that they work extremely hard to pay teachers a living wage to be able to live in Denver, and that most of their businesses were very affected by the Covid pandemic and barely survived, and that it would take them years to become financially viable again. One provider from NW Denver mentioned that “It’s a lot to ask of the community that’s been through so much with Covid.”

Other concerns expressed by providers included: • UPK covers only 9 of 12 months, even for centers that offer year-round programs.

• Lack of clarity about how to stack families’ funding towards their tuition costs.

• Whether funding could become sufficient if they were able to layer funding support through other programs such as CCAP and DPP, in addition to UPK funding.

• UPK’s policy of not allowing providers to charge the difference between the center’s actual cost and the UPK reimbursement was viewed as contrary to providers’ commitment to increase educators’ wages and benefits, which meant that providers felt they could not afford to participate in UPK and also pay teachers appropriately.

31

Talking about the impact on families a NE provider explained, “The number of hours (10 or 15) is what the issue is in combination with the reimbursement rate. The population of families we serve need more than 10 to 15 hours per week as they are working families and the braided funding does not cover the full cost of their tuition. Over 80% of the families we serve qualify for Free/Reduced lunch according to Federal Poverty Guidelines and are unable to cover the difference in their tuition cost. Our agency is a community-based 501c3 and unable to carry the difference for the families. Families will elect to withdraw their students from preschool if there is a [higher] tuition cost associated or not enough hours provided.”

Given all the concerns providers shared about UPK, it is not surprising that 27% of those who completed the online survey felt that there were too many unknowns or restrictions to confidently participate in UPK, therefore, they were not planning to become UPK providers for the 2023-24 school year. On the other hand, 69% of those who completed the survey reported having signed the agreement or being in the process of completing the enrollment to become UPK providers, and another 4% indicated being interested in becoming UPK providers but had not 69% of providers who completed started the enrollment paperwork. A SE provider the survey reported being in the process of completing the expressed that UPK’s roll out has been “incredibly enrollment to become UPK challenging on both the provider and the parent side,” providers, and another 4% while another mentioned that UPK took off a lot quicker indicated being interested in than they should have and that providers were risking becoming UPK providers but had losing more than they would gain if they were to sign not started the paperwork. for UPK at this time. On the positive side, providers recognized that any assistance families receive is wonderful, and another sounded optimistic about how the UPK system was slowly improving and was hopeful that all families would receive the credit without undue delays.

SECTION HIGHLIGHTS – Section 1 of this report included the following highlights: Finding out about DPP • Families reported finding out about DPP primarily from other parents, friends, and neighbors, by asking teachers or administrators at their preschool center, or through the DPP website.

• Many DPP families had an incomplete and inconsistent understanding of DPP and preschools in general, including some not knowing that preschools charge tuition and that DPP pays tuition credits to the preschool provider on behalf of families, and believing that DPP’s financial support is for low-income families only and that highincome families do not qualify for the financial assistance.

32

The application process & awareness of tuition credits • The majority of parents indicated that completing the DPP application was a very easy and straightforward process. Some parents reported not knowing they had applied to DPP at the same time they were completing their preschool enrollment application. Other parents indicated not knowing what they should expect after submitting their application because there was not enough clarity on the “next steps” in this process.

• Providers played a very important role in encouraging parents to apply to DPP, with some of them making the DPP application a requirement for parents.

• Most families found out that DPP offered financial support at the same time they learned about the program in general. However, most families reported not knowing specifically how much DPP financial support they would qualify for or how the tuition credit was calculated. Some families found out the exact amount of tuition credits when they received a bill from the preschool but did not recall receiving communication from DPP with that information.

• Parents would like to have had access to updated information about DPP preschools, such as a comprehensive list of providers, their location, and the number of spots available to enroll children.

Awareness of quality ratings • 41% of families who completed the evaluation survey did not know that DPP preschools receive a quality rating, and only 28% of families know the quality rating of the preschool their child attends.

Outreach to families • Parents believe that the primary reasons why families do not enroll their children in DPP are: 1) they did not know that DPP exists; 2) they believed their income was too high to qualify; 3) they did not know that DPP reduces the cost of preschool by providing tuition credits; 4) they believed that only children enrolled in Denver Public Schools (DPS) qualify to receive tuition credits; and 5) families do not know where to start the DPP enrollment process.

• Providers believe that the main reasons why families do not enroll their children in DPP are: 1) they do not know about DPP; 2) they confuse DPP with other financial support programs and believe they are in DPP when they are not; 3) they have misconceptions/misunderstandings about the benefits they get from enrolling in DPP; 4) they are unable to cover the gap between the tuition credits received and the total cost of tuition; and 5) they believe they will not qualify for tuition credits because their household income is too high or because of their immigration status.

• Families suggested that the most effective methods for reaching out to families to inform them about DPP was through verbal or written communications, preferably from 33

•

their childcare centers, but also from family or friends already in DPP, or DPP personnel.

Families indicated that social media, local radio, and TV news are not preferred methods of communication.

Providers selected the following outreach strategies as the most effective ways to attract families to DPP: 1) interactions with families and friends already in the program (e.g., word of mouth); 2) face-to-face individual or group interactions with providers; 3) social media (FB, Twitter, TikTok, Instagram); 4) face-to-face interactions with DPP staff/representatives; and 5) printed or electronic communications from DPP (e.g., fliers, emails, texts).

DPP and UPK With regard to implementation of the UPK program in Colorado, the evaluation in Year 1 found: • Parents had numerous unanswered questions about what UPK was, how it would interact with DPP, what level of financial assistance would be provided, and to whom it would be provided, whether two separate applications would be required for DPP and UPK, and whether families could qualify for both programs at the same time.

• Providers were pleased that UPK will provide financial support to all Colorado families, however, UPK matching of families to preschools was a major area of concern.

• Providers who completed the evaluation survey, as well as those who participated in the focus groups reported that UPK’s reimbursement rates would not be enough to cover their costs for providing care.

• Despite the challenges with UPK rollout, 69% of providers who completed the evaluation team’s survey reported having signed the UPK agreement or were in the process of completing the enrollment. Another 4% indicated being interested in becoming UPK providers but had not started the enrollment paperwork, while 27% said they were not planning to participate in 2023-24.

In summary, families who participate in DPP usually find out about it through interactions with preschool providers, friends and neighbors, and by accessing the DPP website. Although most families find the enrollment process easy to navigate, they also have a number of significant misunderstandings about the program. Evaluation data found a number of families enrolled in DPP do not know they are receiving tuition credits or the amount of those credits, they also in some cases do not know they actually applied to the program. Families and ECE providers believe the main reasons families do not enroll in DPP are because they do not know of the program, they believe they are already enrolled, they believe they do not qualify, or do not know of the benefits of DPP for their children and families. Families and providers have many unanswered questions about UPK and its relationship to DPP. These findings warrant a targeted communications campaign to further elevate awareness of DPP across the Denver community as well as its role in UPK. Conducting a series of webinars/information sessions with ECE providers could also help clarify misconceptions and increase overall awareness of DPP. 34

2. Customer Service / Interaction with DPP Childcare providers have regular interactions with DPP personnel and their partners (e.g., MetrixIQ) in multiple ways each year. These include when they: 1) apply to become DPP providers; 2) renew their annual agreement with DPP; 3) receive materials for their classrooms; 4) get professional development for their teachers; 5) help families to enroll; 6) report students’ monthly attendance; and 7) receive tuition credits on behalf of DPP families. The evaluation sought to understand through its data gathering the degree to which providers reported that their interactions with DPP were positive and supportive.

Providers participating in the evaluation team’s focus groups mentioned having very positive experiences in their interactions with DPP personnel in several areas: 1) teachers’ professional development and classroom improvement; 2) support for DPP families; and 3) financial support to the preschool centers themselves. Providers mentioned the Providers participating in importance and value of improving teachers’ education through the evaluation team’s DPP-funded professional development as well as receiving focus groups reported classroom materials funded with DPP support, both allowing having very positive experiences in their providers to improve their Colorado Shines’ quality ratings. A interactions with DPP provider from the southeast quadrant of the city shared that personnel.

“Something really remarkable about DPP is that it brings communities together, helping sponsor professional learning communities. [We are] very grateful to be a partner with DPP.” Providers participating in the focus groups were also appreciative of the support that DPP gave to families through what they typically described as a very easy and streamlined enrollment process. They also indicated that DPP responded quickly to requests for information about how much tuition support families would receive. Providers also mentioned how important DPP financial support was for families to afford preschool for their children. One provider shared the case of a family where the mother passed away and the grandmother took guardianship of twin children, which was financially very challenging. The two children qualified for a certain amount of DPP tuition credits, but DPP reached out to the school because the family also qualified for scholarship support through a DPP program that offers a limited number of tuition scholarships that can cover full tuition for three- and four-year old children from low-income households. The provider mentioned that, thanks to DPP, the family felt very supported financially and emotionally during an extremely difficult time in their life.

In terms of the financial support available to centers, one provider explained that the majority of funding the center relied on was from DPP, while other providers explained that DPP allowed them to use the tuition reimbursement to cover overhead costs and teachers’ salaries, which helped them to maintain their teachers in the classroom and keep the center open. One northeast 35

quadrant provider mentioned, “We appreciate all DPP has done for our childcare. I don't know if we could have sustained without their support.”

Providers who completed the evaluation team’s online survey gave very high ratings to different aspects of their relationship with DPP. They rated a series of statements using a scale where 1 = Strongly disagree and 5 = Strongly agree, and gave the following scores to their interactions and communications with DPP and to DPP’s help and support:

Providers who completed the evaluation team’s online survey gave very high ratings to different aspects of their relationship with DPP, typically 4.9 or 4.7 out of a highest possible rating of 5.

4.9 average rating to interactions with DPP: • DPP staff are friendly/approachable.

• DPP staff are professional in their interactions with our center.

4.7 average rating to their communications with DPP: • It is easy for our staff to contact DPP staff.

• I am satisfied with the form/channels of communication we have with DPP. • I am satisfied with the quality of communication with DPP. • I am satisfied with the frequency of communications with DPP. • I am satisfied with the effectiveness of communications with DPP. 4.7 average rating to DPP’s help and support: • It is easy for our staff to get help from DPP for issues related to enrolling families.

• DPP is quick to address any concerns we may have as providers.

• DPP is supportive of us as childcare providers.

Providers must meet several requirements to participate in DPP, Providers must meet several including: 1) licensing; 2) insurance; 3) background checks for all requirements to participate in DPP including: 1) people working in or living in the center’s premises; and 4) licensing; 2) insurance; 3) obtaining a Colorado Shines’ quality rating. The evaluation team’s background checks; and 4) survey data showed that out of these four requirements, only the obtaining a Colorado Shines quality rating slightly discouraged providers from participating in rating. Survey data showed DPP. In relation to the quality rating requirement, a NE provider that only the quality rating requirement slightly wrote the following comment in the survey, “As a newer program discouraged providers from that has not started the rating process it stinks being held to an participating in DPP. introductory rating that seems backlogged and our families are taking the hit for it.” On the other hand, providers mentioned that licensure, insurance, and background checks are important requirements for all childcare centers to meet regardless of their participation in DPP. As explained by one childcare provider from the

36

southeast quadrant of the city, “I think all of these factors are important for providers irrespective of DPP participation.”

The overall assessment of providers’ interactions with DPP was expressed by focus group participants as very positive, including: • “One thing that stands out is their responsiveness.”

• “DPP staff are professional.”

• “DPP is very supportive.”

• “We are happy to participate in DPP.” • “DPP has been great for the center, a really good blessing.”

• “[We are] really thankful to DPP.” • “[DPP] is a 3-way win, a win for the school as they get materials and training, win for parents as they get a break in tuition, and a win for the field of early childhood as quality is improved throughout the city.”

In comparison with providers, parents had much less direct interaction with DPP, with only two parents in the focus groups indicating they had contacted DPP directly on issues related to tuition credits. Both of these parents had different experiences to report: One parent (T5, non-DPS, SE) who completed their child’s DPP application, received confirmation that their application had been received, but stated that they did not hear of a decision about the tuition credits for over six weeks. This parent needed to decide how many days they could afford to enroll their child in preschool based on any tuition credits DPP would grant them. The parent emailed DPP asking about the tuition credits and in response received a letter that had been dated a month earlier indicating the amount of tuition credits approved for the child. When the family received the school bill, it had been adjusted already to account for the DPP tuition credits, which made the parent believe that the DPP notification had been sent, on time, directly to the provider but not to the family. Another parent (T5, non-DPS, NE) also had a direct interaction with DPP when they changed schools after applying for and receiving tuition credits. They reached out to DPP and received a response within one day informing them that the transfer of credits to the new school had been processed.

Another parent could not recall whether they had received a letter from DPP, although they indicated knowing they would get some level of financial support and that eventually they would find out how much it was. It was after their preschool year had started when they found out they would be getting tuition credits effective on a specific date, and that they would receive a credit for the tuition they had already paid to the school.

37

A few parents in the focus groups mentioned that once DPP had approved their tuition credits, the schools had conveniently and efficiently adjusted their monthly bills to reflect DPP’s financial support. These parents were paying the monthly preschool bills but then, around December of 2022, the bills stopped coming and, as one parent recalled, “I remember being really curious about not seeing bills … I just assumed that we had like, paid enough and kind of met the requirement for the year and then we were done. … I didn't want to ask. I just paid the bills when they were required and left it at that” (T3, DPS, NE). Another parent (T3, DPS, NE) with a child in a DPS preschool had the same experience. They Providing parents with called Denver Public Schools and were told there were many faster, more comprehensive changes going on [in DPP] and that she should look for some information on the tuition type of communication arriving in the near future. She reported credits they would be that she never received such communication, and she stopped receiving through DPP could improve overall parent asking about it. A third parent (T5, DPS, NE) mentioned that experiences with the when their preschool bills stopped arriving, she had no idea if program. she still owed any money to the school. These parents found it very intriguing that they were not the only families who had these types of billing experiences.

As these parents’ experiences show, their limited interactions with DPP were mainly positive.

More expedited tuition credit notifications and more comprehensive information on the tuition credits would improve their overall interactions and experience with DPP. One of the parents suggested DPP should consider sending tuition credit notification approval letters electronically (e.g., email) to parents, followed by paper copies, so they would learn about the amount of tuition credit sooner. All the parents in the focus groups appreciated that the preschool providers were also notified by DPP and that the providers discounted the tuition from their monthly bills.

SECTION HIGHLIGHTS – Section 2 of this report included the following highlights: • Parents suggested that more expedited tuition credit notifications as well as more comprehensive information on the DPP tuition credits they receive throughout the year, would improve their DPP experience.

• Parents had very little direct interaction with DPP, but those who did rated their interactions as positive.

• Providers gave very high ratings to different aspects of their relationship with DPP, including: 1) professionalism and friendliness in their interactions; 2) satisfaction with the channels, quality, frequency, and effectiveness of communications; and 3) the support they get from DPP to address any issues or concerns they may have.

• Providers mentioned the importance and value of improving teachers’ education through DPP-funded professional development as well as receiving classroom materials funded with DPP support, which helped providers improve the quality of care.

38

•

Providers must meet several requirements to participate in DPP, including: 1) licensing; 2) insurance; 3) background checks for all people working in or living in the center’s premises; and 4) obtaining a Colorado Shines’ quality rating. The evaluation data showed that out of these four requirements, only the quality rating slightly discouraged providers from participating in DPP.

Overall, families reported minimal, although positive interactions with DPP personnel, and suggested a few changes that would improve their experience. Providers, on the other hand, indicated extensive and very positive interactions with DPP, from receiving funding and curriculum materials, to their professionalism, responsiveness, and effective communications.

Providers indicated that the quality rating requirement is the only requirement that slightly discouraged them from participating in DPP.

3. Tuition Credit System Parents participating in the evaluation team’s focus groups reported that the main benefit of participating in DPP was receiving financial support in the form of tuition credits that made it more affordable for them to send their children to preschool. This was especially true for lowincome families who otherwise could not afford the cost of preschool or could afford only a limited number of hours. The economic impact of the DPP tuition credits was also important for middle- and high-income families, although it manifested in different ways, such as adding weekly hours of preschool or after school care, choosing schools that matched better the needs of the child and family, or being able to pay for other services with a greater peace of mind. While having their child in preschool was not optional for parents who worked full time, having children attending preschool gave parents, mostly mothers, time to obtain employment, increase their working hours and family income, or maintain their full-time jobs. These benefits were strong incentives that encouraged families to enroll their children in preschool.

Parents described how DPP tuition credits allowed them to enroll their child in preschool and increase their work and family income, as well as to compensate for lost income that was sometimes a consequence of the Covid-19 pandemic: “[Lo mejor para mi familia de participar en DPP es] que DPP me haya dado la oportunidad de registrar a mi hijo para que sí continuara la escuela porque tal vez si no hubiera tenido el apoyo financiero pues no lo hubiera llevado por simplemente no tener el dinero para pagarle la colegiatura.” [The best for my family of participating in DPP is] that it gave me the opportunity to register my child to continue going to school, because without the financial support maybe I couldn’t have taken him simply because I don’t have the money to pay tuition.] (T2, non-DPS, NE)

39

“I think [DPP] helped my family dramatically because if I didn't get the assistance I couldn't afford to send them [to preschool]. And if they wasn't going, I wasn't able to work the way I work, and it also allowed me to not worry about if they could go or if they can’t go, if the tuition is paid, if it's not paid, then they can't go. And if I'm working and they can't go, I can't go to work because I have to stay home and watch them.” (T2, DPS, NE)

Parents in focus groups described how DPP tuition credits allowed them to enroll their child in preschool and increase their work and family income.

“I don't think my girls would have gone to preschool because I wouldn't be able to afford both of them to go to preschool. They would have been at home, learning preschool from their mommy, and I don't think that'll be okay… They would have been at home, they wouldn’t have been at school at all.” (T1, non-DPS, NW) “For me [DPP] was great because I could pick up more hours at work, go to work. … I think if we didn't have it [the tuition credits], she wouldn't have gone to preschool, definitely. Yeah, cause there's no way I could afford anything like that for her.” (T2, DPS, NW) “Pues sí [DPP me ayudó] porque pues no tiene uno que estar pagando, como ahora que con eso de la pandemia se bajó mucho el trabajo, pues no tienes que preocuparte por pagar la colegiatura. Sí, porque ese era como el miedo en el 2021 que yo decía, ¿y luego si me cobran? O sea que se bajó el trabajo y yo quería mandar al niño y pues sí me ayudó mucho.” [DPP helped my family] because we didn’t have to be paying tuition, as it was during the pandemic when work slowed down very much, but you don’t have to worry about paying tuition. Yes, because it was that fear in 2021 that I’d say, “and what happens if they charge me for tuition?” So, the work slowed down and I wanted to send my child [to preschool] and [DPP] really helped me very much.] (T2, DPS, NW) “[DPP] gave my wife an opportunity to step into looking for work which will be really important.” (T3, DPS, NE) 58% of family survey

The evaluation team’s survey asked parents if they would be able to respondents indicated send their child to preschool without receiving the DPP tuition they would either not be credits. Fifty-eight percent of families indicated they would either not able to afford preschool or that it would be be able to afford preschool or would struggle to afford preschool. financially difficult Sixty-one percent of low-income families that responded to this without DPP’s tuition question indicated they would not be able to afford preschool, credits. representing 81% of families who made this statement. Forty-five percent of high-income families indicated they would be able to send their child to preschool, but 40

it would be financially difficult, representing 63% of families who made this statement. Details of parents’ survey responses are presented in Figure 3.1., while Figure 3.2 shows GIS mapping of the same responses. In this map, concentrations of “no” responses shown in red can be seen in the southwest quadrant of the city and in the northeast, particularly Montbello, where there are higher concentrations of low-income families, while “yes” responses are dominant in higher income areas such as Washington Park, West Highland, and Sloans Lake.

Figure 3.1. Proportion of families broken down by ability to send their child to preschool without DPP's financial support (n=382) Skipped 11%

Yes, not a problem 27%

I do not know 4%

No, I would not be able to afford preschool 29%

Yes, but it would be financially difficult 29%

Figure 3.2. Parent survey: Would you be able to send your child to preschool without the DPP tuition credits you receive?

41

The evaluation team’s survey also asked parents whether DPP’s tuition credits allowed them to afford more hours of childcare. As shown in Figure 3.3, nearly 50% of all families, which included 55% of low-income families, indicated that they were able to afford more hours of childcare with DPP support. On the other hand, 39% of families, including 47% of high-income families, said the DPP tuition credits did not allow them to afford additional hours of childcare. A total of 12% of respondents skipped the question.

Figure 3.3. Proportion of families able to afford more hours of childcare with DPP support (n=382) Skipped 12%

Yes 49% No 39%

Of those families who indicated they were able to afford additional hours of childcare with DPP support, 45% were able to afford between 1 and 5 additional hours of childcare, 22% afforded between 6 and 10 hours, 19% afforded 15 or more hours, and 9% afforded between 11 and 15 additional hours of childcare. Of those totals, 25% of low-income families stated they were able to afford 15 or more additional childcare hours, and 27% of high-income families stated they were able to afford between 6 and 10 additional childcare hours with the help of DPP’s tuition credits. Figure 3.4 shows the distribution of number of hours afforded with DPP’s financial support.

42

Figure 3.4. Proportion of families broken down by number of additional childcare hours afforded with DPP suport (n=382) Skipped 5% More than 15 19% Between 1 and 5 45% Between 11 and 15 9%

Between 6 and 10 22%

According to parents participating in the focus groups, DPP tuition credits gave families breathing room to deal with the high cost of early childhood education. For example, although high-income families reported receiving a modest credit compared to the very high cost of preschool they were paying, the most common benefit they mentioned was the addition of preschool hours. Other benefits of DPP’s financial support reported by parents included selecting the school that best fit their child’s needs, and just alleviating the financial burden of preschool tuition: “Less stress having to worry about paying the bills. Being able to add more days so I can work more hours. The child going for more days, for the whole week instead of three [days a week].” (T1, DPS, SE) “[DPP] just took a bit of the stress off the financial commitment to pre-k and being able to go to the amount of days that we wanted to do for that pre-k year, significantly lessening the financial burden for our family. … I'm just so thankful for the opportunity to send my kids to the preschools that fit them so perfectly, whereas I probably would have just found something else that fit my budget.” (T3, nonDPS, SE) DPP tuition credits gave families breathing room to deal with the high cost of early childhood education.

“[T]he money aspect, … preschool, childcare is like a mortgage payment or … [like a] monthly rent, it's expensive and you feel it, and you're counting down the days until your kid can be in kindergarten and first grade so you don’t have to be paying that monthly fee. I mean, [DPP] helps, for sure, and especially when your kids are young and money is tight and childcare is hard to come by. … I'm such a

43

firm believer in public education and want my kids to experience that. So we would have made it happen, but [DPP’s tuition credits] helped for sure.” (T6, DPS, NW) “It was definitely the financial aspect, really, [that] made it possible for us to send the kids to preschool. My husband and I work opposite schedules so that we don't really have to pay for childcare … we get a little help from family here and there, but it's being able to have that credit was a tremendous help to us.” (T5, DPS, NW) “We would have [been able to send the child to preschool without DPP’s financial support], but it would have definitely made a difference in our budget. I mean, we're lucky to be able to do that … we would have found a way and it would have happened, but this certainly helped to be able to do that, especially once it got into full days.” (T5, DPS, NW) “Instead of two days [a week], we added a full extra day” (T5, non-DPS, NE) “We were able to send my child to preschool 4 days a week instead of the 3 days a week we had before the tuition credits; [with the DPP credit] the net cost at the end of the month was the same” (T5, non-DPS, SE) “The tuition credits helped us decide to get 5 mornings a week instead of 3 mornings a week.” (T5, non-DPS, SE) Across income levels, parents’ comments highlighted how DPP’s financial support allowed them to continue working or to afford services their children needed, encouraging them to enroll their children in preschool: “[W]e were kind of on the fence about sending our son [to preschool] just because it was going to be a financial stretch to do it, and … our preschool wasn't paid for entirely, but we got some help with it and that was enough to make it so we could swing it without being totally broke. … the little bit of financial support that we received made it so we felt comfortable doing it. It was definitely his speech that was our biggest concern and DPP definitely gave us what we needed to get him into preschool, and it has made a huge difference. Because of DPP’s help we were able to afford speech therapy for my son” [speech problem established in an IEP in the DPS preschool]. (T5, DPS, NE) “For us any credit that we didn't have to pay for childcare we were using for our mortgage or groceries … so I think ultimately it was really, really helpful. And we also paid for after care costs since both of us work, so that helped alleviate basically all of those costs last year with the DPP credits, so that was huge, so we were able to stay working.” (T5, non-DPS, NE) 44

“[DPP] ayuda también a animar a las familias a querer inscribir a los niños a temprana edad … y entonces también puede uno aprovechar el tiempo para trabajar y crear otro ingreso a la familia, entonces pues sí, sí ayuda mucho.” [DPP] helps to encourage families to want to enroll the children at an early age … and we also can take advantage of the time to work and create another income in the family, therefore yes, it helps a lot.]

(T2, non-DPS, NE) 60% of DPP families – “[A] reason why we were really interested in the DPP including 70% of lowprogram was because there were no other affordable income families– indicated options that we knew about nearby. So, qualifying for the that access to DPP tuition credits was a big motivator and made it possible. … I credits encouraged or think there's a good chance that [the children] might not somewhat encouraged them to send their child to have been in preschool had we not qualified for tuition preschool. credits. … I think it really would have depended on what the cost of preschool would have been without the credit.

But I feel that with 4 kids and housing prices in Denver, having to pay full price for preschool would have made it completely unaffordable. Like we would not have been able to have our kids in preschool. (T3, DPS, NE)

Parents who completed the evaluation team’s survey also reported how much the DPP tuition credits encouraged them to send their child to preschool. These data are presented in Figures 3.5 and 3.6. Figure 3.5 shows that 60% of DPP families –including 70% of low-income families– indicated that access to the DPP tuition credits encouraged or somewhat encouraged them to send their child to preschool, while 28% of survey respondents indicated that the tuition credits did not encourage them at all to send their child to preschool –43% of high-income families represented 71% of all families who answered this way.

Figure 3.5. Proportion of families encouraged by DPP's tuition credits to send their child to preschool (n=382) Skipped 12%

Yes 40% Not at all 28%

Somewhat 20%

45

When parents’ survey data are plotted by Denver quadrants, the findings show a distinct geographic distribution of families encouraged or not encouraged by DPP’s tuition credits to send their child to preschool. Figures 3.6 and 3.7 show the families’ geographic distribution by how encouraged they were by the tuition credits, and the families’ geographic distribution by income tiers, respectively.

The evaluation team conducted GIS mapping of the locations of survey responses, by neighborhood, to the question: “Did DPP tuition credits encourage you in any way to send your child to preschool?”

Figure 3.6 shows GIS mapping of the locations of survey responses, by neighborhood, to the question: “Did DPP tuition credits encourage you in any way to send your child to preschool?” The map provides color coding to show whether responses to this question were “yes,” “not at all,” or “somewhat.” As the map shows, there was a dispersion of responses across neighborhoods. However higher concentrations of “yes” and “somewhat” responses to that question can be seen in the Southwest and in the Northeast quadrants (in particular Montbello) of the city. These are areas shown in Figure 3.7 to be predominantly lower income tier areas of the city.

Figure 3.6. Parent survey: Did DPP tuition credits encourage you in any way to send your child to preschool?

46

Conversely, the mapping shows concentrations of “not at all” responses to the question in neighborhoods such as Washington Park, Washington Park West, West Highland, and Sloans Lake, which Figure 3.7 shows as higher income areas. This indicates, perhaps as expected, that the tuition credits had less of an impact on the decisions made by higher income families, even though other data gathered through the evaluation revealed how appreciative these families were to receive the tuition credit support, and that it helped reduce their stress overall.

Figure 3.7. Geographic distribution of families receiving DPP tuition credits by income tier

The GIS maps shown provide a useful reference when considering the differing economic and other characteristics associated with different quadrants and neighborhoods across the city.

Close to 90% of parents who completed the evaluation team’s survey knew that DPP provided money to their child’s preschool to reduce the tuition cost they paid, as shown in Figure 3.8. Of the remaining 11 percent of families who were unaware that DPP was paying to support their child’s tuition costs, almost three quarters (74%) were from low-income households. The comment of a Spanish-speaking parent in the evaluation survey highlights the lack of knowledge of who paid for their daughter’s tuition, and that the tuition is directly paid to the providers on behalf of the families: “No he recibido ninguna ayuda del dpp... solo no me cobraron el curso de mi hija en la escuela” [I have not received any help from DPP... they [the preschool] just didn't charge me for my daughter's course in the school] (T2, DPS, NE) 47

It is possible that a factor contributing to this lack of knowledge of who was covering the tuition cost was that many childcare providers who serve low-income families required them to complete the DPP application alongside with applications to other sources of financial aid, and that these parents were not specifically informed that DPP was one of the sources of support.

Figure 3.8. Proportion of families who know that DPP provides money to reduce the tuition costs that parents pay (n=382) Skipped 3%

No 11%

Yes 86%

More than half of parents who completed the evaluation team’s survey did not know how much money DPP paid to cover their child’s preschool tuition costs, as shown in Figure 3.9. Sixty-eight percent of low-income families did not know how much of their child’s tuition cost was paid by DPP, while only 36% of high-income families did not know the amount.

Figure 3.9. Proportion of families who know how much DPP pays to cover the child's preschool tuition costs (n=382) Skipped 3%

Yes 44% No 53%

48

Childcare providers participating in the focus groups shared some ideas of what they believe may prevent families from having equal access to quality preschool for their children. Their comments addressed two of the factors that DPP uses to determine the tuition credits that families receive: 1) the income of the family; and 2) the preschools’ quality rating.

There was confusion among some providers regarding whether DPP tuition credits are available to all families regardless of income. A few providers in focus groups indicated that DPP’s family income guidelines prevented some families from getting DPP tuition credits, even when living in Denver. Two providers, one from the southwest and the other from the northeast quadrants, indicated that there had been cases of Denver families that There was confusion applied to DPP but were rejected because their household income among some providers was too high. Their comments were as follows: regarding whether DPP “I have examples of families who have been rejected tuition credits are available to all families because they make too much money and they don’t get regardless of income. any money, not even a little amount” (NE provider) “There are two families who say they make too much money, they said they completed the application and were declined. I have seen the DPP letters that say they don’t qualify because they make too much money. That is a printed letter from DPP.” (SW provider) Other providers in the focus groups explained that all families that live in Denver qualify for DPP’s tuition credits, regardless of their income: “I’ve never had a family rejected by DPP, they all get a tuition award, and then they pay the difference; even families who don’t want to declare their income or those who have a high income get something.” (NW provider) “Everyone who applies get DPP tuition, those who make more money get less tuition credit, same for those who do not give all their information, but have never had anyone declined because of income” (NW provider) It is possible that providers who indicated that some families did not get DPP tuition credits because their income was too high, may have been talking about families applying to DPP’s Scholarship programs that have family income requirements. This misunderstanding warrants a review of the different programs offered by DPP and the information that is shared on these programs with DPP providers.

Another factor mentioned by providers as a possible barrier to families having access to higher tuition credits was the childcare center’s quality rating. These providers indicated that given that there are limited numbers of providers with quality rating of 4 and 5, many families are 49

prevented from getting higher DPP tuition credits. Providers mentioned that not all providers have the capacity or interest to get the highest rating possible, and that families are impacted by this. One provider explained: “Having the rating is kind of an incentive for [providers] to be motivated to get a higher rating, or to get those coaching hours. But that is on the Center, not the families, and the families miss out, it is a hindrance because families cannot do anything about the rating of the school, that’s up to the school, and maybe that’s why it’s done, as an incentive to create higher quality, but the way it is set up now is a hindrance for families.” (NW provider) Providers in the focus groups explored different options to address the preschool’s quality rating issue, one of them suggesting that DPP remove the rating system from the calculation of tuition credit amounts and to instead calculate tuition for families based only on their income, household size, and the hours the child attends school (e.g., part- or full-time attendance). These providers suggested that if UPK is not requiring any ratings as part of its Providers in focus program, then DPP should do the same. They commented: groups explored “Give the tuition award based on income, household size different options to and the hours for any child who lives in Denver County to address challenges associated with the have access to go to an ECE program that has a contract current quality rating with DPP.” (NW provider) system.

Providers considered that although this idea sounded attractive, they were also conscious that DPP needed to have some type of accountability system and could not just give money to centers where children were not learning or where there were issues with the quality of the program. They agreed that DPP needed to know that the centers in the program were investing the money in the children, in the curriculum, and in the school environment.

Some providers who had achieved high ratings indicated that DPP had been an important incentive for them to undertake that process. One such provider for instance stated that: “Our center has worked very hard to reach a level 5 rating, to improve the quality of the center. We have used the money to help families, educate the teachers, so we are a quality center. The higher tuition from DPP has been an important incentive to keep a high rating and get the money we need.” (SW provider) Another provider wondered how the decision was made about giving more money to centers with higher ratings, instead of giving greater financial support to low-rated centers so they could improve.

One provider from NW Denver raised the issue of childcare deserts and low-income families as important conversations that need to take place:

50

“The issue is about families who live in low-income neighborhoods, in childcare deserts where families need full tuition support, [those families] are really in need of the tuition support, if they are asked to pay one dollar [to cover the tuition gap], they cannot pay that, that is how desperate it is to have access and give them as much tuition support as possible.”

Providers who completed the evaluation team’s survey were asked to rate the four factors that determine the amount of tuition credit per child given by DPP to ensure funding support is provided for low-income families. Figure 3.10 shows the ratings from all providers together as well as broken down by city quadrant. Data show similar ratings for three of the four factors: number of hours the child attends preschool, number of people in the household, and family income. However, use of provider quality rating as a factor had slightly lower ratings from all but the SW quadrant. NW and SE providers gave their highest rating to family income.

Figure 3.10. Providers' ratings of importance of factors to determine amount of tuition credit per child to support low-income families (n=52)

Family income

Number of people in the household

Number of hours the child attends preschool

Provider’s quality rating

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Rating scale: 1 = Not important at all; 5 = Very important Total

NE

NW

SE

SW

Survey respondents suggested a few additional factors that could be taken into consideration to determine the amount of tuition given to families, including, family composition (e.g., single parent, number of income-earners in the household), families with high needs (e.g., families under temporary hardships), and families with children with risk factors (e.g., IEP, special needs). The provider survey included another question to rate the importance of three additional factors to be included to determine the amount of tuition credit per child: Family composition 51

(e.g., single parent with 2 children vs dual parents with 1 child, or number of children vs number of working adults), adequate vs scarce supply of childcare providers where the family’s residence is located (e.g., childcare deserts), and revising the model that determines the provider’s quality rating.

Figure 3.11 shows the ratings from all providers together and also broken down by city quadrant.

Data show that family composition was rated the highest, followed by supply of childcare providers, and revising the quality rating model. Of note, providers from the southwest quadrant gave a much higher rating to family composition and supply of childcare providers, than providers from the other quadrants. Also worth noting, providers from the southeast quadrant gave a slightly lower rating to the supply of childcare providers as a possible new factor in the determination of tuition credits for families.

Figure 3.11. Providers' ratings of importance of other factors to determine amount of tuition credit per child (n=52)

Family composition

Supply of childcare providers close to family's residence

Revising the model that determines the provider’s quality rating 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Rating scale: 1 = Not important at all; 5 = Very important Total

NE

NW

SE

SW

Finally, providers who participated in the focus groups were asked to reflect on what factors are driving the change in DPP families’ demographics, with increasing numbers of high-income families and decreasing low-income families. Providers offered three possible explanations for such demographic tendency: 1. Families do not know about DPP. Before the Covid-19 pandemic providers said DPP participated in showcases, fairs, open houses and community events in different locations in the city easily accessible through public transportation. Marketing strategies were used to attract families to attend (e.g., DPP will distribute posters to providers), and families would learn about DPP and be encouraged to apply and could get some type of incentive such as tickets to the zoo or a museum for attending the events or enrolling their children in DPP. According to providers, such events gathered significant numbers of low-income 52

families and community members and were very successful. Providers mentioned they are willing to support DPP in engaging families and the community and suggested that DPP continue and possibly expand outreach efforts to let families of all income levels know about the resources DPP can provide to support enrollment of their children in preschool.

2. With the pandemic, families got used to keeping their children at home, and some providers believed it is likely that, as this occurred, families found other support systems outside of the preschools. For some low-income families, finding such alternative support systems could mean that they have gotten used to supporting the child less expensively and with less “hoops” to have to go through even though such requirements could be beneficial. One provider explained, “School systems require a lot, for example, if the child has some type of development delay, we are telling parents, rightly so, helping them, to get their kid on track, but now it’s one more thing [for the parent] to think about, along with how to pay their rent, along with whatever else they had, we are saying, now you have to take your kid to special ed or something like that. They would rather have a home support system than to worry about putting them into schools, especially if they don’t get full tuition, then it has to come out of their pocket instead of paying something else. That’s one more thing and too much on their plate.” (SW provider) 3. Demographics of the city are changing, overall school enrollment is declining, especially in low-income areas. Gentrification is occurring in numerous neighborhoods according to some providers who indicated that low-income families may be moving away because they can no longer afford to live in the city, with rising housing costs. Some providers further indicated that at the same time that they are experiencing declining enrollment they have reduced their hours of operation due to a shortage of labor and not being able to meet the needs of the families, resulting on fewer families served overall.

SECTION HIGHLIGHTS – Section 3 of this report included the following highlights: • DPP tuition credits have an important economic impact on parents and families. Parents in focus groups reported the main benefit of DPP was receiving tuition credits that made preschool more affordable. This was especially true for low-income families.

• The economic impacts of DPP tuition credits were also important for middle- and high-income families, allowing them to add additional hours of preschool or after school care, or to choose schools that better matched better their child’s specific needs, or to pay for other needed services with greater peace of mind.

• 58% percent of families indicated they would either not be able to afford preschool or would struggle to afford preschool without DPP tuition credits.

• GIS mapping of survey responses to the question: “Did DPP tuition credits encourage you in any way to send your child to preschool?” showed a dispersion of responses across 53

•

•

•

•

•

•

•

neighborhoods. Higher concentrations of “yes” and “somewhat” responses could be seen in the Southwest and Northeast quadrants (in particular Montbello). These are predominantly lower income areas of the city. Conversely, the mapping showed concentrations of “not at all” responses to the question in neighborhoods such as Washington Park, Washington Park West, West Highland, and Sloans Lake.

Having children attend preschool gave parents more time to obtain employment, increase their working hours and family income, or maintain their full-time jobs. These benefits were strong incentives that encouraged families to enroll their children.

Nearly 50% of all families indicated they were able to afford more hours of childcare because of DPP’s tuition credit support. This was true for both low- and high-income families.

About 86% of parent survey respondents knew that DPP provided money to their child’s preschool to help reduce tuition costs, while 11% were unaware of this. Almost three quarters of those who were not aware of this were from low-income households. One factor contributing to this lack of knowledge might be that childcare providers who serve low-income families often require them to complete the DPP application alongside applications to other sources of financial aid, and these parents are not specifically informed that DPP is one of the sources of support.

More than 50% of parent survey respondents did not know how much money DPP paid to cover their preschool tuition costs. Sixty-eight percent of low-income families did not know the amount paid by DPP, compared with 36% of high-income families.

Data gathered by the evaluation team indicates there is still a need for more communication to parents and families that every family who enrolls in DPP receives a tuition credit of some kind, regardless of income.

There was confusion among some providers regarding whether DPP tuition credits are available to all families regardless of income. This misunderstanding warrants a thorough review of the different scholarships offered by DPP and the information that is shared about these programs with providers.

A survey asked providers to rate the importance of additional factors to determine the amount of DPP tuition credit per child. Data showed that family composition was rated the highest factor to consider, followed by supply of childcare providers, and revising the current provider quality rating model.

It is clear that DPP tuition credits have important and positive effects on families. The degree of effect can vary by income level, with lower income families more likely to respond that they would not be able to send their child to preschool without the credits. However, a mix of both higher and lower income families indicated DPP tuition credits enabled them to afford more hours of childcare. More efforts are needed to ensure that parents are aware of the actual amount of tuition credits they receive, and that there is higher awareness that all Denver families, regardless of income, are eligible to receive DPP tuition credits.

54

4. Choosing Preschool The evaluation team gathered data through a family survey and a series of focus groups to answer several key questions on the topic of choosing preschool, including: • What factors influence parent decisions when choosing a preschool for their children? Do these factors vary by parent characteristics?

• What are the primary challenges and barriers to accessing preschool?

• Do parents feel that their preschool provider offers services that are accessible and culturally responsive?

Parents described different experiences enrolling their children in preschool. For some, the process was relatively easy, with the process involving identifying the school they wanted, inquiring about availability, and moving on with the enrollment. Other parents went through the DPS school choice process and were able to register their child in their first or second preschool choice. However, other parents reported feeling the process overwhelming at times, especially because they could not find availability for the type of services they needed such as half-, full-, or extended-day, the number of days of the week, or a provider that was a good match for their family and child. A parent explained why making decisions about preschool was difficult, “There is so much information to process at that time that it feels overwhelming at times with all the different choices.” (T3, non-DPS, SE). While another parent offered a comment in the evaluation survey that shed additional light about her experience: “I think it is hard for families to find information about DPP and busy families don't have the time to search multiple places for program information. It is also hard to find reliable childcare (many centers have long waitlists) and I know of families that didn't apply to DPP because they were concerned they would lose the spot they have at a daycare center. The additional/new application from DPS made it more confusing for families.” (T5, DPS, NE) The family survey sought to understand if families experienced being priced out of their preferred preschool or if they experienced limited availability of their preferred preschool. The survey also aimed to uncover families' perspectives on the current supply of quality childcare centers in their area. Survey questions and focus groups helped to shine the light on a variety of these family perspectives.

The following four topics pertaining to how families choose a preschool for their child are addressed in further detail below: 1. Affordability 2. Accessibility 3. Quality 4. Cultural responsiveness.

55

Affordability The survey found that 47% of families found preschools they liked but were too expensive for them (see Figure 4.1). More than half of high-income families (53%) and 41% of low-income families shared this experience. Close to one fifth (18%) of low-income families opted to skip this question, representing 59% of all families who opted to skip this question.

Figure 4.1. When you were looking for a preschool for your child, did you find any preschools that you liked but were too expensive for you?

Skipped 12%

Yes 47%

No 41%

In the focus groups, families shared how expensive not only preschool itself is once a child is enrolled, but also that the process of finding a preschool can be expensive itself due to the deposits required to place their child on a waitlist. As one parent mentioned, “This is EXPENSIVE, and if you don’t end up going there, you don’t get the money back. This was an expensive thing, deposits are $50-100/child. You lose the deposits on ones you don’t use!” (T5, non-DPS, NE) The fact that almost half of all respondents to the parent survey indicated that they found preschools they were interested in for their child but they could not afford, underlines the continuing challenge that the City of Denver faces in terms of general accessibility of affordable preschool for all children.

Accessibility To understand the accessibility of preschool services in Denver, the evaluation team analyzed the total number of families (4,359) in DPP, by quadrant of the city and compared these numbers to

56

the total number of DPP providers (school- and community-based) available in each quadrant.

These numbers are shown in Table 4.1 below.

Table 4.1. Percentage of families and DPP providers across Denver quadrants Quadrants Families DPP Providers10

NE 51% 44%

NW 14% 20%

SE 18% 19%

SW 17% 17%

As shown in Table 4.1, some of the quadrants in Denver are in relative balance in terms of the percentage of DPP families in the area and the percentage of DPP providers (e.g., the southwest quadrant represents 17% of the families in DPP and contains 17% of DPP providers). Other quadrants, however, are less in balance, for instance, the northeast quadrant represents 51% of the families in DPP but contains only 44% of providers.

Survey data collected from families found that only 36% of families believed there are enough preschools to choose from close to where they lived (see Figure 4.2), and nearly 20% indicated they did not know whether there were enough preschools to choose from. This challenge appears to be the case for both low- and high-income families, with only 41% of low-income and 31% of high-income families believing there are enough preschools to choose from close to where they live.

Figure 4.2. Do you think that there are enough preschools to choose from close to where you live?

Skipped 11%

Yes 36%

I don't know 19%

No 34%

This finding is particularly critical since in focus group discussions, parents reiterated that the location of a preschool and its proximity to where they live was a highly influential factor when 10

School-based providers were located across the city in the following percentages: NE (42%); NW (23%); SE (14%); SW (21%). Community-based providers were located across the city in the following percentages: NE (46%); NW (18%); SE (22%); SW (14%).

57

choosing a preschool, especially for low-income families. Several families stated that there are not enough preschool centers to serve children in their area for the necessary number of hours, while other families stated that they did find preschools they would have liked to enroll their child into, but the availability of space was often a limiting factor. One parent explained that the only reason they would hesitate about enrolling their child in preschool was if they could not find availability close to home, “Some times all the sites are filled up. The closest to my house might not have an opening and that would be the only reason I would be discouraged [to enroll their child in preschool]. Sometimes they have openings, but the opening might be like a 20-minute drive from my house instead of something nearby.” (T1, DPS, SE) The evaluation team conducted GIS mapping of the question that provided those data, “Do you think that there are enough preschools to choose from close to where you live?” Figure 4.3 shows the results of this mapping analysis. While the number of survey responses relative to the total population of DPP families in general can limit the generalizability of findings, they can still be instructive of potential differences in parent perceptions and experiences across specific neighborhoods and quadrants of the city.

Figure 4.3. Responses to parent survey question: Do you think that there are enough preschools to choose from close to where you live?

58

As shown on the map, survey responses from the southwest quadrant of the city appear to support the data contained in Table 4.1, which showed that the percentage of families in DPP aligned with the percentage of DPP providers in that quadrant (17%). That is, the survey responses in Figure 4.3 show parents in the southwest quadrant of the map tended to answer “yes” to whether there were enough preschools where they lived.

The northeast quadrant responses somewhat reflect the data, in that there is a mix of “no” responses, potentially reflecting the fact that the northeast quadrant is more out of balance in terms of the percentage of DPP families served and percentage of DPP providers located in the quadrant. However, there are also a reasonable amount of “yes” responses to the survey question.

More investigation is needed in future evaluation years specifically focused on these city regions.

Interestingly, the northwest quadrant of the city showed a larger number of “no” responses than “yes” responses, even though data (Table 4.1) show the NW region holds a larger percentage of DPP providers relative to the percentage of DPP families served. More investigation into this phenomenon in future evaluation years can shed light on the challenges that parents face in each region in terms of having sufficient choice of preschools close to where they live.

The survey found (Figure 4.4) that nearly half (48%) of families found preschools they liked but did not have available spots for their child. Sixty percent of high-income families had this experience, representing 51% of all families who experienced this situation, while 35% of lowincome families shared this experience, representing 28% of all families who had this experience.

Figure 4.4. When you were looking for a preschool for your child, did you find any preschools that you liked but they were full and not accepting more children?

Skipped 11%

Yes 48% No 41%

59

In the focus groups, parents reiterated that the real challenge was finding a preschool with availability. Many parents discussed the challenges of navigating a waitlist or going through the DPS choice process. Some parents explained that this system can In focus groups, be increasingly difficult when their older child is already enrolled parents shared a in a center, but they are not sure if their younger child will also significant set of have a spot at the same center. Additionally, families expressed challenges associated challenges with finding full-, half-, or extended-day programs. with finding an appropriate preschool While some families needed full- or extended-day programs due to with availability. their work schedules, other families strongly preferred half-day programs for the experience of their child. Both sets of families shared frustration in finding programs with openings that worked well for their families. One parent explained their frustration in trying to find the preschool for their child, “I would say the number one challenge was hours because when we work full time and a preschool is open 9 to 1 or 9 to noon, I don’t know how you are supposed to pick up your child in the middle of the day … I work from home now and I have more flexibility and we thought if we needed to make this work we could, but it really limited the options of what was available, when it’s just half day programs, so our focus was more on who can be a full time daycare and that would serve the age range that we need and less about Colorado Shines rating and less about tuition, it was about we need this to make this work, because I can’t work part time and I can’t pick up The evaluation team the kid at 1 o’clock.” (T5, non-DPS, SE) The evaluation team conducted GIS mapping of the survey question: “When you were looking for a preschool for your child, did you find any preschools that you liked but they were full and not accepting more children?” Results of this mapping are shown in Figure 4.5. While a mix of “yes” and “no” answers can be seen on the map, higher concentrations of “no” answers are observed in the Far-NE neighborhoods, and higher concentrations of “yes” answers show in the NW quadrant and in the southern border of the SE quadrant. These data suggest that further exploration in subsequent evaluation years could be valuable to better understand the underlying reasons behind these results.

60

conducted GIS mapping of the survey question: “When you were looking for a preschool for your child, did you find any preschools that you liked but they were full and not accepting more children?” Higher concentrations of “yes” answers were reported in certain city quadrants.

Figure 4.5. Responses to parent survey: When looking for a preschool did you find any that you liked but they were full and not accepting more children?

All these data suggest that families with children who want them to attend preschool face a number of accessibility challenges, including general availability of preschool centers close to where they live, available spots in the centers they prefer, and availability of spots with part-, full-, or extended-day spots that meet their needs.

Quality As shown in Section 1 of this report, most families (57%) who completed the online survey were not aware of the quality rating of their child’s preschool (Figure 1.3). Survey data were used to conduct GIS mapping to analyze the geographic distribution of the families’ awareness of their preschool’s quality rating, presented in Figure 4.6. As shown in the figure, the red “no” responses appear to be spread rather evenly across neighborhoods and quadrants, indicating a somewhat consistent lack of knowledge regarding childcare facility ratings across Denver communities.

61

Figure 4.6. Responses to Parent Survey Question: Do you know what the quality rating of your child’s preschool is?

In the evaluation team’s focus groups, a few families shared that they did use the quality ratings to identify potential childcare centers, using four stars as the cutoff point, though this was not most families' top priority.

• “So really narrowing down what kind [of preschool] I wanted and then I didn’t know where to go, I didn’t have any other friends [who could recommend a preschool] … so I went strictly out of the ratings.” (T3, non-DPS, SE) • “We ended up using the rating system that all the preschools get rated. I don't remember what it's called. We used the quality ratings and then we were specifically looking at preschools inside of DPS public schools, so I also looked at the school itself and kind of some of their test scores, demographics and things like that.” (T5, DPS, NE) • “We used the QR, the quality rating QRAS, I think it's called, and made a short list of schools that met the 4 stars or higher, I think was kind of my cut off.” (T5, non-DPS, NE) When asked which quality aspects of a preschool were most important, 81% of families that completed the survey indicated that the preschool director and the teachers having the appropriate training, certificate, or license was the most important aspect of a quality preschool.

This was true across all income levels. Eighty-two percent of high-income families also indicated that the type of program used for teaching was also a very important aspect of a quality

62

preschool. How long the preschool has been in operation ranked the lowest importance for families, across all income levels.

Culturally Responsive Services The family survey asked respondents to share their experience around culturally responsive services provided by their DPP preschool providers. Families were asked about the respect they felt they received from staff as well as share specific characteristics of preschools they valued most. When families were asked if they agreed that the directors, teachers, and other employees at their child’s preschool valued and respected their culture, 80% of families indicated that they agreed with the statement, 10% of families opted to skip the question and the remaining 10% disagreed. The responses did not show variation across income levels or quadrants of Denver.

Mapping of survey data, as shown in Figure 4.7, reveals the degree to which most respondents from across quadrants and neighborhoods felt that providers were respectful of their family’s culture. These data show a preponderance of respondents either “agreed” (shown in light green) or “strongly agreed” (shown in dark green) with the question: How much do you agree that the director, teachers, and other employees in your child’s preschool value and are respectful of your culture? As the map shows, there is not a clear concentration of “disagree” (pink) or “strongly disagree” (red) responses by neighborhood. Certain neighborhoods, such as Central Park, Park Hill, East Colfax, and others in the central part of the northeast quadrant are almost completely green, indicating a high consistency of parent survey responses and satisfaction in these areas.

Figure 4.7. Responses to Parent Survey Question: How much do you agree that the director, teachers, and other employees in your child’s preschool value and are respectful of your culture?

63

Additionally, when asked which characteristics of a preschool were most important, 82% of families indicated that having the preschool listen to their concerns and being nice/good/respectful to their child and family were the most important characteristics. This was true across all income levels. Being close to public transportation and sharing with a family's religious beliefs were indicated as the least important for families, across all income levels.

In focus groups, several families shared that they conduct ample research into a preschool’s culture before enrolling their child to ensure a healthy and safe fit for their child.

“I always go to the state website to search and see if there is any type of violations, child abuse allegations, I read a lot of the reviews to make sure that the environment is going to be safe. And I also visit the schools. I go and visit and spend time so I can get to know that culture and community.” (T1, non-DPS, NE)

When asked which preschool characteristics were most important, 82% of families indicated that having the preschool listen to their concerns and being nice//good/respectful to their child and family were the most important characteristics.

“[A]s a former educator, I'm one of those people who will go into the school and observe and then ask a ton of questions. And so, I basically would tour every school and then ask the providers my long list of questions, see if their responses were comprehensive and then make a decision from there.” (T5, non-DPS, NE)

SECTION HIGHLIGHTS – Section 4 of this report included the following highlights: • 47% of parent survey respondents found preschools they liked but were too expensive, underlining a continuing affordability challenge which Denver faces.

• The survey found that only 36% of families believed there were enough preschools to choose from close to where they live (see Figure 4.2). This finding is critical since parent focus group participants indicated that proximity to home was a highly influential factor when choosing a preschool.

• Nearly half (48%) of families responding to the survey found preschools they liked but did not have available spots for their child. About 60% of high-income families had this experience while 35% of low-income families shared this experience.

• Some parents reported feeling the process of finding a preschool overwhelming and could not find availability for the services they needed such as half-, full-, or extendedday care, or a provider that was a good match for their family and child.

• Data analyses found some disparity between the proportion of available preschool centers and the number of families receiving DPP tuition credits, with the Northeast quadrant showing a higher proportion of families than available preschools.

64

•

•

• •

•

In the focus groups, families shared how expensive not only preschool itself is once a child is enrolled, but also that the process of finding a preschool can be expensive itself due to the deposits required to place their child on a waitlist.

The evaluation team conducted GIS mapping of the survey question: “When you were looking for a preschool, did you find any that you liked but they were full and not accepting more children?” The data indicate more exploration is needed in subsequent evaluation years to explore underlying reasons behind differences in results for neighborhoods such as Montbello, Central Park, Sloans Lake, and others.

A majority of families (57%) across all income levels stated they were unaware of their preschool’s quality rating.

About 80% of families agreed that the directors, teachers, and other employees at their child’s preschool valued and respected their culture. Survey responses did not show variation across income levels or the quadrants where families lived.

82% of families indicated that having a preschool that listens to their concerns and that is nice/good/respectful to their child and family were the most important characteristics of a preschool. This was true across all income levels.

Data gathering in this section highlights a key overall finding that affordability and availability of childcare slots at preschools that meet parent needs is a major challenge for the city. Roughly half of parent survey respondents indicated challenges finding open slots in preschools that were affordable or desirable to them and only about a third of respondents said there were enough preschools available to choose from where they lived. Current and past evaluations have consistently found location close to home to be a crucially important factor for families.

A lack of affordable and available slots appears to be contributing to what parents describe as an “overwhelming” process at times in finding appropriate and available preschools. Additional research in subsequent years is needed to further explore the underlying reasons behind some difference in parent responses by neighborhood (for instance, differences in parent perceptions of preschool availability in far northeast neighborhoods like Montbello and Green Valley Ranch versus neighborhoods in the northwest quadrant of the city).

The next section of this report addresses evaluation questions pertaining to DPP impacts on families and family access to childcare.

65

5. Maximizing DPP Impacts and Access to Childcare The evaluation team gathered data through a parent survey and a series of focus groups to address several key topics, including:  What childcare options there were for families if preschool were not accessible;  Perceptions of the benefits that children experienced by having access to childcare; and  Perceptions of benefits to families and caregivers by having access to child care.

With regard to childcare options for families if preschool were not accessible, the parent survey asked respondents to answer the question: Who would care for your child if preschool was unavailable? As shown in Table 5.1 when asked about what alternatives families would utilize if preschool was not available to them, 42% of survey responses, evenly distributed across income tiers, indicated that they would have to stay home to care for their child, 27% said someone in their family would care for their child, and 14% indicated they did not know who would care for their child if preschool was not available to them.

Table 5.1. Parent survey responses on childcare options Who would care for your child if preschool was unavailable?

I would have to stay home to care for my child Someone in my family I don’t know Nanny A friend or neighbor I would need to bring my child with me to work

Responses 42% 27% 14% 9% 4% 4%

Having close to half of respondents indicate they would need to stay home to care for their child has several implications regarding potential DPP impacts on families and the affordability of childcare. First, as indicated by some parents in the evaluation team’s focus groups, children remaining at home would miss an extremely valuable opportunity for socialization with other children and adults if preschool were not available or affordable to them. Parents believed that the socialization which takes place in preschool, along with access to outside expertise to support development of child social and behavior skills, yields profound impacts on child development and social emotional health:

66

“Having my child in preschool gave me that village, just that backup of his teachers and the paraprofessionals and all of the resources that were there to help navigate raising your child, you know, it Parents believed that the has made a huge difference. And [my son] knowing that socialization that takes there are other adults that care about him and his place in preschool, along growth that, we can kind of pool our resources and talk with access to outside expertise to support child about behaviors, and kind of have that resource to figure social and behavior skills, out, you know, is this normal behavior? Is this yields profound impacts on abnormal? …And being able to have that team and that child development. village to develop strategies that's been so huge, it's been just huge, it's been huge for him.” (T5, Non-DPS, NW) Numerous parents also pointed specifically to the increased need for the types of socialization and emotional development opportunities afforded by preschool, due to the COVID-19 pandemic. Parents described such socialization as critical to their child’s development and for their successful kindergarten readiness in the current, post-pandemic period: “I think a lot of us we have pandemic kiddos that were home for several years and seeing my daughter foster relationships and friendships with new kids, because they're different than her, was really, really powerful and…she had this new-found confidence which was really cool to see.” (T6, DPS, NW) “For me I saw a huge growth in her social skills because she was home for so much of the pandemic and wasn't able to socialize when she was so young that it was really nice for her to be able to build those social skills in a play-based program. I saw a huge amount [of growth], like way less tantrums at home, just her ability to do things on her own was so big.” (T3, Non-DPS, SE) “For our son, there were huge benefits of socialization. We kind of holed up during the pandemic and so this was his first experience in the real world, and I think it was really great for him to start to make friends, figure out his boundaries, and then, of course, early literacy and math skills in the program were great.” (T5, Non-DPS, NE) Second, in addition to the enormous benefits to children associated with increased socialization from participation in preschool, parents also indicated their belief that participation in preschool enhanced the development of their child’s core academic skills that would lead to improved school readiness and academic success in kindergarten: “I think being in the preschool program they learn how to write their name faster, they can read, they know their colors, their shapes, their numbers. Because I compare them with other kids that didn't go to that program, that started at 5 years old and they didn't know their name, they didn't know how to write their name. And my 5-year-old knows her 67

first and last name, she knows how to write her numbers, she knows how to write her name, and she knows how to read like 2 to 3-word books.” (T2, DPS, NE) “All the benefits that come along with being in preschool, Parents also indicated such as socialization, the structure and organization for the their belief that participation in preschool day, getting a head start on the letters, numbers and colors, enhanced the and all the things preparing for kindergarten which with development of their our first child my wife was doing at home just naturally, but child’s core academic it was because she only had one kid at home. With the skills that would lead to middle kids there was a baby and that's where my wife's improved school readiness and academic energy and attention were going and so the natural success. preschool education that was happening with the first kid wasn't able to happen with the later kids. So that was a massive benefit of getting the kids ready to go into kindergarten.” (T3, DPS, NE) As described above, the input received from parents and caregivers who participated in the evaluation team’s focus groups and survey clearly point to the perception of significant, positive impacts from participating in preschool on child social emotional well-being, academic development, and kindergarten readiness. DPP contributes to these successes in multiple ways, including the impact its tuition credits have on making preschool more affordable to families (see the section above in this report discussing the Tuition Credit System) and its impact on making more families aware of the potential benefits of having their child participate in preschool. As one parent in the evaluation team’s focus groups stated: “DPP has given me the opportunity to register my child to go to school, because without the financial support I couldn’t have taken him simply because I don’t have the money for tuition.” (T2, Non-DPS, NE) In addition to the benefits for children associated with DPP, data collected by the evaluation team also point to clear benefits to parents and caregivers. For instance, as Figure 5.1 below shows, when asked about the benefits provided by having access to preschool for their children – which can be made more affordable and feasible to families as a result of DPP’s tuition credits – parent survey responses varied but fell into three distinct categories: Self care, Economic improvements, and Care for others.

68

Figure 5.1. Parent survey responses on how access to preschool helps families.

The hours that your child is in preschool helped you to…

Reduce your stress or improve your mental health Attend other personal responsibilities

Self Care

Take better care of yourself Improve your personal and family relationships Stay employed or increase your work hours and income Economic Improvement

Have the time to get a job Have the time to get more education Care for other children you have at home

Care for Others

Care for other adults 0%

2%

4%

6%

8%

10% 12% 14% 16% 18% 20%

Percentage of parents choosing each option

Comments offered by parents in the evaluation survey illustrate these benefits, • “They have allowed for me, the primary caregiver, to have a small break everyday to take care of my mental health.” (T2, DPS, SW) • “Reduce my stress.” (T3, DPS, NE) • “It helped our family not feel stressed getting the money each month.” (T3, DPS, NE) • “Have enough time to catch up with stuff around the house” (T1, DPS, SE) • “I look forward to being able to work more hours to maintain bills” (T3, DPS, NE) • “Work and actually see my paycheck - rather than applying that all to my children (3) and their overall care.” (T5, non-DPS, SE) • “The cost savings has helped with our general budgeting, especially when having our second child and incurring additional childcare costs for child #2.” (T4, nonDPS, SE) • “Tener tiempo suficiente para atender a mi bebe que Tengo aun en casa ya que es mucho trabajo.” [To have enough time to take care of my baby that I still have at home because it is a lot of work] (T2, non-DPS, NE) • “Afford childcare for my other children” (T5, DPS, NE) Survey respondents were asked about the benefits of having access to childcare. They were able to select as many response options that were applicable to them from a set of nine options. The four options of the Self Care category were chosen approximately 65% of the time (e.g., reduce stress or improve mental health was selected by 17% of parents, take better care of yourself was selected by just over 16% of parents, and so on) indicating that access to childcare allowed 69

parents and caregivers additional time to spend on stress reduction, mental health, and self-care.

This applied to both high and low-income families, with more than 90% of high-income families selecting a response in the category of Self Care.

It also applied rather consistently across Denver’s four quadrants and neighborhoods, as shown in the GIS map in Figure 5.2 below, which shows a relatively even distribution of “yes, this is my case” answers across the city to the survey question: Have the hours that your child is in preschool reduced your stress or improved your mental health?

Approximately two-thirds of low-income and highincome respondents selected at least one survey response that indicated access to childcare allowed for improvement of their economic conditions.

In the post-pandemic world – with skyrocketing mental health challenges for adults and primary caregivers as well as children – this finding suggests an additional enormous value brought about by DPP which serve to boost both awareness of, and access to, quality childcare for families of all incomes.

Figure 5.2. Responses to parent survey question: Have the hours that your child is in preschool reduced your stress or improved your mental health?

70

Approximately two-thirds of low-income and high-income survey respondents (63% and 65%, respectively) selected at least one survey option that indicated access to childcare allowed for some improvement of their economic conditions. This included selecting options such as allowing more time to get a job, stay employed, increase work hours and income, or more time to pursue education to improve their job prospects. Approximately 37% of low-income family respondents indicated that the hours of childcare afforded them the time to get a job, representing 53% of families selecting this response. About 62% of high-income families indicated that the hours of childcare afforded them the opportunity to stay employed or increase their hours and income, representing 45% of families selecting this response. As shown in Figure 5.3, survey responses appear relatively evenly distributed across quadrants and neighborhoods of the city for parents that answered “yes, this is my case” to the question: Have the hours that your child is in preschool helped you to stay employed or increase your work hours and income?

Figure 5.3. Responses to parent survey question: Have the hours that your child is in preschool helped you to stay employed or increase your work hours and income?

Lastly, 10% of survey responses indicated that accessibility to preschool allowed them to invest additional time and effort on caring and supporting others. This included caring for other children at home or caring for other adults. Responses in this category were proportionate across income tiers and reflect again an important ancillary value to Denver families as a result of their participation in DPP. 71

Perhaps indicative of the survey findings discussed above Nearly 40% of survey regarding the impact of increased access to preschool on reducing respondents indicated that parent stress and improving economic conditions, survey DPP tuition credits enabled respondents indicated that DPP tuition credits helped them meet families to afford and meet a basic economic needs. For instance, as shown in Figure 5.4 below, variety of basic needs – shown in the figure as food, more than a quarter of respondents indicated DPP tuition credits housing, and health care. helped reduce the cost of preschool to the extent that it supported their ability to pay other bills. In addition, nearly 40% of survey respondents indicated that DPP tuition credits enabled their families to afford and meet a variety of basic needs – such as food, housing, and health care. For low-income families, the most popular basic need they were able to consume more of was food for their family, while the most popular basic need high-income families were able to consume more of was housing costs. The following comments, written by parents in the evaluation survey, exemplify some of the basic needs families were able to meet thanks to DPP’s financial support: • “As a single mom, every dollar counts. Saving on preschool helps me pay for many of the other necessities without such a financial burden.” (T4, non-DPS, NE) • “Save for college” (T5, non-DPS, NE) • “Saving money for not paying baby sitter” (T2, DPS, NE) • “Tener un poco más de dinero para comida” [To have a bit more money for food] (T2, DPS, NE) • “Being able to save what money would have gone towards ECE and put it towards healthier (more expensive) grocery items is what makes this program rock” (T3, DPS, SE)

72

Figure 5.4. Parent survey responses on how DPP tuition credits helped families meet basic needs.

The DPP tuition credits helped you and your family...

Pay other bills Put some money into savings Buy food for your family Basic Needs

Pay for housing (rent, mortgage, etc.)

Reduce hours worked and spend more time with family

Afford health insurance Buy a car and decrease your commute time Reduce hours work and go back to school 0%

5%

10%

15%

20%

25%

30%

Percentage of parents choosing each option

Smaller numbers of respondents identified other benefits associated with receiving DPP tuition credits, including being able to: reduce workload to spend more time with family; purchase a vehicle to decrease commuting time; or reduce work hours in order to go back to school and pursue more education opportunities. Taken together, all of these benefits associated by families with receiving DPP tuition credits pointed to broad impacts across income levels related to the types of stress reduction, economic enhancements, and meeting basic human needs such as food and shelter that can only serve to contribute to more stability at home for parents, caregivers, children, and their extended families.

The next section of this report discusses evaluation team findings with regard to presenting findings about DPP to a variety of audiences.

SECTION HIGHLIGHTS – Section 5 of this report included the following highlights: • When asked about what alternatives families would utilize if preschool was not available, 42% of survey responses, evenly distributed across income tiers, indicated they would have to stay home to care for their child, 27% said someone in their family would care for their child, and 14% indicated they did not know who would care for their child.

• Parents believe preschool gives children important opportunities for socialization and emotional development, that they cannot get at home.

• Parents believe preschool helps their children to better develop early literacy and math skills that lead to improved school readiness and academic success in kindergarten.

73

•

•

• •

Parents benefited in different ways from the hours that their child was in preschool, including better self care (reduced stress, increased mental health, taking better care of themselves). At least one option in the self care category on the parent survey was chosen about 65% of the time. Benefits applied to low- and high-income families from all four quadrants of Denver and across neighborhoods as shown through GIS mapping.

Approximately two-thirds of low-income and high-income survey respondents (63% and 65%, respectively) selected at least one survey option that indicated access to childcare allowed for some improvement of their economic conditions (allowing time to get a job, stay employed, increase work hours and income, or time to pursue education). Again, these benefits appear across quadrants and neighborhoods in Denver.

DPP tuition credits helped families meet their basic needs such as paying other bills, saving some money, buying food for their families, paying for housing, etc.

Overall, the reported benefits of receiving DPP tuition credits pointed to broad impacts across income levels, city quadrants, and neighborhoods related to the types of stress reduction, economic enhancements, and meeting basic human needs that can provide more stability at home for parents, caregivers, children, and their extended families.

Having close to half of respondents indicate they would need to stay home to care for their child if preschool was not available or affordable has several implications when it comes to the reality of DPP’s impacts and the messaging of these impacts to the public. First, children remaining at home would miss out on an extremely valuable opportunity for socialization with other children and adults. Parent focus group participants believed that the socialization which takes place in preschool, along with access to outside expertise to support development of child social and behavior skills, yields profound impacts on child development and social emotional health.

Second, if preschool were not available or affordable, children would miss a critical opportunity to be better prepared for kindergarten and success in school. Parents in focus groups consistently indicated their belief that participation in preschool enhanced the development of their child’s core academic skills and led to improved school readiness and academic success in kindergarten, input that has considerable support in the existing literature. DPP should consider incorporating continued messaging to the public around these important benefits that accrue when preschool is both more available and affordable.

74

6. Communicating DPP evaluation findings to multiple audiences Communicating DPP’s impact on children and families is important to inform and engage other families who can benefit from DPP’s financial support. Similarly, it is essential to let the Denver community know of the difference that their tax dollars make on children’s education. To convey this information, families participating in the APA evaluation team’s focus groups were asked about their thoughts on the types of messages that could and should be shared with the Denver community. DPP childcare providers and parents also offered feedback about which communication channels are the most appropriate to use and who the most trusted sources are for information.

Parents who participated in the focus groups mentioned that DPP should focus on sharing with the public the following key messages and information, including that: 1. DPP helps families send their children to preschool.

2. Attending preschool has many benefits for children such as building strong academic and social emotional foundations that will get them ready for kindergarten and support success in their future education.

3. Messaging should include the number of children that DPP is helping, and how many of them would just stay home, go to daycare, or be cared for by friends or relatives if it were not for DPP. 4. Research correlates children attending quality preschool programs with increased high school graduation rates, and reduced delinquent behaviors later in life by helping them become functional, well-adjusted human beings, and that these benefits will lift up the city as a whole.

5. Enrolling children in preschool benefits families. For instance, DPP tuition credits partially or fully cover tuition costs, easing family financial burdens.

6. Expanding access to preschool supports parents’ ability to go to work without having to pay for expensive childcare costs, which supports economic stability, boosts household income, and increases the amount of taxes that are generated and contributed back to supporting DPP, creating a complete “win-win” situation for Denver.

7. DPP provides not only tuition support, but also provides resources to help parents find preschools and invests significant resources to increase the quality of Denver’s preschool programs.

8. DPP is not the same as UPK. In fact, DPP needs to do a better job of becoming more visible and distinguishing itself from UPK to reduce parent confusion between the two programs.

9. DPP is for all Denver families, not only for low-income families, and that all families get financial support to cover their children’s tuition.

75

Comments from parents participating in the evaluation team’s focus groups provide further color and depth to the types of messaging that DPP should focus on sharing with the public. One parent for instance commented about their own experience: “Any help is great and just being part of the [DPP] program felt like another part of a family, you know, like somebody's out there helping us and we're helping other people, and in a city where it can feel so busy, it was like a grounding center, like they're gonna help everybody. You guys are gonna make sure kids go to preschool, which is going to make our city better.” (T3, non-DPS, SE) DPP parents in focus group conversations suggested messages about DPP should include research findings that: 1) correlate early childhood education with eliminating the school-toprison pipeline, in such a way that investing in education solves the problem of overcrowded jails; and 2) show that by using tax money to support preschool, the city increases high school graduation rates, showing that Parents in focus groups suggested DPP messaging community investment into preschool lifts the education of the emphasize not only the entire city, and improves everyone’s quality of life, and that a financial supports offered but strong community leads to economic growth and benefit for all.

also resources to help parents find preschools and investments that increase the quality of preschools in Denver.

Finally, parents thought that DPP needs to do a better job of becoming more visible, distinguishing themselves from UPK, and communicating to the community what they offer, not only financially, but also underscoring the resources to finding preschools and increasing the quality of the preschool programs in Denver.

Parents who completed the evaluation team’s survey were given a list of seven different findings about DPP and asked to indicate which of those findings they were interested in learning about.

Over 60% of those who completed the survey indicated they were “a bit interested” or “very interested” in learning about each of those findings as follows: ▪ Whether children participating in DPP do better in school 67% ▪ Whether and how DPP helps improve preschool centers and their personnel 67% ▪ Whether children participating in DPP are more likely to graduate HS 65% ▪ Whether and how DPP helps families 64% ▪ Whether DPP saves taxpayers more money than it costs 63% ▪ How much money DPP provides to reduce the cost of preschool for families 63% ▪ How DPP provides tuition credits based on different levels of family income 61% These top topics that were of most interest to parents on the survey corroborated the top findings from the focus groups. This included a similar emphasis on communicating more information to the public about: 1) DPP impacts on later student performance in school and on high school graduation rates; 2) how DPP support is used to improve the quality of preschools in Denver; and 76

3) how DPP supports families – which could include increased messaging around how DPP eases family financial burdens and supports parents seeking expanded employment opportunities.

Making sure that a message reaches the community involves having a trusted person, group of people, or organizations deliver the message. Parents, as well as preschool providers who completed the surveys, were each given a list of groups and were asked to indicate how much they trust each group when it comes to hearing about research findings on the impact of DPP on children, families, and preschools. Parents were asked to indicate if they had: No trust, Low trust, Moderate trust, or High trust regarding 10 different groups. Six of these groups were rated as Moderate or High trust by more than 50% of responding parents, while the other four groups received that rating from less than 40% of respondents. Table 6.1 shows details about these data.

Table 6.1. Percentage of parents with moderate or high trust in specific groups of people to deliver information about the effects of DPP on children, families, preschools Group of people

Percentage of parents indicating trust 78 75 70 69 62 54 39 29 21 19

Childcare director or teacher Personnel from the Denver Preschool Program Denver’s Early Childhood Council Researchers Representatives from Denver Public Schools Community organizations Elected officials or civil leaders Religious leaders Radio personalities TV personalities

Preschool providers were given a slightly different list of people and were asked to rate their level of trust on a scale where 1 = Lowest trust and 5 = Highest trust. Five of these groups received an average rating of 3 or higher. Table 6.2 shows details about these data.

Table 6.2. Trust level average rating given by preschool providers to groups of people to present research findings about the impact of DPP on children, their parents, and providers (Scale: 1 = Lowest trust and 5 = Highest trust) Group of people DPP leaders Preschool providers ECE leaders Researchers Community led organizations / nonprofits Policy makers

Average trust level 4.6 4.5 4.3 4.0 3.7 2.9

77

Religious leaders TV/radio personalities Politicians Policy makers

2.7 2.5 2.3 2.9

Data from Tables 6.1 and 6.2 show that, with very slight differences, families and providers trust the same groups of people to share information about the impact of DPP. The top trusted sources of information across families and providers were consistently: 1. Preschool providers; 2. DPP leadership or staff; 3. Leaders from the ECE community (such as Denver’s Early Childhood Council); and 4. Researchers.

These findings are encouraging for several reasons. First, DPP is itself a trusted source of information for parents and providers. This places the program in a strong position to ramp up its communications to the public, parents, and providers with confidence that these groups will give credence to the information that DPP shares. Second, the high confidence expressed in researchers indicates that DPP is also in a strong position to leverage and disseminate findings from its current and past investments in program evaluation, which have yielded a wealth of information over time. The high confidence in researchers also suggests that DPP can with confidence: 1) share and disseminate findings from the local and national literature pertaining to impacts of ECE on children and communities in order to continue reinforcing the soundness of current public investments in DPP; and 2) continuing exploring avenues of research into DPP impacts, through evaluation or other research.

Findings have some strong, positive implications for DPP. They demonstrate a preference for hearing information directly from DPP or through providers, which indicates trust in these sources.

Finally, the way in which the message is delivered is also important to ensure it reaches its intended audience. Families completing the survey were given a list of 13 possible ways to receive information about the effects of DPP on children, families, and preschools, and were asked to select the three ways they most preferred. Emailed newsletters were ranked the highest across both low- and high-income families, followed by DPP’s website, and 1-2-page summaries with graphics and findings. These were the only three forms that were selected as preferred by more than 30% of parents. TV advertisements, 5-10-page detailed reports with findings, and Radio advertisements were ranked the lowest across both lowand high-income families.

These findings again have some strong, positive implications for DPP. First, they demonstrate the preference for hearing information directly from DPP or through providers – which again corroborates the trust that parents and providers have for these sources – rather than through secondary sources such as TV and radio. This suggests that DPP should view itself as a primary 78

purveyor of information about the program. A clear advantage to this finding is that it allows the program to maintain an enhanced control over the messaging associated with program activities and with findings from ongoing research and evaluation activities. In particular, DPP should target dissemination of information – through its website and email – of brief newsletters and /or 1–2-page summaries using a mix of text and graphics to convey information. Extensive investments in TV/radio advertising do not appear to be warranted according to survey findings.

Table 6.3 shows the ranking of the 13 ways to get information and the percentage of parents who selected each form in the evaluation team’s survey.

Table 6.3. Best ways for parents to get information about the effects of DPP on children, families, and preschools.

Ways to get information Emailed newsletters DPP’s website 1-2-page summary with graphics and findings Social media (Twitter, Facebook, Chats, YouTube, TikTok, etc.)

Text messages Printed newsletters Reported in the news on radio, local TV, or newspapers In-person presentations Virtual presentation/webinars 3-4-page report with findings TV ads 5-10-page detailed report with findings Radio ads

Percent selected 50 42 36 28 21 15 15 8 7 5 3 2 2

Similar to families, childcare providers were given a list of ten formats to be informed about research findings and were asked to rate their preference for each of the formats using a scale where 1 = Low preference and 5 = High preference. Table 6.4 shows providers’ average preference for each of these formats from highest to lowest preference, indicating four of those formats getting an average preference higher than 3, including: 1) Written handouts/newsletters; 2) In meetings organized by the Denver Early Childhood Council or another organization; 3) Infographics; and 4) Verbal presentations/webinars. Long/comprehensive reports, Radio/TV ads, and Tweets, received the lowest preference rates.

Table 6.4. Providers’ preference of format to get informed about research findings.

Formats to get informed Written handouts / newsletters In meetings organized by the Denver Early Childhood Council or another organization Infographics

79

Average preference 3.9 3.8 3.5

Verbal presentations/webinars

By DPS Through specific news sources such as radio and/or TV Text messages Long/comprehensive reports Radio/TV ads Tweets

3.4 2.9 2.5 2.3 2.2 2.2 1.9

SECTION HIGHLIGHTS – Section 6 of this report included the following highlights: • Parents who participated in evaluation indicated DPP should focus on sharing with the public the following key messages and information, including: o DPP helps families to send their children to preschool. o Attending preschool has many academic and social emotional benefits that help children get ready for kindergarten. o Attending preschool supports success in future education and reduces negative behaviors later in life. o Families benefit from DPP by reducing their financial burden.

• Parent and provider input suggested that DPP needs to more clearly distinguish itself from UPK. • Parents indicated top trusted sources to deliver information about the impacts and effects of DPP are: 1) childcare providers; 2) DPP personnel; 3) the Denver’s Early Childhood Council; 4) researchers; and 5) DPS representatives.

• Providers indicated top trusted sources to deliver information about the impacts and effects of DPP are: 1) DPP leaders; 2) other preschool providers; 3) ECE leaders; and 4) researchers.

• Families prefer to receive information about DPP via email newsletters, the DPP website, and 1–2-page summaries with graphics and findings.

• Providers prefer to receive information about research findings in written handouts/newsletters, or in meetings organized by the DECC or another organization, through infographics, and verbal presentations/webinars.

Parents and ECE providers participating in DPP made several suggestions about the messages that should be shared with the community, what channels of communication would be preferred to share the messages, and who are the most trusted people or organizations to share the messages. Overall, the messages should focus on the benefits to children and families of participating in DPP; childcare providers and DPP leaders should communicate those benefits; and the dissemination of information should be made via its website, email, or brief newsletters or short summary reports.

80

IV.

Summary of Findings

Following are listed the section-by-section highlights which are also included at the end of each of the six main sections of this report.

Highlights of Section 1: Perceptions of DPP Finding out about DPP • Families reported finding out about DPP primarily from other parents, friends, and neighbors, by asking teachers or administrators at their preschool center, or through the DPP website.

• Many DPP families had an incomplete and inconsistent understanding of DPP and preschools in general, including not knowing that preschools charge tuition and that DPP pays tuition credits to the preschool provider on behalf of families, and believing that DPP’s financial support is for low-income families only and that high-income families do not qualify for the financial assistance.

The application process & awareness of tuition credits • The majority of parents indicated that completing the DPP application was a very easy and straightforward process. Some parents reported not knowing they had applied to DPP at the same time they were completing their preschool enrollment application. Other parents indicated not knowing what they should expect after submitting their application because there was not enough clarity on the “next steps” in this process.

• Providers played a very important role in encouraging parents to apply to DPP, with some of them making the DPP application a requirement for parents.

• Most families found out that DPP offered financial support at the same time they learned about the program in general. However, most families reported not knowing specifically how much DPP financial support they would qualify for or how the tuition credit was calculated. Some families found out the exact amount of tuition credits when they received a bill from the preschool but did not recall receiving communication from DPP with that information.

• Parents would like to have had access to updated information about DPP preschools, such as a comprehensive list of providers, their location, and the number of spots available to enroll children.

Awareness of quality ratings • 41% of families who completed the evaluation survey did not know that DPP preschools receive a quality rating, and only 28% of families know the quality rating of the preschool their child attends.

81

Outreach to families • Parents believe that the primary reasons why families do not enroll their children in DPP are: 1) they did not know that DPP exists; 2) they believed their income was too high to qualify; 3) they did not know that DPP reduces the cost of preschool by providing tuition credits; 4) they believed that only children enrolled in Denver Public Schools (DPS) qualify to receive tuition credits; and 5) families do not know where to start the DPP enrollment process.

• Providers believe that the main reasons why families do not enroll their children in DPP are: 1) they do not know about DPP; 2) they confuse DPP with other financial support programs and believe they are in DPP when they are not; 3) they have misconceptions/misunderstandings about the benefits they get from enrolling in DPP; 4) they are unable to cover the gap between the tuition credits received and the total cost of tuition; and 5) they believe they will not qualify for tuition credits because their household income is too high or because of their immigration status.

• Families suggested that the most effective methods for reaching out to families to inform them about DPP was through verbal or written communications, preferably from their childcare centers, but also from family or friends already in DPP, or DPP personnel.

Families indicated that social media, local radio, and TV news are not preferred methods of communication.

• Providers selected the following outreach strategies as the most effective ways to attract families to DPP: 1) interactions with families and friends already in the program (e.g., word of mouth); 2) face-to-face individual or group interactions with providers; 3) social media (FB, Twitter, TikTok, Instagram); 4) face-to-face interactions with DPP staff/representatives; and 5) printed or electronic communications from DPP (e.g., fliers, emails, texts).

DPP and UPK With regard to implementation of the UPK program in Colorado, the evaluation in Year 1 found: • Parents had numerous unanswered questions about what UPK was, how it would interact with DPP, what level of financial assistance would be provided, and to whom it would be provided, whether two separate applications would be required for DPP and UPK, and whether families could qualify for both programs at the same time.

• Providers were pleased that UPK will provide financial support to all Colorado families, however, the UPK matching of families/children to preschools was a major area of provider concern.

• Providers who completed the evaluation survey, as well as those who participated in the focus groups reported that UPK’s reimbursement rates would not be enough to cover their costs for providing care.

• Despite the challenges with UPK rollout, 69% of providers who completed the evaluation team’s survey reported having signed the UPK agreement or were in the 82

process of completing the enrollment. Another 4% indicated being interested in becoming UPK providers but had not started the enrollment paperwork, while 27% said they were not planning to participate in 2023-24.

In summary, families who participate in DPP usually find about it through interactions with preschool providers, friends and neighbors, and by accessing the DPP website. Although most families find the enrollment process easy to navigate, they also have a number of significant misunderstandings about the program. Evaluation data found that a number of families enrolled in DPP do not know they are receiving tuition credits or the amount of those credits, they also in some cases do not know they actually applied to the program. Families and ECE providers believe the main reasons why families do not enroll in DPP are because: they do not know of the program, they believe they are already enrolled, they believe they do not qualify, or do not know of the benefits of DPP for their children and families.

Families and providers have many unanswered questions about UPK and its relationship with DPP. These findings warrant a targeted communications campaign to further elevate awareness of DPP across the Denver community as well as its role in UPK. Conducting a series of webinars/information sessions with ECE providers could also help clarify misconceptions and increase overall awareness of DPP.

Highlights of Section 2: Customer Service / Interaction with DPP •

• •

•

•

Parents suggested that more expedited tuition credit notifications as well as more comprehensive information on the DPP tuition credits they receive throughout the year, would improve their DPP experience.

Parents had very little direct interaction with DPP, but those who did rated their interactions as positive.

Providers gave very high ratings to different aspects of their relationship with DPP, including: 1) professionalism and friendliness in their interactions; 2) satisfaction with the channels, quality, frequency, and effectiveness of communications; and 3) the support they get from DPP to address any issues or concerns they may have.

Providers mentioned the importance and value of improving teachers’ education through DPP-funded professional development as well as receiving classroom materials funded with DPP support, which helped providers improve the quality of care.

Providers must meet several requirements to participate in DPP, including: 1) licensing; 2) insurance; 3) background checks for all people working in or living in the center’s premises; and 4) obtaining a Colorado Shines’ quality rating. The evaluation data showed that out of these four requirements, only the quality rating slightly discouraged providers from participating in DPP. 83

Overall, families reported minimal, although positive interactions with DPP personnel, and suggested a few changes that would improve their experience. Providers, on the other hand, indicated extensive and very positive interactions with DPP, from receiving funding and curriculum materials, to their professionalism, responsiveness, and effective communications.

Providers indicated that the quality rating requirement is the only requirement that slightly discouraged them from participating in DPP.

Highlights of Section 3: The Tuition Credit System •

•

• •

•

•

•

DPP tuition credits have an important economic impact on parents and families. Parents in focus groups reported the main benefit of DPP was receiving tuition credits that made preschool more affordable. This was especially true for low-income families.

The economic impacts of DPP tuition credits were also important for middle- and high-income families, allowing them to add additional hours of preschool or after school care, or to choose schools that better matched better their child’s specific needs, or to pay for other needed services with greater peace of mind.

58% percent of families indicated they would either not be able to afford preschool or would struggle to afford preschool without DPP tuition credits.

GIS mapping of survey responses to the question: “Did DPP tuition credits encourage you in any way to send your child to preschool?” showed a dispersion of responses across neighborhoods. Higher concentrations of “yes” and “somewhat” responses could be seen in the Southwest and Northeast quadrants (in particular Montbello). These are predominantly lower income areas of the city. Conversely, the mapping showed concentrations of “not at all” responses to the question in neighborhoods such as Washington Park, Washington Park West, West Highland, and Sloans Lake.

Having children attend preschool gave parents more time to obtain employment, increase their working hours and family income, or maintain their full-time jobs. These benefits were strong incentives that encouraged families to enroll their children.

Nearly 50% of all families indicated they were able to afford more hours of childcare because of DPP’s tuition credit support. This was true for both low- and high-income families.

About 86% of parent survey respondents knew that DPP provided money to their child’s preschool to help reduce tuition costs, while 11% were unaware of this. Almost three quarters of those who were not aware of this were from low-income households. One factor contributing to this lack of knowledge might be that childcare providers who serve low-income families often require them to complete the DPP application alongside applications to other sources of financial aid, and these parents are not specifically informed that DPP is one of the sources of support.

84

•

•

•

•

More than 50% of parent survey respondents did not know how much money DPP paid to cover their preschool tuition costs. Sixty-eight percent of low-income families did not know the amount paid by DPP, compared with 36% of high-income families.

Data gathered by the evaluation team indicates there is still a need for more communication to parents and families that every family who enrolls in DPP receives a tuition credit of some kind, regardless of income.

There was confusion among some providers regarding whether DPP tuition credits are available to all families regardless of income. This misunderstanding warrants a thorough review of the different scholarships offered by DPP and the information that is shared about these programs with providers.

A survey asked providers to rate the importance of additional factors to determine the amount of DPP tuition credit per child. Data showed that family composition was rated the highest factor to consider, followed by supply of childcare providers, and revising the current provider quality rating model.

It is clear that DPP tuition credits have important and positive effects on families. The degree of effect can vary by income level, with lower income families more likely to respond that they would not be able to send their child to preschool at all without the credits. However, a mix of both higher and lower income families indicated that DPP tuition credits enabled them to afford more hours of care for their child. More efforts are needed to ensure that more parents are aware of the actual amount of tuition credits they receive, and that there is higher awareness that all Denver families, regardless of income, are eligible to receive DPP tuition credits.

Highlights of Section 4: Choosing Preschool • •

•

•

47% of parent survey respondents found preschools they liked but were too expensive, underlining a continuing affordability challenge which Denver faces.

The survey found that only 36% of families believed there were enough preschools to choose from close to where they live (see Figure 4.2). This finding is critical since parent focus group participants indicated that proximity to home was a highly influential factor when choosing a preschool.

Nearly half (48%) of families responding to the survey found preschools they liked but did not have available spots for their child. About 60% of high-income families had this experience while 35% of low-income families shared this experience.

Some parents reported feeling the process of finding a preschool overwhelming and could not find availability for the services they needed such as half-, full-, or extendedday care, or a provider that was a good match for their family and child.

85

•

•

•

• •

•

Data analyses found some disparity between the proportion of available preschool centers and the number of families receiving DPP tuition credits, with the Northeast quadrant showing a higher proportion of families than available preschools.

In the focus groups, families shared how expensive not only preschool itself is once a child is enrolled, but also that the process of finding a preschool can be expensive itself due to the deposits required to place their child on a waitlist.

The evaluation team conducted GIS mapping of the survey question: “When you were looking for a preschool, did you find any that you liked but they were full and not accepting more children?” The data indicate more exploration is needed in subsequent evaluation years to explore underlying reasons behind differences in results for neighborhoods such as Montbello, Central Park, Sloans Lake, and others.

A majority of families (57%) across all income levels stated they were unaware of their preschool’s quality rating.

About 80% of families agreed that the directors, teachers, and other employees at their child’s preschool valued and respected their culture. Survey responses did not show variation across income levels or the quadrants where families lived.

82% of families indicated that having a preschool that listens to their concerns and that is nice/good/respectful to their child and family were the most important characteristics of a preschool. This was true across all income levels.

Data gathering in this section highlights a key overall finding that affordability and availability of childcare slots at preschools that meet parent needs is a major challenge for the city. Roughly half of parent survey respondents indicated challenges finding open slots in preschools that were affordable or desirable to them and only about a third of respondents said there were enough preschools available to choose from where they lived. Current and past evaluations have consistently found location close to home to be a crucially important factor for families.

A lack of affordable and available slots appears to be contributing to what parents describe as an “overwhelming” process at times in finding appropriate and available preschools. Additional research in subsequent years is needed to further explore the underlying reasons behind some difference in parent responses by neighborhood (for instance, differences in parent perceptions of preschool availability in far northeast neighborhoods like Montbello and Green Valley Ranch versus neighborhoods in the northwest quadrant of the city).

86

Highlights of Section 5: Maximizing DPP Impacts and Access to Childcare •

• • •

•

• •

When asked about what alternatives families would utilize if preschool was not available, 42% of survey responses, evenly distributed across income tiers, indicated they would have to stay home to care for their child, 27% said someone in their family would care for their child, and 14% indicated they did not know who would care for their child.

Parents believe preschool gives children important opportunities for socialization and emotional development, that they cannot get at home.

Parents believe preschool helps their children to better develop early literacy and math skills that lead to improved school readiness and academic success in kindergarten.

Parents benefited in different ways from the hours that their child was in preschool, including better self care (reduced stress, increased mental health, taking better care of themselves). At least one option in the self care category on the parent survey was chosen about 65% of the time. Benefits applied to low- and high-income families from all four quadrants of Denver and across neighborhoods as shown through GIS mapping.

Approximately two-thirds of low-income and high-income survey respondents (63% and 65%, respectively) selected at least one survey option that indicated access to childcare allowed for some improvement of their economic conditions (allowing time to get a job, stay employed, increase work hours and income, or time to pursue education). Again, these benefits appear across quadrants and neighborhoods in Denver.

DPP tuition credits helped families meet their basic needs such as paying other bills, saving some money, buying food for their families, paying for housing, etc.

Overall, the reported benefits of receiving DPP tuition credits pointed to broad impacts across income levels, city quadrants, and neighborhoods related to the types of stress reduction, economic enhancements, and meeting basic human needs that can provide more stability at home for parents, caregivers, children, and their extended families.

Having close to half of respondents indicate they would need to stay home to care for their child if preschool was not available or affordable has several implications when it comes to the reality of DPP’s impacts and the messaging of these impacts to the public. First, children remaining at home would miss out on an opportunity for socialization with other children and adults. Parent focus group participants believed that this socialization, along with access to outside expertise to support development of child social and behavior skills, yielded profound impacts on child development and social emotional health. Second, if preschool were not available or affordable, children would miss a critical opportunity to be better prepared for kindergarten and success in school. Parents in focus groups consistently indicated their belief that participation in preschool enhanced the development of their child’s core academic skills and led to improved school readiness and academic success in kindergarten, input that has considerable support in the existing literature. DPP should consider incorporating continued messaging to the public around these important benefits that accrue when preschool is both more available and affordable.

87

Highlights of Section 6: Communicating DPP evaluation findings to multiple audiences •

• •

•

• •

Parents who participated in evaluation indicated DPP should focus on sharing with the public the following key messages and information, including: o DPP helps families to send their children to preschool. o Attending preschool has many academic and social emotional benefits that help children get ready for kindergarten. o Attending preschool supports success in future education and reduces negative behaviors later in life. o Families benefit from DPP by reducing their financial burden.

Parent and provider input suggested that DPP needs to more clearly distinguish itself from UPK. Parents indicated top trusted sources to deliver information about the impacts and effects of DPP are: 1) childcare providers; 2) DPP personnel; 3) the Denver’s Early Childhood Council; 4) researchers; and 5) DPS representatives.

Providers indicated top trusted sources to deliver information about the impacts and effects of DPP are: 1) DPP leaders; 2) other preschool providers; 3) ECE leaders; and 4) researchers.

Families prefer to receive information about DPP via email newsletters, the DPP website, and 1–2-page summaries with graphics and findings.

Providers prefer to receive information about research findings in written handouts/newsletters, or in meetings organized by the DECC or another organization, through infographics, and verbal presentations/webinars.

Parents and ECE providers participating in DPP made several suggestions about the messages that should be shared with the community, what channels of communication would be preferred to share the messages, and who are the most trusted people or organizations to share the messages. Overall, the messages should focus on the benefits to children and families of participating in DPP; childcare providers and DPP leaders should communicate those benefits; and the dissemination of information should be made via its website, email, or brief newsletters or short summary reports.

88

V.

Recommendations

The following recommendations are proposed by the evaluation team based on a review of all the findings contained in this 2022-23 evaluation report. Recommendations are not listed in order of priority and are reflective of all parent and provider focus group and survey data, as well as GIS mapping analyses and other data analyses conducted by the APA evaluation team.

There remains a broad range of what families know about DPP before applying and even after receiving tuition credits. This includes significant percentages of parents who do not understand if they are eligible to receive tuition credits and/or the amount of credits they are eligible to receive. More efforts may therefore be needed to send consistent messages out to the Denver community about how DPP works and the benefits it provides to families. In this regard, evaluation data suggest there are multiple key audiences to consider, including: a) Parents with children who are younger than preschool age; b) Parents with preschool age children who have not signed up for DPP; c) Parents with preschool age children who have signed up for DPP; d) Childcare providers. There are multiple potential avenues by which DPP can seek to communicate with each of these audiences, including: •

•

Survey and focus group data indicate parents have a high degree of trust for receiving verbal or written communications from their childcare provider, other parents, and from DPP’s staff and website. Verbal communications directly from their preschool provider carry a particularly high weight. DPP should seek to maximize investment in communication strategies that prioritize leverage of these more trusted outlets.

To encourage parents to enroll their child in DPP, the program could expand existing efforts to have a DPP representative or partner visit more preschools to explain and walk providers and parents through the entire DPP enrollment process and the benefits of participating. These presentations could take place in August, close to when school starts, either in the schools or another community location close to the school. Based on feedback from the evaluation team’s survey and focus groups, important messages to convey during these presentations include: o That DPP tuition credits are available to all Denver families, regardless of income. o The basics associated with calculating the amount of tuition support that DPP provides to a specific family, including how provider quality ratings work in the calculations. o How DPP interacts with and differs from other support programs such as Head Start, and Colorado’s UPK program. o The timing for when parents can expect to receive the credits and how they will be notified once credits begin to flow.

89

•

•

•

o The social and emotional development benefits that participation in preschool provides to children. o The benefits associated with academic achievement and kindergarten readiness associated with participation in DPP and preschool. o Findings from research indicating short-, medium-, and long-term benefits to students, including improved high school graduation rates. o Input from past parents and families that participated in DPP regarding the benefits they saw from preschool for their families and children and the ease of the DPP application process. o The difference between DPP and DPS. Clarifying the differences will help reduce existing confusion between these two organizations and to ensure that Denver families who are not planning to be served by a DPS preschool site still apply to participate in DPP. Since almost three quarters of those parents who were not aware that DPP provided tuition credits came from low-income households, DPP efforts to reach out to and train providers on this topic should focus on providers serving predominantly lower income areas of the city. One factor that such training could attempt to address is that childcare providers who serve low-income families often require parents to complete the DPP application alongside applications to other sources of financial aid, and that these parents are not specifically informed that DPP is one of the sources of support. Clarifying this process for parents could help reduce the chances that parents are not aware of the benefits they receive specifically through DPP. Parents indicate a need for access to updated information about DPP preschools, such as a comprehensive list of providers, their location, and the number of spots available to enroll children. DPP could work in collaboration with Denver’s Early Childhood Council and other early childhood education organizations to develop a process for such a list to be produced and promoted online each year. The list would likely need to be updated online at regular intervals. Such updating would represent a significant ongoing investment and commitment of staffing resources since the existing ECE provider landscape is in a constant state of flux. However, such an investment could pay dividends in helping parents locate an appropriate and available preschool provider for their children.

Since evaluation data indicates a high level of parent trust in childcare providers as sources of information, DPP could develop brief webinars/informational sessions and other materials to train more providers about DPP and ensure they are aware of: 1) The fact that, regardless of income, all Denver families are eligible to receive DPP tuition credits; 2) the different programs (e.g., scholarship programs for 4-year and 3-year-old children) offered by DPP and their requirements and eligibility rules; and 3) the basics associated with how DPP calculates the amount of tuition credits that families can expect to receive. Such trainings should seek to motivate and encourage providers to speak 90

•

•

•

•

directly with each of their families to encourage them to enroll in DPP. It is suggested that the webinars be conducted live with providers, but they could also be recorded and offered as a resource to those who are not able to participate in the live sessions, or those who may have questions throughout the year.

Similar proportions of parents who took the evaluation survey reported they found out about DPP by asking their preschool providers (24%) or visiting the DPP website (23%).

This finding indicates that the DPP website is an important tool that parents use to find information about the program and preschool in general. To leverage and further expand parent trust in its website, it is recommended that DPP continue to promote the DPP website as a premier source of information about the program and childcare in general.

To help ensure that the web site and its tools are as informative and user-friendly as possible for a variety of families – including those with different literacy levels – it is recommended that a pilot be conducted of the website and all its elements (pages and tabs) with a sample of families from the Denver community who can provide feedback about the web site’s clarity and usefulness. Special emphasis in such a pilot should be given to how parents and caregivers interpret and understand the information provided by the web site’s existing tuition credit calculator.

To leverage trust in other parents who have been through the process of choosing and enrolling their child in preschool and who have participated in DPP, the program could work through DPP providers to identify and reach out to a group of families/parents who have been in DPP to act as "DPP ambassadors" to engage parents new to the system to enroll their children in DPP. These ambassadors should be paid a stipend to recognize their time. Preferably these ambassadors would be organized by neighborhood or at a minimum by the quadrant of Denver in which they live or sent their child to preschool so that they have an intimate knowledge of the childcare landscape in a specific area of the city. Such ambassadors could be selected from a mix of parents whose children were close to completing or recently completed their year in DPP. Evaluation data collected from parents and providers indicate that such ambassadors would be ideally suited to serve as trusted sources of information to the next generation of prospective DPP parents.

Parents suggested that DPP join Bright by Text to send text reminders about enrolling their children in DPP and also inserting printed information in the packages that parents receive when they register in Bright by Text. Messages sent out through this resource could include information on the cognitive and socio-emotional developmental stages that 4-year-old children go through and the benefits of enrolling them in preschool.

DPP should review its current process for notifying parents about the tuition credits they have been awarded. Parent focus group input suggests that more expedited tuition notifications, more comprehensive information regarding the tuition credits, and added clarity of what parents should expect after submitting their application, could improve parent interactions with and appreciation of DPP. Sending tuition credit notification 91

•

•

•

approval letters electronically (e.g., email) to parents, followed by paper copies to parents and providers could help parents become more aware of the tuition credits they receive.

In terms of communications strategies DPP should keep in mind that only 22% of survey respondents identified any social media platform as a preferred communication method, and of those who identified this method the most popular platform was Facebook.

More than half of parents who completed the evaluation team's survey did not know how much money DPP paid to cover the child's preschool tuition costs. This group was dominated by low-income families (68% percent of low-income families did not know how much of their child's tuition cost was paid by DPP, while only 36% of highincome families did not know the amount). If DPP wishes to increase parent awareness of the benefits of participating in the program, it should focus on promoting knowledge of the amount of tuition credits parents receive, particularly working through providers and parents in lower income areas of the city. GIS mapping of parent survey data suggest a potential focus on the following Denver quadrants and neighborhoods: o Southwest Quadrant: ▪ Mar Lee ▪ Ruby Hill ▪ Harvey Park ▪ Harvey Park South ▪ Bear Valley o Southeast Quadrant: ▪ Kennedy o Northeast quadrant: ▪ Montbello ▪ Gateway-Green Valley Ranch ▪ Elyria Swansea ▪ Cole ▪ Clayton o Northwest Quadrant: ▪ Barnum West ▪ Barnum ▪ Chafee Park With regard to finding affordable and available preschools, the evaluation team in 202223 found roughly half of parent survey respondents indicated challenges finding open slots in preschools that were affordable or desirable to them and only about a third of respondents said there were enough preschools available to choose from where they lived. Current and past evaluations have consistently found location close to home to be a crucially important factor for families when choosing a preschool. A lack of affordable and available slots appears to be contributing to what parents describe as an “overwhelming” process at times in finding appropriate and available preschools. DPP 92

•

should consider using its ongoing investments in research and evaluation to further explore the underlying reasons behind differences in parent responses by neighborhood (for instance, differences in parent perceptions of preschool availability in far northeast neighborhoods such as Montbello and Green Valley Ranch versus neighborhoods in the perceptions in the northwest quadrant of the city such as Sunnyside and Sloan Lake).

Input from parents and providers regarding the impacts associated with DPP tuition credits ranged from reducing family financial stress, improving parent mental and emotional health, and enabling parents to seek expanded employment or educational opportunities for themselves and their families. On their own, these impacts represent potentially enormous value to the City of Denver and its citizens. These impacts could also be an extremely fertile ground for DPP to consider exploration of a return-oninvestment (ROI) analysis associated with the program’s investment of tuition credits. Such an ROI analysis would contribute not only to telling the full story of DPP impacts but would aldo add significantly to the national literature regarding the complete range of impacts of taxpayer funded preschool initiatives. DPP should explore coupling such an ROI study with ongoing local and national research on the short-, medium-, and long-term impacts of preschool on student academic achievement. ROI research on the economic, employment, mental health, and other impacts of DPP on parents and families could yield rich and valuable new data and insights for both Colorado and the nation.

93

Appendix A: Evaluation Advisory Group (EAG) The evaluation in 2022-23 incorporated input from an Expert Advisory Group (Advisory Group) who are Denver community and ECE policy experts. The Advisory Group participated as project advisors in year one over multiple online meetings with the evaluation team to provide input and guidance. The evaluation team sought the Advisory Group’s input in multiple areas, including: 1. Insights on how best to gain access to marginalized communities and to obtain input to the evaluation from leaders and representatives from these communities.

2. Review of all data collection instruments created for the evaluation (such as the parent survey instrument, and protocols for conducting focus groups and interviews). This review by the Advisory Group was to ensure the survey and focus group questions were appropriate and on target to collect the desired data.

3. Input on the interpretation and analysis of data gathered by the evaluation. This input helped ensure accuracy in the evaluation team’s interpretation of data findings.

4. Recommendations on any potential modifications to the evaluation’s data collection tools or research questions. Obtaining such recommendations helped ensure that data and lessons learned could inform the continuous improvement of the evaluation over time, and that evaluation research questions were informed by ongoing changes in national, state, and local ECE policies.

5. Insights into the interaction of prior and ongoing state, local, and national policies pertaining to ECE. Members of the Advisory Group were selected in collaboration with DPP leaders to ensure there was an appropriate level of expertise and diversity to provide guidance and input. Advisory Group members were also selected to inform the evaluation and DPP about opportunities to collaborate and communicate with other organizations focused on policy development. For instance, with regard to Colorado’s recent passage of legislation pertaining to universal pre-K, the evaluation’s Advisory Group advised on opportunities to collaborate with Colorado’s Early Childhood Leadership Commission (ECLC) and its relevant subgroups, including the ECLC’s Universal Preschool Implementation Subgroup.

The Advisory Group in 2022-23 was made up of eight Denver residents with personal and/or professional experience in Early Childhood Education. Two advisers self-identified as males and six as females; six as Latino/a or Mexican American, and one as Black; five are fully bilingual (English/Spanish), one primarily an English speaker, and one primarily a Spanish speaker.

Coupled with a diverse array of professional and personal experience and backgrounds related to early childhood education, these advisors provided valuable input into the evaluation’s work in 2022-23.

94

Appendix B: Evaluation Research Questions The evaluation of the DPP operations during year 1 (2022-23), sought to address the following research questions: Perception of DPP and UPK ➢ What are parents’ perceptions and understanding of DPP and UPK? ➢ How do families hear about DPP? How do they learn how to apply for DPP tuition credits? ➢ How can DPP improve its outreach to families?

Customer Service/Interaction with DPP ➢ How do providers describe their interactions with DPP? Concerning tuition credits?

Concerning quality improvement? ➢ To what extent do families have direct interactions with DPP? How do they describe these interactions? ➢ Does the DPP application system make it easy or difficult for providers to participate?

What prevents providers from participating in DPP? ➢ Is accessing DPP easier or harder for families from different socioeconomic and language backgrounds?

Tuition Credit System ➢ Do tuition credits encourage families from all income levels to send their children to preschool? ➢ What levers of the tuition credit system can be changed to make a greater impact on families in Denver? ➢ What families are accessing tuition credits? What families are not accessing the tuition credits?

Choosing Preschool ➢ What factors influence parent decisions when choosing a preschool for their children (e.g., transportation, affordability, hours available, childcare deserts, etc.)? Do these factors vary by parent characteristics (e.g., income level, language spoken by parents, neighborhood where they live)? ➢ What are the primary challenges and barriers to accessing preschool (e.g., transportation, affordability, hours available, childcare deserts, etc.)? Do these barriers vary by parent characteristics? ➢ Do parents feel that their preschool provider offers services that are accessible and culturally responsible?

95

Maximizing DPP Impacts and Access to Childcare ➢ What are parent and provider perceptions of DPP impacts on participating families (i.e., does it increase DPP families’ ability to enter the workforce, find more stable, full-time employment, or pursue higher education opportunities)? ➢ How can DPP operations or funding supports be changed to improve access to childcare in communities that are childcare deserts? ➢ How can provider ratings or other metrics be changed or created to improve access to childcare in communities that are childcare deserts?

Additional question ➢ What do multiple audiences say are the most effective ways to present complex findings tailored to different interests (providers, parents, policymakers, and community leaders)?

96