Recommendations and Conclusions

Lessons Learned from the Early Childhood Comprehensive Systems Collaborative Improvement and Innovation Network: FINAL EVALUATION REPORT

RECOMMENDATIONS AND CONCLUSIONS

RECOMMENDATIONS AND CONCLUSIONS

The Early Childhood Comprehensive Systems Collaborative Improvement and Innovation Network (ECCS CoIIN) was a five-year nationwide effort to improve outcomes in population-based children’s developmental health and family well-being, funded by the Health Resources & Services Administration’s (HRSA) Maternal and Child Health Bureau (MCHB). The National Institute for Children’s Health Quality (NICHQ) and its partners served as the Coordinating Center (CC) for the project, providing capacity-building technical assistance (TA) to the ECCS CoIIN participants. Early Childhood Comprehensive Systems (ECCS) are partnerships between interrelated and interdependent agencies and organizations striving to develop seamless systems of care for children from birth to kindergarten entry at the national, state, and community levels. As such, recommendations span many stakeholders involved in Early Childhood Systems (ECS) building efforts. Recommendations summarized here were shared directly by project teams in evaluation activities or identified by the evaluation team during analysis of different evaluation activities. Recommendations are organized by the overarching theme as it pertained to ECCS CoIIN.

EXPAND, BRAID AND ALIGN CROSS-SECTOR FUNDING STREAMS

Due to the collaborative nature of developing and implementing program and policy changes across different stakeholders, a reported enabling factor to several evaluation areas of the ECCS CoIIN project implementation (Policy Transformation, Partnership Development, State and Local Connections, ECS Capacity Building) was braiding and aligning different funding streams, including federal, state, and philanthropic dollars. Expanding availability of funding sources that encourage collaborative cross-sector work building ECS could engender better circumstances for partnership, capacity building, and policy and program implementation. An expansion of funding sources that promote collaboration could also encourage more buy-in and political will from state agencies and sectors such as the medical community, academia/nonprofits, and executives/legislative leadership, all areas in which ECCS CoIIN participants reported partnership barriers around funding limitations (Partnership Development, ECS Capacity Building).

COORDINATE NATIONAL AND STATE EVALUATION

Though ECCS CoIIN participants demonstrated progress in their efforts to build and strengthen ECS, not all participants focused their ECS building efforts in the same areas. Moreover, some participants’ ECS were more developed at baseline compared to others (evidenced through examples such as some participants already having an integrated data system or preexisting collaborative groups and partnerships). Further, even among participants who focused and measured ECCS progress in the same areas, participants’ individual evaluation strategies were not standardized. As a result, state-level evaluation work from the ECCS CoIIN participants could not be aggregated and compared with one another due to differences in systems maturity, measure collection, and areas of focus.

To support coordinated evaluation of systems building work and examine aggregate results at project end, a coordinated state level and national evaluation strategy around systems maturity could be beneficial. Given differences in the approaches, measures, and focuses for the state-led implementation, technical assistance to align and coordinate state-level evaluations as well as developing a comprehensive national evaluation strategy could allow for better comparability at project end. Specifically, a guiding framework should include flexibility to encompass each state’s ECS maturity, as well as engender cross-state comparability (one such framework is summarized in the ECCS CoIIN Systems Maturity Brief). A coordinated state and national evaluation strategy would allow for results to interpreted and understood within the context of each state’s ECS maturity.

RECOMMENDATIONS AND CONCLUSIONS

DIVERSIFY MEASUREMENT STRATEGIES

While addressing equity was acknowledged in the new ECCS CoIIN Logic model as a guiding principle and was evident in participants’ ECCS activities, participants did not consistently quantify or measure progress in addressing equity and social determinants of health (SDOH) standardly. Thus, while participants understood the importance of infusing equity into ECCS CoIIN implementation and utilized several strategies to support and build equitable systems of care for their constituencies, there were limited opportunities to formally measure and assess equity work in the project (ECS Improvement and Sustainability). Incorporating equity principles into project conceptualization could enhance measurement and addressing diverse issues of disparities, access, and reach in ECS-building initiatives.

Further, when discussing challenges with progress and outcome measurement, participants noted that the overarching measurement strategy chosen for the project was not relevant for all states and communities. Some ECCS CoIIN participants suggested allowing states and communities to select indicators relevant to their population needs, constituency, process, and project goals (ECS Improvement and Sustainability). Incorporating individualized measurement activities could enhance future iterations of the ECCS project and other systems-change initiatives by better assessing and quantifying systems building activities within unique contexts.

Finally, future ECCS iterations may consider focusing on secondary national data sources (i.e., the National Survey of Children’s Health) to track progress at a population level in key indicators related to ECS building and growth (i.e., developmental screenings for children aged 9-35 months, kindergarten readiness, etc.). In addition, technical assistance focused on building and supporting state-level capacity to obtain, analyze, and utilize secondary data sources to drive ECCS implementation could be beneficial. This work could be supported by national data sources in several ways. Because current national data releases often lag by two or three years, which hinders states’ abilities to utilize data to track real-time progress, increased timeliness of national data releases could help address these barriers. Moreover, national data sources could consider increasing sampling strategies to disaggregate data by sub-state geographies and race/ethnicity. Increased data stratification could serve states and localities in better addressing system-level disparities in key outcomes.

INVEST IN FAMILY LEADERSHIP AND FAMILY ENGAGEMENT

Another method to support equity in ECS capacity building is through continued investment in family leadership and family engagement. States that reframed community and family investment from a deficit-based approach to a strengths-based approach shifted their ECS paradigm to harness the wisdom inherent in lived experience to better support community-level assets. As evidenced by the evaluation, many states made a concerted effort to move from family engagement to family leadership, with some expanding their systems building strategies to encompass the active role of families in ECS. Giving communities the opportunity to define the terms of their ECS building would require a fundamental shift in how outside stakeholders often view systems building work but could offer circumstances to better develop equitable ECS that are specifically tailored to unique and diverse population needs.

RECOMMENDATIONS AND CONCLUSIONS

STRENGTHEN INFRASTRUCTURE BUILDING IN PURSUIT OF SYSTEMS MATURITY

Throughout several areas of the ECCS CoIIN project, infrastructure building was discussed as a facilitator to further partnership on the state level, but not on the community level (Partnership Development, ECS Capacity Building). As such, creating more opportunities for local-level partners to participate in infrastructure building was reported as an area for potential partnership development and mission alignment. Other areas of the evaluation (ECS Improvement and Sustainability, State and Local Connections) discussed placing more emphasis on small scale, local-level activities and processes in pursuit of larger state infrastructure goals. Some concrete examples shared included growing the early childhood workforce to expand service coordination, the inclusion of community members in policy and program decision making and supporting cross-sector communication in data system development. Thus, emphasizing concrete local-level activities in pursuit of larger infrastructure goals could be a key area for state agencies to support ECS maturity. Refer to the ECCS CoIIN Systems Maturity brief for a summary of the framework for systems growth developed by NICHQ as informed by the ECCS CoIIN implementation.

INCREASE STAFFING, TIME, AND CAPACITY IN SUPPORT OF PARTNERSHIP DEVELOPMENT

Staffing, time, and capacity constraints were a common reported barrier on both the community and state level within several areas of the ECCS CoIIN project (Partnership Development, ECS Improvement and Sustainability, State and Local Connections, ECS Capacity Building). With partnership building and developing community champions both serving as important activities and strategies in building state and local connections, turnover of key individuals could lead to a breakdown of successful ECS implementation. Expanding staffing opportunities to specifically support cross-sector relationship development and partnership could help address these barriers and guard against turnover of key parties to systems building efforts.