Evaluation of "Being Trauma Aware: Making A Difference in the Lives of Children and Youth"

Evaluation of “Being Trauma-Aware: Making A Difference in the Lives of Children and Youth�

Prepared By Megha Bhavsar, Evaluation Specialist, Centre for Addiction and Mental Health May 2018

Contents Executive summary ............................................................................................. 5 Competency drivers....................................................................................... 7 Leadership drivers ....................................................................................... 10 Organization drivers ................................................................................... 11 Background ........................................................................................................ 13 Evaluation objectives ........................................................................................ 16 Evaluation approach ......................................................................................... 17 New World Kirkpatrick model ....................................................................... 17 Level 1: Reaction ........................................................................................... 17 Level 2: Learning .......................................................................................... 17 Level 3: Behaviour ........................................................................................ 18 Level 4: Results ............................................................................................. 18 Implementation science ................................................................................ 19 Evaluation methodology .................................................................................. 21 Quantitative .................................................................................................... 22 Qualitative....................................................................................................... 24 Findings ............................................................................................................. 26 1. Course and survey completion ................................................................ 26 2. Demographics ........................................................................................... 27 3. Knowledge, awareness and confidence .................................................. 30 1

Module 1: Childhood maltreatment and trauma ...................................... 32 Module 2: Trauma and its impact on brain development ........................ 33 Module 3: Impacts of trauma ...................................................................... 34 Module 4: Trauma-informed practice ........................................................ 37 Module 5: Fostering resilience .................................................................... 39 4. Beliefs ........................................................................................................... 40 5. Effectiveness of learning activities ......................................................... 41 Overall .......................................................................................................... 41 Module 1 ....................................................................................................... 43 Module 2 ....................................................................................................... 43 Module 3 ....................................................................................................... 44 Module 4 ....................................................................................................... 45 Module 5 ....................................................................................................... 45 6. Effectiveness of course delivery and design ........................................... 46 Learning elements ....................................................................................... 47 Time .............................................................................................................. 48 Technical elements...................................................................................... 48 Clarity around completion ......................................................................... 49 7. Effectiveness of content ........................................................................... 49 Diversity and inclusion ............................................................................... 51 Relevance of content to disciplines............................................................ 52 2

8. Application of content to practice .......................................................... 53 9. Challenges and barriers to practice implementation ........................... 55 Reflection, discussion and actions ............................................................. 56 How to move forward .................................................................................. 57 10.

Collaboration across sectors ................................................................. 58

11.

Organizational readiness for trauma-informed practice .................. 62

12.

Organizational Perspective................................................................... 71

Willingness to learn ..................................................................................... 71 Integration and collaboration.................................................................... 72 Knowledge and previous training .............................................................. 72 Barriers ......................................................................................................... 73 Summary of findings ........................................................................................ 73 Determine the value and effectiveness of curriculum content and delivery methods for helping front-line service providers improve their practice ......................................................................................................... 74 Increase knowledge and awareness of child abuse, trauma, impacts on the victim, family and community ............................................................ 75 Increase confidence in addressing issues of child abuse and trauma .... 76 Determine the extent to which the curricula reflect the diverse clientele that each discipline services ....................................................................... 76 Increase recognition of the reach and impact of child abuse to multiple sectors and the value of working collaboratively across sectors ............. 77

3

Determine the extent to which an integrated and collaborative approach is achieved when working within the target population ......................... 78 Determine if curriculum was implemented with fidelity to the design . 78 Determine the feasibility of scaling practice changes .............................. 79 Limitations......................................................................................................... 81 Recommendations ............................................................................................ 82 Competency Drivers .................................................................................... 82 Leadership Drivers ....................................................................................... 85 Organization Drivers ................................................................................... 86 Conclusion ......................................................................................................... 87 Acknowledgements ........................................................................................... 88 References .......................................................................................................... 89 Appendix A: Knowledge, awareness, confidence ............................................. 90 Appendix B: Module-based measures ............................................................... 92 Appendix C: Beliefs ............................................................................................ 96

4

Executive summary This report provides findings, analysis and recommendations from the evaluation of the online course, “Being Trauma-Aware: Making A Difference in the Lives of Children and Youth.”

The eight evaluation objectives were organized within four domains: knowledge, awareness, confidence and commitment; relevance to discipline and clients; cross-collaboration strategies; and, scaling and future development. To develop indicators, the evaluation approach was grounded in two models: New World Kirkpatrick’s Model and Implementation Science.

A mixed-methods approach was taken, using quantitative and qualitative data, and data was collected through registration, surveys and focus groups.

Findings show diversity in the sectors and professions represented. The majority of participants worked in their sector less than ten years (62%, n=375), while the remainder worked in their sector more than ten years.

Most participants reported not having received previous training in traumaawareness (70%, n=375). Among those who had not received previous training, the trend follows their time in sector: more participants reported having received training the longer they were in the sector.

Participants showed high knowledge and awareness of child abuse, trauma and its effects. However, there were some knowledge gaps with respect to: what trauma means in the context of child advocacy centres; impacts of intergenerational trauma; stepping stones to being trauma-informed; and resilience. In addition, participants reported increased confidence in addressing issues of child abuse and trauma (+37%, n=375). Findings also show that the participants’ knowledge and confidence fell at the three5

month follow-up, indicating the needs for ongoing reviews of course material and opportunities to discuss, reflect and act on the material.

Participants understood the reach and effect of child abuse and trauma to multiple sectors, and valued collaborating across sectors. They believed that child advocacy centres model this well. Participants shared challenges and barriers to collaborating across sectors, which require organizational and systematic support.

Overall, participants found the program valuable as a foundational course for multiple sectors to develop a common language about trauma and its impacts. Participants praised the design and delivery, and reported that the program reflected the diverse clientele that most organizations serve. They provided feedback for improvements to enhance learning, as well as ideas for additional training.

At the three-month follow-up, it was too soon for changes in policies and procedures to have happened, and for the development of formal crosssectoral collaborations. However, findings support the potential for an integrated and collaborative approach.

The following quotes highlight the participants’ experiences with the course: “[The] biggest takeaway for me - it has enlarged my compassion for the clients and made me think more about self-care and self-compassion.”

“People were commenting how valuable it was, people who hadn’t taken any courses or exposed to this formally.”

6

“The course has opened my eyes to the prevalence of trauma and its effects on society at large. Furthermore, this course has helped to better inform my practice when relating to and working with clients.�

Based on the evaluation findings, and the desire to rollout the course broadly and influence systemic change, the following 36 recommendations are made, framed by the three implementation drivers necessary for the success of an intervention (NIRN, 2014): Competency drivers

Course delivery 1. Based on challenges accessing training due to cost and distance, offering this training free to organizations across a wide range of sectors, including those in the pilot phase.

2. Offer the training online, which allows participants to complete it at their own pace and reaches a wider range of people (e.g., shift workers).

3. Allow access to course after upon completion to review material as some may prefer accessing the course online.

4. Consider developing summaries of modules with key take-away points so that participants have access to the information without having to log in if they prefer that.

5. To enhance learning, share facilitation guides with organizations and ensure that the guides provide discussion questions so that organizations can take the course together, or reflect/discuss topics together 6. Provide guidance on how the course could be shared with clients and families so it is not overwhelming or re-traumatizing, and service providers can answer questions or direct them to the necessary resource.

7

Course design and content 7. Maintain variety of learning elements (various media, quizzes, personal stories, examples, case studies) as these helped in engaging participants.

8. Consider developing exercises to be more challenging.

9. Maintain the simplicity, clarity and informativeness of content, as well as build-up from Modules 1 to 5.

10. Maintain flow of content, but consider improving transitions between conversations with characters to videos of real-life stories.

11. When collecting written information, provide transparency regarding confidentiality. This could include redeveloping or reframing these activities as personal journals that are exportable (if such function can be developed).

12. While the Sheldon Kennedy videos were deemed impactful, their inclusion of some offensive language upset some viewers. Consider warnings, removal or replacement.

13. Set expectations regarding the time for completion, as it exceeded the advertised 90 minutes.

14. Provide timing for each module and a progress bar, so as learners progress they are aware of how much time is expected and how much time is left. Communicate ahead of time that learners will not be able to pick up where they left off, if they must stop in the middle of a module.

15. Enable options to rewind parts of modules to review concepts that were missed or hard to follow given the diversity of learning styles.

With respect to skipping sections, it is recommended that caution be taken with such an option as learners may skip ahead and hamper their learning.

16. Provide closed captioning and/or connect to Google Translate to view content in another language to aid diversity of learners, potentially 8

those whose primary language may not be English or those who may have accessibility needs.

Future trainings 17. Consider developing courses that extend from this initial training through advanced training opportunities, particularly intergenerational trauma.

18. Consider offering trainings on the following trauma-related topics: a. Loss and grief, loss of culture b. Culture/community specific trauma c. Impact of post-migration on previous traumas among newcomers d. Trauma experienced by Indigenous people and the integration with the Truth and Reconciliation Commission report e. Impact of trauma for LGBTQ children/youth 19. Ensure trainings do not duplicate efforts and are integrated into current professional development plans, given workload issues and compassion fatigue.

Evaluation 20. Integrate pre- and post-module surveys and completion surveys for future evaluation purposes, and to provide participants with an opportunity to check-in with their learning.

21. Consider providing participants with immediate feedback to be used as a learning tool and to prompt review of concepts not easily understood.

22. Provide a summary of the project, including high-level evaluation findings, to participants who provided their data. Engage participants in future developments, with next steps and updates.

9

23. If follow-up evaluation will be included in future iterations, consider engaging participants through incentives or direct engagement to increase response rate in follow-up activities.

Leadership drivers

Broad engagement 24. Train staff at all levels, including supervisors, management, administrative staff, within and across organizations to ensure a common and consistent language applied.

Time 25. Learners require protected time to complete trainings such as these.

SKCAC should promote the need for protected time when connecting with organizations.

Awareness of trauma 26. Train leadership on course content as they are positions to facilitate change processes. Given that trauma is not a mainstream issue, having leadership across sectors on the same page could propel action. A topdriven approach may tackle issues related to staff resistance to change.

27. Provide opportunity (e.g., through administrative and team meetings) to discuss trauma-informed practices.

Systems integration 28. Post-secondary educational programs should consider opportunities to embed trauma-aware principles in their curriculum and facilitate cross-sectoral integration as early as possible. By having a shared understanding of trauma-aware principles guiding their work, there are increased opportunities to explore cross-sectoral integration and collaboration.

10

29. Provide opportunities for staff across sectors to train together in future training courses, with active opportunities for interprofessional and cross-sectoral collaboration.

30. Consider building awareness in the community, perhaps through a collaboratively-developed awareness campaign on trauma and its effects.

Organization drivers

Buy-in 31. This set of recommendations require that leadership and staff agree on the importance of being trauma-aware in their practice. Direct engagement activities should be conducted to develop buy-in.

32. In scaling the course, SKCAC should use champions from the pilot phase representing a variety of organizations and sectors, which could promote buy-in from all levels. This should include leadershiplevel champions as well as staff-level champions to reduce resistance to change.

33. Use champions from the pilot phase to help in developing necessary material (as outlined in course delivery recommendations). This could take the form of a train-the-trainer model to enhance the application of the content across a variety of contexts and to increase participant engagement and recruitment across sectors and levels.

Marketing 34. When scaling the program, ensure a multi-faceted marketing strategy is in place to reduce the gap between the availability of training and awareness that such training exists.

Supporting policies and procedures 35. Trauma-informed policies and procedures need to be developed and in place to support staff in applying course content to practice. SKCAC 11

should follow-up with representatives from organizations three to nine months after project completion (i.e., at the six month or one year mark) to understand changes (if any, as well as barriers) and provide consultation policy development.

Evaluation 36. Future scaling should emphasize longitudinal evaluation (e.g., six months to a year) to determine if and how organizations have changed. This may require partnership and investment of participating organizations in these shared evaluation efforts.

12

Background The Sheldon Kennedy Child Advocacy Centre (SKCAC) is a Calgary-based notfor-profit organization, established to create a community that responds collectively to child abuse and empowers those who are affected by child abuse to lead healthy and productive lives. To fulfill its mandate, SKCAC receives information and resources from partner agencies: the Royal Canadian Mounted Police (RCMP), the Calgary Police Service (CPS), Alberta Child and Family Services, Calgary Region (CFS), Alberta Health Services (AHS) and Alberta Justice and Solicitor General. SKCAC also works in collaboration with partner agencies in Alberta to best serve children and families involved in the investigation and treatment of child abuse.

In 2016, the Public Health Agency of Canada (PHAC) awarded SKCAC a twoyear grant to develop, deliver, evaluate and disseminate an interactive, webbased and foundational training program, which would provide consistent and introductory teaching about childhood maltreatment and trauma, and would support the wide range of professionals working with children who may have been affected by childhood maltreatment and trauma. To ensure that the pilot on-line training program would be applicable, relevant and valuable to all target audiences, a Request for Proposal (RFP) for evaluators was developed.

Located in Toronto, Ontario, the Centre for Addiction and Mental Health (CAMH) is Canada’s largest mental health and addiction teaching hospital.

CAMH focuses on clinical care, research, education, health promotion and health policy to achieve its mission of helping to transform the lives of people affected by mental health and addiction issues. CAMH works with 13

partners to provide trauma-informed, client-centered care, and its reach extends throughout the province, the country and internationally. Given the alignment in missions with SKCAC, the Education Evaluation team at CAMH responded to and received the RFP for “Fundamentals of Trauma-Informed Practice – Evaluation of Online Training Program” in October, 2016.

The project was a collaboration between SKCAC, the curriculum developers (Performance Group), the production company (Respect Group Inc.) and the Education Evaluation team from CAMH (herein: “Evaluation team”). The Content Advisory Team included subject matter experts from across Alberta (e.g., government, school boards, religious organizations, immigration/newcomer centres and child advocacy centres) and Canada (Canadian Mental Health Association and Canadian Centre for Substance Abuse). In April, 2017, to better reflect the content and purpose of the training, SKCAC and its collaborators renamed the program, “Being TraumaAware: Making A Difference in the Lives of Children and Youth” (herein: “Being Trauma-Aware”).

The course launched in October, 2017, and was completed by the end of December, with follow-up evaluations completed by the end of March, 2018.

The course was delivered to staff and teams in two phases: Initial phase (October, 2017): Participants came from the Sheldon Kennedy Child Advocacy Centre, the Zebra Child Protection Centre and emerging child advocacy centres in Central and Southern Alberta.

Second phase commenced six weeks later (in November, 2017), and was delivered to participants from community organizations in Alberta, Calgary 14

Board of Education, Calgary Catholic School District, Calgary Police Services and RCMP. The program evaluation measured the extent to which the on-line training program educated, informed and enhanced trauma-aware practice among front line service delivery providers of children, youth and families affected by abuse. The evaluation objectives aligned with curriculum objectives.

15

Evaluation objectives Evaluation covered four areas of interest: 1. Knowledge, awareness, confidence and commitment:  Determine the value and effectiveness of curriculum content and delivery methods for helping front-line services providers improve their practice.  Increase knowledge and awareness of child abuse, trauma, impacts on the victim, family and community.  Increase confidence in addressing issues of child abuse and trauma (via service provision or referral).

2. Relevance to discipline and clients:  Determine the extent to which the curricula reflect the diverse clientele that each discipline services including the lived experiences of children, youth and adults of First Nations, immigrant groups and disenfranchised/marginalized groups.

3. Cross-collaboration strategies:  Increase recognition and impact of the reach of child abuse to multiple sectors and value of working collaboratively across sectors.  Determine the extent to which an integrated and collaborative approach is achieved when working within the target population.

4. Scaling and future development:  Determine if the curriculum was implemented with fidelity to the design.  Determine feasibility of scaling practice changes.

16

Evaluation approach Two approaches guided the process and outcome evaluation: The New World Kirkpatrick model (Kirkpatrick et al., 2016) guided the development of indicators for measuring individual-level change.

Implementation science guided the development of indicators for measuring organizational-level change.

New World Kirkpatrick model The evaluation team chose the New World Kirkpatrick model because of its prevalence within education, and the literature review supported this choice. The model consists of four levels (see fig. 1): Level 1: Reaction

This measures the degree to which participants are satisfied with the training and find it engaging and relevant to their practice. Domains: Customer satisfaction, engagement and relevance.

Level 2: Learning

This measures the degree to which participants acquire the intended knowledge, skills, attitudes, confidence and commitment through the training. Domains: Knowledge, skills, attitudes, confidence and commitment.

17

Level 3: Behaviour

This measures the degree to which participants apply what they have learned to their practice. This depends on the processes and systems in place that allow the participant to apply their learnings. Domain: Required drivers.

Level 4: Results

This measures the degree to which targeted outcomes occur as a result of the training and the support and accountability package. Domain: Leading indicators.

Figure 1: New World Kirkpatrick model

Copyright 2009 - 2016 Kirkpatrick Partners, LLC. The model begins with the desired results in mind and measures the extent to which the program achieves the anticipated outcomes (described in the work plan and program logic model). The model is most effective when evaluation is considered during program design and implementation phases. This is when program planners determine the knowledge, attitudes and skills that must be in place to bring about the desired behaviours. The training program must be received favourably among participants for knowledge, attitude and skill development to occur.

18

Implementation science Implementation science is the study of the conditions necessary for a program to be implemented effectively. The evaluation team used this framework to understand how the on-line training program was applicable, relevant and valuable to all target audiences. An implementation science framework is critical to:  investigating strategies implemented to practice in organizations  informing recommendations about the feasibility of scaling such practice changes.

Implementation science involves three drivers, described as the “engine” behind implementation (Fixsen et al., 2005) (see fig. 2): 1. Competency drivers (i.e., the “people factor”) include staff selection, training and coaching. They ensure the requisite skills and support are in place so any intervention is practiced as intended with consistency.

2. Organization drivers (i.e., the “structural factor”) include systems interventions, facilitative administration, and decision support data system. They address the local and larger political environments, policies and procedures, funding and cultural issues, and data systems.

3. Leadership drivers (i.e., the “directional factor”) include good management of issues (technical) and guidance through complexity and change (adaptive) (NIRN, 2014).

19

Figure 2: Implementation science framework

Implementation science fit well with the New World Kirkpatrick Model, specifically its levels three and four (behaviour and results), as neither can be realized without the necessary drivers in place.

20

Evaluation methodology The evaluation team worked with SKCAC, Respect Group Inc. and the Performance Group to execute an evaluation of “Being Trauma-Aware” They reviewed the logic model and existing evaluation strategy to understand the design of the educational intervention and how best to situate the evaluation to achieve its objectives. They developed an evaluation framework for individual- and organizational-level indicators, and reviewed the corresponding survey measures with SKCAC and the Performance Group, who provided expert input. They created a privacy document in collaboration with SKCAC, which outlined how consent to participate in the program evaluation was obtained and how confidentiality and privacy of individuals and organizations would be maintained.

The evaluation team conducted a literature review (“Literature Review: Evaluating trauma-informed care education in child advocacy centres”) to determine best practices in evaluating trauma-informed practice and partnerships. This provided necessary information for the appropriate design and implementation of the evaluation (e.g., validated measures and potential indicators used in similar educational courses). The review found evaluations of trauma-informed training grounded in the New World Kirkpatrick Model (though limited to the first two levels) and supported an implementation science approach. The review also helped anticipate challenges related to implementation of training material into practice. The team submitted the literature review to SKCAC, and can make it available upon request (contact Project Coordinator, Brenda Neis).

21

Through online surveys and focus groups (via teleconference and in-person options), the team used a mixed-method approach to collect quantitative and qualitative data from pilot participants. Two groups from the pilot requested a facilitated in-person delivery of the program. Therefore, these groups received paper-pen surveys.

Quantitative Online registration data determined demographic information (e.g., profession) as well as registration and completion rates throughout the duration of the project. This data also helped to describe the participant profile and related levels of engagement.

Implemented via online learning management software, several online surveys collected evaluation data throughout the project:    

Baseline survey Five post-module surveys Completion (i.e., post-course) survey Three-month follow-up survey

This method increased access and availability of the surveys to participants.

For evaluators, this also decreased data collection and entry time and effort, which lowered costs and potential data entry errors, compared to traditional paper-pen surveys.

Quantitative data provided insights on changes in participants’ knowledge, awareness and confidence. Specifically, these surveys helped determine if there was any increase in:  knowledge and awareness of child abuse, trauma and the effects on the victim, family and community 22

 awareness of the reach of child abuse to multiple sectors, and of the value of working collaboratively across sectors  confidence in addressing issues of child abuse and trauma.

The post-module surveys provided an opportunity to collect information on participants’ satisfaction with:  the training content and delivery methods  the perceived relevance of the content  the extent to which they believe the curricula reflects the lived experiences of their diverse clientele, including First Nations, immigrant groups and disenfranchised/marginalized populations (Kirkpatrick Level 1: Reaction).

The post-module surveys also evaluated the participants’ commitment to applying what they learned through the training (Kirkpatrick Level 2: Learning) and the extent to which they applied the training to their practice (Kirkpatrick Level 3: Behaviour). Measures aligned with learning objectives for each module, as well as overall course outcomes.

The literature review identified two organizational assessment tools, which the team adapted: the Trauma-Informed System Change Instrument (TISCI) and the Trauma System Readiness Tool (TSRT). These evaluated the readiness for trauma-informed change (e.g., through staff training supports and supervision), team culture (e.g., supportive environment, communication channels), policies in place to support a trauma-informed space and engagement of current or former consumers. The team conducted this organizational assessment prior to the commencement of the online training, and then again, three months after the training, to determine any changes due to participation in the online training. The pre-post

23

organizational assessment surveys were also delivered online, via a survey website.

Middle-management participants from pilot organizations completed a prepost organizational-assessment. Using the assessment scores for organizations, the team identified strengths and gaps based on the drivers within implementation science. This informed the necessary supports required to promote an integrated approach to working with SKCAC’s target population and the extent to which there was:  increased cross-sectoral collaboration  effective information sharing strategies  emerging partnerships and collaborations.

All survey data was analyzed using IBM SPSS Statistics 24. Descriptive and inferential statistical data methods were used. For non-parametric data (e.g.

5-point Likert Scale data), a Wilcoxon sign rank test was conducted to test for statistical significance among matched pairs. For parametric data, a paired sample t-test was used. Due to variability in response rates at follow-up, baseline and completion survey data was compared, and completion and three-month follow-up data were compared. Analysis was also conducted by sector; however, there was low variability.

Qualitative The team conducted online and in-person focus groups three months after the training ended. These contributed to understanding the:  participants’ experience with the online training program, including their satisfaction and the perceived value for their discipline and areas of improvement 24

 degree to which participants applied what they had learned to their practice and the associated outcomes  effects of the online training program on establishing informationsharing strategies, partnerships and cross-sectoral collaboration.

The surveys built into the online training also contained qualitative measures aimed at capturing participant satisfaction, training improvement suggestions, and barriers and challenges to implementing material in practice.

25

Findings This section covers the following topics: 1. Course and survey completion 2. Demographics 3. Knowledge, awareness and confidence 4. Beliefs 5. Effectiveness of learning activities 6. Effectiveness of course delivery and design 7. Effectiveness of content 8. Application of content to practice 9. Challenges and barriers to practice implementation 10. Collaboration across sectors 11. Organizational readiness for trauma-informed practice 12. Organizational strengths 1. Course and survey completion There were 701 registrations of the course, and 412 were certified as completed. Participants had the option to opt out of the surveys, and others may have registered but not completed surveys. There were 375 baseline, post-module and completion surveys received (355 from online, 20 from the workshop). For the three-month follow-up survey, response rate was low at 10% (39/375).

For the organizational assessments, the pre-organizational assessment was intended for middle-managers (however, some organizations forwarded the survey link to multiple team members). In total, 54 pre-organizational assessments were received. As a result, respondents were requested to enter their email addresses to conduct a concentrated follow-up survey. The

26

response rate for the post-organizational assessment, conducted threemonths after the course ended, was 18% (9/49).

2. Demographics Participants provided postal codes, which indicated the majority of participants (99.7%) were from Alberta, with the remaining from British Columbia (n=354). The cities represented included Calgary (47%), Lethbridge (20%), Red Deer (11%), Cochrane (4%), Edmonton (3%). The remaining 15% comprised other cities.

Participants provided their sector (see fig. 3). Those who reported “other� worked in shelters, faith-based organizations or immigrant serving sector, or they were students.

Figure 3: Sectors represented (n=374)

27

Participants provided their profession (n=375). The variety of professions included those working directly with children (e.g., teachers, social workers and nurses), and those working to support children and their families without providing direct care (e.g., kitchen/cook, administration/reception staff and program/project staff). Interestingly, only one physician and 13 practicum students completed the training. The most common professions reported were social workers (21%), police/RCMP (10%), teachers (6%) and case worker/manager/supervisor (6%).

Slightly over a third of the participants (35%) indicated working in their sector for less than five years, while just over a quarter (27%) indicated 5-10 years, and the remainder indicated more than ten years (see fig. 4).

28

Figure 4: Length of times participants have worked in their sector (n=375)

More than two-thirds (70%) reported not having received previous training in being trauma-aware (n=375). However, more participants reported having received training the longer they were in a sector, compared to those with fewer years in a sector (n=263) (see fig. 5). Of those who reported having received previous training, a common theme was that any training had been based on their educational and professional backgrounds. The most commonly mentioned trainings included “Brain Story�; Alberta Health Services training on trauma-informed care, non-violent crisis intervention and cognitive based therapy training; a three-day workshop by Dr. Bruce Perry.

Figure 5: Participants who reported that they have not had training in traumaaware (n=263)

29

3. Knowledge, awareness and confidence Using a five-point Likert scale (i.e., strongly disagree, disagree, neutral, agree, strongly agree), participants rated their agreement to nine statements about their knowledge and awareness of child abuse, trauma and its impacts, and confidence in addressing issues of child abuse and trauma. For each statement between baseline and completion, the change in participants reporting agreement (agree/strongly agree) was statistically significant (see fig. 6).

Figure 6: Participants’ agreement to statements related to their trauma knowledge, awareness and confidence

30

*Statistically significant at completion; †Statistically significant at follow-up The three-month follow-up survey indicates decreases in knowledge, awareness and confidence ranging from 2% to 19%. The largest decrease was among those reporting that they felt confident in addressing issues of childhood maltreatment and trauma. Although these results are statistically significant, it is important to note that the small sample size may affect the ability to make strong conclusions related to maintenance of knowledge,

31

awareness and confidence at follow-up. Appendix A shows the scores for the tests of significance.

For each module, a number of knowledge measures were implemented aligning with learning objectives. The following findings are organized by module for clarity.

Module 1: Childhood maltreatment and trauma

Measures in Module 1 aimed to gauge knowledge by asking participants to define:  trauma in the context of child advocacy centres  complex trauma.

The majority of participants had high understanding of complex trauma but difficulty correctly defining “trauma” in the context of child advocacy centres (see fig. 7).

As a “check all that apply” measure, participants chose a variety of options. A reporting challenge came from the option “post-traumatic stress disorder (PTSD),” which is experienced primarily by adults (i.e., not strictly accurate for the child advocacy context). At baseline and post-module, the majority of participants selected PTSD, which explains the drop between baseline and post-module of those correctly answering this question.

Figure 7: Participants correct on childhood maltreatment and trauma knowledge measures

32

‡N/A for statistical testing Module 2: Trauma and its impact on brain development

In Module 2, three measures gauged knowledge:  Select “prefrontal cortex” as the correct term.  Identify the statement “positive stress and toxic stress impact the brain architecture” as true.  Select “plasticity” as the correct term.

At each time-point, the majority of participants correctly identified each knowledge measure in Module 2 (see fig. 8). The 6% increase between baseline survey and post-module (for the measure related to plasticity) was statistically significant.

Figure 8: Participants correct on trauma and its impact on brain development knowledge measures

33

*Statistically significant at completion Module 3: Impacts of trauma

There were six knowledge measures in Module 3:  Define adverse childhood experiences.  Select the following statements as true: o “Childhood maltreatment is a risk factor for early onset of substance use later in life.” o “There are greater rates of alcohol, drug use, suicide, attempted suicide among adolescents who experience childhood maltreatment than those who do not.”  Select all the domains of impairment.  Identify the potential impacts of adverse childhood events in adolescence and adulthood.  Define intergenerational trauma.

The majority of participants selected the correct option to most of the knowledge measures. The measure related to intergenerational trauma had the greatest variability in responses (see fig. 9). Further analysis of this 34

measure showed that at post-module, only law enforcement participants showed an increase in this measure (+28%), while all other sectors saw a decrease. The challenges came from identifying that intergenerational trauma is not only trauma that can be passed down through generations, but that it can also be:  the result of being removed from primary caregivers  due to a trauma experienced by a community.

Figure 9: Participants correct on trauma and its impact on brain development knowledge measures

35

36

*Statistically significant at completion; †Statistically significant at follow-up; ‡N/A for statistical testing Module 4: Trauma-informed practice

Module 4 contained three knowledge measures:  Match four trauma-related terms (trauma-informed, -aware, responsive, -sensitive) to their definitions.  Select re-traumatization as the correct response.  Select the correct actions to help children feel safe when they disclose information.

On matching trauma-related terms to their definitions, there were increases of 5%-39% from baseline to post-module, showing that participants had more knowledge for these terms after taking the course. The exception involved the term “trauma-sensitive,” which remained stable at baseline and completion. The changes for “trauma-informed” and “trauma-aware” were statistically significant at post-module.

Participants demonstrated high knowledge for re-traumatization at each time point. For actions to to help children feel safe when they disclose information, there was an increase in the percentage of participants who understood correct actions (see fig. 10).

Figure 10: Participants correct on trauma-informed practice knowledge measures

37

*Statistically significant at completion;‥N/A for statistical testing 38

Module 5: Fostering resilience

Module 5 included these knowledge measures:  Define resilience.  Select the following statements as true” o “The single most common factor to foster resilience is at least one stable and committed relationship with a supportive adult.” o “An individualized self-care plan is different for everyone and includes things that we can do to take care of ourselves in efforts to prevent the effects of vicarious trauma.”

Participants demonstrated high knowledge of the latter two statements.

However, in correctly defining resilience, the majority of participants struggled (see fig. 11).

Figure 11: Participants correct on trauma and its impact on brain development knowledge measures

39

*Statistically significant at completion; ‥N/A for statistical testing 4. Beliefs Participants rated their agreement on 5-point Likert scale to four statements about their beliefs about the value of working collaboratively across sectors.

For each statement, the majority of participants agreed (agree/strongly agree) across the three time points (see fig. 12) Figure 12: Participants’ agreement regarding collaboration across sectors

*Statistically significant at completion

40

5. Effectiveness of learning activities Overall

The online training was very well-received by the pilot participants, who indicated delivery and content as clear strengths. Here are some of the comments from participants:  “The content and the way it was laid out was very well done.”  “I thought it was excellent, too. I really appreciate the expertise, when you can see the number of organizations that put their best experts towards the curriculum and development and unique to the Canadian environment…”  “People were commenting how valuable it was, people who hadn’t taken any courses or exposed to this formally.”  “[The] biggest takeaway for me: it has enlarged my compassion for the clients and made me think more about self-care and self-compassion.”  “[W]e remember the real life scenarios rather than the words on the screen. The combination of doing those two things in the training was good.”

Participants reported that the training was “easy to follow,” “user-friendly,” and “engaging and interactive.” The online delivery allowed participants the convenience of working at their own pace and not having to finish the entire training in one go, and it did not require travel to attend. Participants liked the ability to access training at any time, a quality underscored by shift workers. Additionally, participants said it was an asset to offer the pilot program and the broader roll-out for free.

Participants considered the learning elements to be effective for their overall learning. These elements included various media, quizzes at the end

41

of the modules, graphics and personal stories. This is corroborated with survey data from post-module and completion surveys.

Overall, participants rated each component of the course effective to their learning:  94% reported that the module content and learning objects were effective  89% reported that the case studies and the online course delivery were effective  82% reported that the exercises were effective (see fig. 13).

Figure 13: Participants ratings of learning elements as effective to their learning

Note: Percentages <5% are not labeled in figures.

42

Participants also provided insights on the effectiveness of specific module components.

Module 1

Participants rated the most effective activity of the Module 1 to be the one involving SAMHSA’s definition of trauma. They rated “using the traumainformed lens” less effective (see fig. 14). Note that percentages less than 5% are not labeled.

Figure 14: Effectiveness of Module 1 learning activities

Module 2

Participants rated the videos (88%), matching statement exercises (84%) and “filling in the brain” diagram (81%) as being effective, while fewer felt that way about the exercise related to “serve and return interactions” (68%) (see fig. 15). Note that percentages less than 5% are not labeled.

43

Figure 15: Effectiveness of Module 2 learning activities Module 3

More than three-quarters of participants rated each activity in Module 3 as being effective to their learning. However, there was more variability with the ACE questionnaire (see fig. 16). This is unsurprising given qualitative findings related to privacy and transparency concerns over entering personal information in the online system. Note that percentages less than 5% are not labeled.

Figure 16: Effectiveness of Module 3 learning activities

44

Module 4

A high proportion of participants rated each learning activity in Module 4 as being effective (see fig. 17). Note that percentages less than 5% are not labeled.

Figure 17: Effectiveness of Module 4 learning activities

Module 5

Despite high ratings of effectiveness, there was slightly more variability in the proportion of participants who found the case study in Module 5 to be effective (see fig. 18). This may be linked to earlier findings on resiliency as a gap in knowledge, or later comments related to length of time required to complete modules. Note that percentages less than 5% are not labeled.

Figure 18: Effectiveness of Module 5 learning activities

45

6. Effectiveness of course delivery and design Several themes emerged from the feedback that participants provided on elements related to the program delivery and design:            

Ability to complete at their own pace Interactive and engaging course Online delivery User-friendly format Ability to check understanding with check-ins within the module and surveys Ability to learn through a variety of learning mechanisms Videos including the personal stories from Sheldon Kennedy Examples Exercises Case studies Graphics/animations including following the main characters Flow of content, module to module

Here are some of the comments participants provided:  “The course is different than other online course that I had taken. It is interactive.”

46

 “The delivery is soft and compassionate and the information is very sound.”  “Excellent example of a quality online learning experience by incorporating videos, scenarios and information.”  “I liked the build-up from Module 1, which was very basic, to Module 5, which was very specific.”  “I also liked the opportunity to ruminate on what I was learning days after, before going to the next section.”  “The visuals left an impression on me. [They’re] simple to understand.”  “The personal stories lend humanity to the content. [W]hen you hear people’s stories, [there are] emotions and connections.”

Participants identified four key areas of improvement: 1. Learning elements o Exercises o Feedback o Language 2. Time 3. Technical elements o Ability to rewind/forward o Transitions 4. Clarity on completion certificate requirements Learning elements

Participants suggested including more challenging exercises and delving deeper into each topic. They also wanted on-the-spot feedback on quizzes, which would help them accurately and objectively assess their understanding. A few participants highlighted potentially offensive language in some videos and suggested adding a warning or removing them.

47

Time

Participants reported that it took more than the advertised 90 minutes to complete the training. When asked what could be removed, participants reported that the content was excellent and nothing should be removed.

Instead, they suggested setting realistic expectations about length of the training as a whole, and each module more specifically. In addition, a progress bar within each module would be helpful.

The short time-frame for completing the program for the pilot phase added to this challenge. Participants also reported not always receiving protected time to complete the training.

Technical elements

One participant suggested having better transitions between videos featuring conversations with characters and those of Sheldon Kennedy.

Participants felt restricted by the inability to rewind or forward parts of the modules, especially in four types of learning situations: 1. Technical difficulties led some screens to freeze, and participants would have to restart the entire module rather than pick up where they left off. Participants seemed to understand that as a pilot there are opportunities to smooth over the technical issues.

2. Challenges with time also led to restarting the entire module rather than picking up where they left off.

3. Some learners may have English as their second, third or fourth language, and might need to review certain parts of videos. Other suggestions included closed captioning, connecting to a translation service (e.g., Google Translate), or automatically displaying captions and text in different languages.

48

4. Learners who already received training on foundational concepts might wish to skip sections.

Participants suggested communicating ahead of time that they would be unable to pick up where they left off, or otherwise setting the expectation that the module might not be completed in one sitting. For example, here is one participant’s comment: “There should be a way of marking where you left off. It wastes time to rewatch something.”

Clarity around completion

To enhance their learning, participants wanted to access the training again after completion to review the training material and summaries of each modules, as well as ideas on application.

Although participants appreciated the opportunities to reflect on their own traumas, they indicated a need for more clarity in the information being collected. For example, the open boxes for the self-care plan should be explicitly prefaced with information about privacy.

In addition, participants wanted clarity regarding the certificate and how all aspects must be complete to be certified.

7. Effectiveness of content Participants saw value in providing a common language in matters relating to trauma. Many considered this the most valuable part of the training.

Participants reiterated that the training provided “a solid foundation” and was a “good refresher” to being trauma-aware. The use of plain language contributed to learning and ease of understanding. Participants praised the 49

training for teaching about trauma and its effects on brain development and on life.

Their feedback regarding the content contained three key themes: 1. Content was thorough and informative.

2. Content, including language and narration, was clear, simple and easy to follow.

3. Content was relevant to practice.

Here are some of the comments from participants:  “It was easy to understand – plain language – [I] didn’t have to look up terminology.”  “Some pieces were repetitive for me but then I remember for other staff members it is new.”  “Quite comprehensive…a good foundational tool to learn about brain development, how trauma affects that and that the trauma can be lifelong. [D]amages can last longer on [the] brain…a lot of people aren’t aware of that.”  “I put the seven domains of impairment on a post-it on my monitor.”

Participants reported that the following content pieces left an impression on them:  Vicarious trauma  Self-care and self-compassion  Reflecting on their own trauma With respect to content that was missing, participants felt that the following topic areas could be covered in this or subsequent training:    

Loss and grief Loss of culture Trauma associated with specific communities Post-migration/newcomer trauma 50

 Intergenerational trauma Diversity and inclusion

Participants praised the training for representing diverse clientele such as newcomers, women recovering from violence, and Indigenous people.

Participants felt the content reflected diversity effectively and did not “pigeon-hole any one group.” Here are two comments in this regard:  “…the course seems to reflect diversity. Our programming contains a third Indigenous and a third identify as having immigrated to Canada…always open to learning more about working with folks from many backgrounds.”  “…The population I work with is not overly diverse, but it is getting more diverse. I think the presentations reflected a number of different diverse populations and the concerns for some of those populations…it will help our teachers to reflect about the possible differences that they may see or interact with in the future.”

Participants made the following suggestions for making the course more representative:  Cover different types of trauma, such as war or killings, that people from specific backgrounds/communities experience, and the associated experiences (e.g., intergenerational trauma).  Reflect post-migration trauma and the effects of acculturation on previous trauma among newcomers.  Further address the loss of culture, intergenerational trauma, trauma experienced by First Nations, Metis and Inuit.  Provide information on the effects of trauma for LGBTQ children/youth.

Here are three related comments from participants: 51

 “I think you tried if I’m being honest. I work with immigrant youth and families and there’s a lot of backgrounds: Sudanese communities, East Africa, West Africa. You can’t touch on every single community but I wish there was more about the different forms of trauma …I think you tried. I think it could have been broader.”  “…I noticed that there were people missing. The culture brings specific trauma because of where they’re from…”  “A whole module related to the TRC (Truth and Reconciliation Commission of Canada) is missing…the timing is even right for the TRC.” Relevance of content to disciplines

Participants found the training relevant, especially in learning about using a trauma-informed approach with children and their families, and even with the participants’ colleagues. Participants also noted the relevance of vicarious trauma, self-care and intergenerational trauma. Here are two related comments:  “I don’t think I’ve ever worked with anyone who hasn’t experienced trauma to some degree or another. It effects people in different ways.

Even co-workers. We have our own too.”  “It’s something we should always keep in our mind and we don’t know how that impacted the generation of family members that we’re seeing now.”

From the perspective of educators, particularly at the high school level, participants anticipated two main challenges with their staff:  The training uses elementary school examples, which might not appear relevant to the high school context (i.e., “That doesn’t apply to me.”).  There is a tendency for some people to look at teenage behaviours as “hormonal” (i.e., not related to trauma).

52

Participants suggested connecting childhood trauma to its effects on teens.

Participants also recognized that further training could fill gaps in this program, and/or focus on actionable steps to implement content to practice.

Here is a related comment from a participant:  “As a foundational course, it’s well-designed [and] there’s lots to apply to any population. If you want more specifics, you’re looking at another course or a facilitator to take that material and [answer the question] ‘What does it mean for an indigenous population?’ or ‘What does it mean in the classroom?’ There’s lots to do that with.”

8. Application of content to practice Participants saw the training as a tool to develop awareness and enable conversations about trauma-awareness in their organizations. When asked what they would like to apply to practice, the following themes emerged from the completion surveys (n=342):  Being mindful of trauma that clients have experienced o Being trauma-sensitive and trauma-informed o Using a trauma-informed lens o Keeping in mind “what happened to you?” instead of “what’s wrong with you?”  Practicing self-care  Awareness of vicarious trauma among peers  Empowering victims/providing choices for victims to empower them  Advocating for trauma-informed practice at their agency  Improved approach to children and their families  Better communication with colleagues, children and their families o Using trauma-sensitive language  All material presented

53

At three-month follow-up, 77% of participants reported having had opportunities to use knowledge from the course to inform their practice (n=39). These included:  keeping trauma in mind daily  being trauma-sensitive when speaking to their colleagues  when working with clients, understanding clients’ behavior as result of trauma  working to build more awareness in their workplace, particularly schools.

Some participants engaged in workplace discussions about their physical environments (e.g., new office), and took steps to remove barriers and set up infrastructures in trauma-informed ways. For example, they ensured that the magazines in the waiting area were trauma-responsive. Here are two related comments:  “The course has opened my eyes to the prevalence of trauma and its effects on society at large. Furthermore, this course has helped to better inform my practice when relating to and working with clients.”  “I find that there is more discussion around trauma, what might have happened to a person to make them act the way they do/did…”

The remaining 23% of survey respondents reported not having had opportunities to use knowledge from the course in practice because of their role (e.g., student, working indirectly with clients) (n=39).

Overall, participants felt the training had not led to widespread organizational changes that were not already occurring. However, individual, colleague-to-colleague conversations occurred as a result of the training.

54

9. Challenges and barriers to practice implementation Participants identified several key challenges and barriers to implementing content to practice (in no particular order):  Time for paradigm shifts and related changes to take place at an organizational and systematic level  Knowledge and confidence with concepts, and knowledge consistency among staff  High caseloads and workloads  Building capacity: o Within and across organizations, not enough people (e.g., staff level, management, supervisors) are trained in traumaawareness, which affects their ability to work with others who share clients o The concepts of being trauma-aware and trauma-informed might have different meanings for different people, and might therefore require clarity and agreement in the workplace o Biases and attitudes toward changing practice  Lack of agency-to-agency collaboration, often due to legalities, privacy concerns and lack of communication (see following section “Collaboration across sectors”)  Lack of organizational support: o Not enough buy-in from everyone, including management/leadership and staff, within and across organizations o Lack of supporting processes or policies that support being trauma-aware and working with others  Clients: o Lack awareness of the effects of trauma o Clients’ culture/traditions that influence understanding of, and approach to trauma o Language barriers between client and service provider(s) 55

 A need for reflection, discussion and actions (detailed in proceeding section)  How to move forward post-training actions (detailed in proceeding section) Reflection, discussion and actions

Many participants expressed the need for opportunities to discuss the material. They wanted opportunities to engage with others in the training program (e.g., a cohort, their colleagues, or a professional to follow-up with).

Several participants also suggested facilitator-led and/or self-guided discussion questions for engaging and reflecting on the content. Here are two related comments:  “I hope we’ll be able to use it for initial conversation and follow-up with more conversation.”  “If there were some guided questions at the end for reflection that would be helpful for framing for longer conversations that will inevitably bubble out.”

Participants suggested face-to-face contact. For example, school counsellors could facilitate the material for teachers, to answer their questions and help develop actions they could implement.

Another suggestion involved teams taking the training together, pausing after each module to discuss content, and developing an action plan prior to proceeding to the next module.

Participants also identified the need for face-to-face contact with people who have lived experience as newcomers to Canada or who have experienced trauma.

56

How to move forward

To make the content more useful, participants wanted summaries of each module, with key take-away points. Participants suggested using digital flashcards or another printable resource, which would facilitate reviewing the main concepts and applying them to practice. Participants posed several questions regarding moving forward. For example:  “How do we integrate this into our organizations to change our practice or change something in your organizations? These are our questions now. I’d like to see how to build in organizational pieces.”  “Depending on where you are in your career, you’re going to take or leave things from the content [because] some of it’s going to resonate with you and some many not. That’s where we have to look at who our audience is and who are we rolling it out to. For some people, who may need time to integrate this, how do we follow-up with them at a later date?

In addition to end-of-module summaries, participants also suggested:    

facilitator guides a PowerPoint workbook for staff to take notes follow-up resources for staff advanced trainings for further training.

Related to moving forward, some suggestions focused on sharing the training (or parts of it) with clients. Participants felt this could allow their clients to be on the same page with them regarding trauma. However, many participants also raised points of consideration. For example:  For those opening the training to their volunteers, there was a fear that volunteers/practicum students may take on a role that is out of scope (e.g., role of a counsellor, without proper facilitation and

57



 



training). Participants must keep in mind the boundaries of their roles.

The material could be overwhelming for clients and families, so there is a need for integration with other modalities, tools and programs that are already available to them.

Language options should also be available for variety of clients.

If shared broadly, the material needs to be facilitated and prescriptive in how to use this, as it might re-traumatize clients and/or their families.

Related comment: “I wonder if there are parts we use with clients, like serve-and-return and the toxic stress video. [It would be] very easy for anyone to follow. [It] makes a lot of sense…”

Finally, participants expressed interest in learning about the results of the pilot project and further involvement with next steps.

10. Collaboration across sectors Participants saw value in working collaboratively across sectors to address issues of childhood trauma. For some, the training confirmed this work.

Participants highlighted that child advocacy centres already model crosssectoral collaboration well, including how people can work together with a common language. Participants said this model forces them to ask, “Who are we missing from the table?” However, participants also identified several challenges to achieving an integrated and collaborative approach (in no particular order):  Capacity and resources: o Time: Participants identified not having time was a challenge, as well as not receiving protected time. They wanted time to participate in discussions related to content, follow-up discussions, further training, and with partners.

58

o Availability of training: Training opportunities were limited and/or had costs associated with them. o Related comment from a focus group participant: “The age-old [issues of] time, resources…everybody’s doing more with less and it’s very hard to be trauma-informed when staff is beyond stress.

Hard to walk the walk.”  Logistical challenges: Organizations are not typically co-located and are geographically dispersed. This presents challenges, such as scheduling meetings, maintaining relationships, and privacy concerns between agencies.  Differences of priorities: Cross-collaboration requires everyone to be coordinated and in agreement with the process, but this is not always the case. Currently, collaboration relies heavily on relationships between individuals, and turnover affects this process. A systems approach is limited. o Related comment: “People see things from different lenses and things are based on relational aspects…you may find that you have a very good relationship with someone in another system but they move and then you find yourself retraining all over again, so how do you move from the relational aspect of things to a systematic response?”  Lack of awareness about trauma: Limited recognition of the reach and effects of childhood trauma affects the ability to respond to it in a collaborated, integrative and systematic way. o Trauma is not a mainstream issue: This reinforces ignorance among people from outside of traditional, trauma-related circles of care. o Related comments:  “… [B]ut there’s also ignorance. People are just not aware of how widespread this is…People don’t believe there’s such a thing, it can’t possibly have these impacts.”

59

 “I think the other issue is that most of this is not mentioned out there unless you’re in the area of mental health nursing, social work…working with people on an everyday basis who encounter these things, but these concepts never come up in mainstream education, [they’re] pushed under the carpet and they’re ignored.

That’s the challenge of making this a mainstream issue where people have to realize that the trauma that’s maybe out there is pretty widespread…”  Attitudes: When approaching a challenging situation (e.g., teachers with disruptive or defiant students), staff and organizations require a paradigm shift to change their lens. Some feel this is out of their scope of practice. This indicates the need for organizational support in applying trauma-aware concepts to their practice. o Resistance to change adversely affects efforts for integrated practice.  Compassion fatigue: Staff from all organizations shared that they are spread thinly, which challenges how much they can do to change their current practice. o There is a need to support staff in efforts to build a more integrated system, which cannot be a burden to staff. Training and other efforts for collaboration could be integrated into current practices.

Participants valued the training as a support that could help them to start building understanding and awareness of trauma. They acknowledged that the training cannot be stand-alone. Follow-up activities and direct engagement activities are necessary to facilitate the paradigm shift to being trauma aware. Participants also acknowledged that building awareness takes time. For example: “When you’re shifting perspective, [you] can’t expect it to happen overnight. It happens in discussion and in reflection.”

60

This was particularly evident with those who work with Aboriginal Peoples and on reserves, and with recommendations related to the TRC. The challenges that these groups face is rooted in a history of distrust and betrayal. For example: “We have some parishes on the reserves and it’s quite difficult for on-the-ground groups…when we speak of what’s happening on the reserves, the lack of trust, the betrayals, right now we’re far from even providing counselling. Even to open the box with the trauma stuff is going to be hard, but if [there are] willing participants, it could be a very useful tool for facilitating conversations.”

No participants reported starting new collaborations as a result of taking the training. Some organizations shared how the training highlighted working across sectors. For example, the training emphasized efforts to prevent re-traumatizing people. The emphasis on collaborative work validated the work that organizations had already started and encouraged them to continue.

Participants also reported that the training highlighted the need to apply a systems approach starting with training more people to be trauma-aware.

Here are two related comments:  “I can see a lot of potential for collaboration but I don’t think we’re there yet.”  “…I think there’s a growing recognition that one organization can’t solve complex problems [which requires] time and opportunities and developing those relationships.”

To help with encouraging collaboration, participants suggested using collective impact models or Calgary-based conferences for traumaawareness. They also suggested making the training available to more 61

professionals, which could help ensure that anyone who interacts with trauma would be on the same page, and reduce confusion for clients.

Participants also expressed the need to consider the marketing and promotion of the training. For example, a previous training (Brain Story) was available for many years before some participants found out about it.

Participants recommended a thoughtful marketing and promotions strategy to ensure the right people receive the training in a timely manner.

Finally, participants identified that systematic change begins within the education system, at the undergraduate level, where students learn about trauma and its effects in early professional programs.

11. Organizational readiness for trauma-informed practice As representatives of their organizations, participants provided insights on the readiness for trauma-informed practices. Findings from preorganizational assessment data is provided below. Please note: The postorganizational assessment data encountered low response rate, with just nine respondents and low representation via sector (n < 5 for each sector).

Thus, the quantitative findings have been omitted for comparison.

For the following data, “other” includes the faith sector, human services, immigrant-serving agency and victim services. Representation from “other” community organizations and health was low (n<5) and therefore, values appear inflated.

To understand an organization’s level of trauma-awareness (trauma-aware, trauma-sensitive, trauma-responsive, trauma-informed), participants rated their organization on statements on a Likert Scale. Among the three sectors 62

with greater representation (Education, Justice, Social Services), there was a general downward trend. More organizations described themselves as trauma-aware than trauma-informed (see fig. 19).

Figure 19: Level of trauma-awareness by sector

Participants rated their agreement to statements related to receiving training in the past year on a variety of trauma-related topics: ď&#x201A;ˇ Social service organizations received training on almost all topics listed, except trauma triggers/reminders and their effects on childhood behaviour. ď&#x201A;ˇ Few justice sector organizations received training on trauma-related topics. ď&#x201A;ˇ Less than 50% of any sector received training on cultural differences related to how children and family understand and respond to trauma, and how trauma affects childhood development (see fig. 20).

Figure 20: Sectors that have received training in trauma-related topic areas in the past year

63

64

Participants rated how many of their organization’s staff members were described by statements related to their knowledge with trauma and its impacts on children:  Among the three better-represented sectors, more social service staff rated that “most” or “all” staff were knowledgeable about trauma.  With respect to vicarious trauma, staff at all levels do not receive training on vicarious trauma, nor is it addressed in meetings (see fig.

21).

Figure 21: Sectors describing that “most” or “all” staff at their organization have knowledge about trauma topic

65

66

Participants rated their agreement to statements about their organization’s ability to collaborate with other organizations:  Generally, there is encouragement for direct service providers in each sector to regularly contact other service providers.  Fewer education and justice sector organizations agreed that service programs work together to provide integrated care.  Many workers do not receive cross-trainings with other child-serving systems.  Education sector staff had low agreement when asked about joint meetings with other providers and if there was a system of communication in place with other agencies working with the child (see fig. 22).

Figure 22: Sector agreement to statements regarding collaboration across sectors

67

68

Participants rated their agreement to statements related to policies and procedures at their organization:  Less than 50% of the better-represented sectors (notably 0% of the justice sector) had written policies committing to trauma-informed practices, or formal systems for reviewing whether staff use traumainformed practices.  Less than a third of these same sectors reported having structures in place to support consistent trauma-informed responses.  The majority of respondents from the justice and social services sectors reported that strategies to manage personal and professional stress is included in supervision. Only 40% of education sector staff reported this.  Less than 50% of education and justice sector organizations agreed that understanding the effects of trauma is incorporated into daily decision-making.  Less than 50% of these organizations reported that timely traumainformed assessment is available and accessible (see fig. 23).

Figure 23: Sector agreement to statements related to trauma-informed practice

69

70

Participants were asked if there had been any changes to policies or processes to be more trauma-aware in the three months following the pilot:  Five out of eight participants reported that there had not been any changes and elaborated that nothing formal has occurred in the past three months.  Three participants reported changes such as training on trauma and child development, formation of a working group focused on vicarious trauma and organizational health, and updating organizational policy documents.  A participant from the education sector felt the organizational assessment was too difficult to complete given the uniqueness of each school, and that from a larger system perspective, more viewpoints may be necessary.

Participants were asked to provide further insights on their organization’s readiness to implement turama-informed principles. Insights from the preorganization assessment are organized by the strengths of the organization in their readiness to implement trauma-informed principles (n=48), and the barriers to implementing trauma-informed principles (n=38).

12. Organizational Perspective Willingness to learn

Across sectors, many participants expressed “commitment” as a strength to implementing trauma-informed principles. Commitment was defined by participants as:    

willingness to learn openness to collaborate with partner agencies availability of professional development time recognition that building capacity in this area will support their respective clients.

71

Here are three related comments:  “We are open to the idea that we need to increase our capacity around [trauma-informed principle].”  “The organization is forward thinking and working to bring in this training, we are slow to get started, but very optimistic.”  “Everyone is aware, excited and eager to be more informed and implement trauma informed practice.”

Integration and collaboration

Across sectors, participants referred to integration between staff, leadership and staff, as well as between partner organizations as their strength. One participant noted that information-sharing between partner organizations prevents re-traumatization for clients, as they do not have to share traumatic experiences more than once. Another participant from the education sector noted that they have “a multidisciplinary team consisting of an intercultural wellness team, psychologists, and outside agencies.” From the justice sector, one participant noted that they have an “integrated model of practice and awareness.”

Knowledge and previous training

Some participants from the education and social service sectors indicated that staff had been trained in varying capacity in trauma-informed principles. For example:  “All of our mental health service providers and many of our teachers and administrators have been trained in the Attachment, Regulation and Competency (ARC) model of trauma informed practice.”  “We are beginning to incorporate trauma informed questioning into intake and assessment and through the stages of intervention. We are aware of how trauma impacts brain development and try to educate 72

our service providers on the same. We are fully supported to put in services and supports for anyone who may benefit from traumainformed approaches.”

Barriers

Across sectors, staff noted that limited time was a challenge to implementing trauma-informed approaches, specifically noting high caseloads, competing priorities, and volume of staff as barriers. Within the education and social service sectors, participants expressed a need to address leadership organizational constraints, including capacity of management to take on implementation and buy-in from partner organizations to adopt trauma-informed approaches.

Here are some related comments from participants:  “Providing professional development on trauma-informed practice can be challenging to provide to every teacher and administrator.”  “Not enough time, not enough support, too many kids.”  “Finding time to train and educate front line responders.”  “Organization has multiple priorities and cannot focus on a few.”  “The organizational structure- there is not one specific position that has the capacity to exclusively manage the changes that need to happen.”

Summary of findings As a reminder, these were the evaluation objectives: 1. Knowledge, awareness, confidence and commitment:  Determine the value and effectiveness of curriculum content and delivery methods for helping front-line services providers improve their practice.

73

 Increase knowledge and awareness of child abuse, trauma, impacts on the victim, family and community.  Increase confidence in addressing issues of child abuse and trauma (via service provision or referral).

2. Relevance to discipline and clients:  Determine the extent to which the curricula reflect the diverse clientele that each discipline services including the lived experiences of children, youth and adults of First Nations, immigrant groups and disenfranchised/marginalized groups.

3. Cross-collaboration strategies:  Increase recognition and impact of the reach of child abuse to multiple sectors and value of working collaboratively across sectors.  Determine the extent to which an integrated and collaborative approach is achieved when working within the target population.

4. Scaling and future development:  Determine if the curriculum was implemented with fidelity to the design.  Determine feasibility of scaling practice changes.

This section summarizes finding for each objective: Determine the value and effectiveness of curriculum content and delivery methods for helping front-line service providers improve their practice

Participants saw the course as being valuable and foundational in providing multiple sectors with a common language about trauma and its effects.

Regarding course improvements, participants suggested nothing should be removed to shorten it.

Participants’ ratings of the effectiveness of the curriculum show that the course is well-designed. The use of case studies and personal stories was 74

shown to be impactful to learning. Participants feedback on challenges (related to accessing trainings due to costs and travel time) are addressed through a free online training such as this.

Participants’ feedback on the need for direct, in-person engagement activities (e.g., discussion, facilitation) speak to the need for more organizational support in applying concepts to their practice.

Despite these challenges, participants reported that since taking the course they are mindful of trauma (among clients and colleagues ) and working to build more awareness in their workplace.

Increase knowledge and awareness of child abuse, trauma, impacts on the victim, family and community

Participants demonstrated high knowledge and awareness on most of the course topics. Given that many commented on the course being a refresher for them, it is unsurprising they came in with high knowledge.

Although the sample size was small at follow-up, the decreases in knowledge indicate the need for further review of course material. This aligns with feedback from participants around the needs for:  accessing the course again  material to help them review the course  in-person facilitated discussion.

There were some gaps in their knowledge, perhaps for new concepts. For example, participants need clarification about what trauma means in the context of child advocacy centres, and that post-traumatic stress disorder (PTSD) does not apply to children (covered in module 1). This may be because

75

participants see adult clients with PTSD, or interpreted the question broadly (i.e., as being unspecific to children).

Intergenerational trauma is another area in which participants had gaps, and need further clarification on its effects (module 3). Participants seemed aware of the importance and relevance of this topic, but also suggested further training in this area, indicating their desire for further support.

Participants also need clarification on the difference between being traumaaware, trauma-sensitive, trauma-responsive and trauma-informed, as there was variability in correctly identifying these statements (module 4). Finally, participants may need further clarification on resilience (module 5).

Increase confidence in addressing issues of child abuse and trauma

The proportion of participants who reported confidence in addressing issues of child abuse and trauma increased at completion to over 90%, but fell to just under 75% at follow-up. This indicates the course positively affected their confidence, but as time passed, they needed more support to maintain this. Feedback from participants indicated the need for opportunities post-course to:    

discuss reflect put content into action review material to maintain their knowledge and confidence.

Determine the extent to which the curricula reflect the diverse clientele that each discipline services

As a foundational course, the training seemed to reflect the diverse clientele that most disciplines service. Participants provided feedback on ways to be 76

more representative and to follow-up with future training. Many of these topics focus on communities and cultures that bring their own trauma, post-migration trauma, inter-generational trauma and trauma experienced

by Indigenous people.

Comments from educators focused on potential challenges in applying concepts to high school settings when the course explains them as being experienced by children. This may not necessarily need to be addressed indepth in the course. Rather, it could be addressed through an accompanying facilitation guide, or specific training focused on adolescents with trauma.

Increase recognition of the reach and impact of child abuse to multiple sectors and the value of working collaboratively across sectors

Participants had very high recognition of the reach and impact of child abuse to multiple sectors, as well as the importance of direct care providers:  working with other service providers for the care of the child  integrating services  cross-training with those working in child-serving systems.

Participants recognized the model used by child advocacy centres and its benefits to collaborative work. However, participants were much more pragmatic about the challenges and barriers to collaborating across sectors.

They highlighted the need for organizational support to address many of these, including capacity and resource challenges, and attitudes of staff who feel that being trauma-informed is out of their scope.

Interestingly, despite the participants’ recognition of the reach and impact of childhood trauma to multiple sectors, they highlighted that not everyone has this recognition. This indicates the need for training more people across 77

sectors together. Participants also emphasized the need for systematic support in moving toward a paradigm shift (i.e., moving towards a system that is trauma-informed and works collaboratively across sectors).

Determine the extent to which an integrated and collaborative approach is achieved when working within the target population

Although it may be too early to see an integrated and collaborative approach achieved as reported through qualitative findings, individual and organizational level findings support the potential for integration. All participants supported the idea that the effects of complex trauma extend to multiple sectors, and that cross-training, integrated service and regular contact between service providers should exist. At the organizational level, the extent to which collaboration is occurring varies across sectors and is influenced by the organization and system. Participants identified two aspects that are generally missing and that may help in facilitating partnerships: a lack of cross-trainings of workers across sectors, and a communication system across agencies.

Determine if curriculum was implemented with fidelity to the design

The curriculum, overall, was implemented with fidelity to design, allowing 412 participants to be certified on â&#x20AC;&#x153;Being trauma-aware.â&#x20AC;? The curriculum was intended for a wide range of professionals working with children who may be affected by trauma, or working in environments that support children.

According to the demographic information, a wide range of sectors and professions were represented by the pilot program, and results show that findings are generally common across sectors.

78

Feedback from participants indicates ways to enhance the design to ensure uptake. In two instances, participants received the online training with a facilitator, as it became clear early on that these groups would not complete the training without the modified model. This is an important learning to keep in mind when scaling the program. Participants may find it challenging to complete the course alone, or may not try to at all. They may benefit from taking the online course together. This aligns with findings that participants desire direct engagement activities in the form of facilitated discussion. Moving forward, utilizing the existing facilitation guide may enhance uptake and consistent application to practice.

On the technical front, the participantsâ&#x20AC;&#x2122; desire to forward/skip raises some concerns about fidelity. As a foundational course aimed at providing all participants with a common language, the option to skip parts may dilute this outcome. Their suggestions on rewinding videos and having closed captioning are sound and aim to enhance learning, as are those to improve the technical glitches (e.g., re-starting a module because of freezing).

However, skipping ahead of content may adversely affect learning.

It is true that many felt that the course was a refresher, but the needs of some should be balanced with the needs of the whole. If time is a concern, realistic expectations should be built in, and organizational support to complete the training should be encouraged.

Determine the feasibility of scaling practice changes

Given the short time-frame for this project, it is unsurprising that most participants and organizations have not reported change in policies and procedures related to trauma-informed practice.

79

Organizations are at or approaching the level of being trauma-aware, but they are far from being trauma-informed. This may be affected by the variety of training they already deliver. However, all sectors lack training on culture and trauma, trauma triggers, the impact of trauma on developmental stages, and vicarious trauma.

Keeping this in mind, it is important that scaling reach as many people across sectors to provide a consistent language and knowledge base. This may affect the development of policies, procedures and other structures that promote trauma-informed practices.

80

Limitations The low response rate of the three-month follow-up survey affects the ability to generalize findings to the greater pilot population. Furthermore, the postorganizational assessment surveys were also implemented at the threemonth mark and experienced a low response rate and low representation by sector.

As the literature review indicated, follow-up is a common challenge. Despite this, it would be interesting to explore knowledge maintenance, application to practice and challenges/barriers to implementation at follow-up.

A three-month follow-up was selected to fall within project requirement timelines. Given the results from the evaluation (the focus group in particular) three months is insufficient for significant organizational or systematic changes. Investment in longitudinal follow-up with organizations and trained participants may provide greater insight into applications to practice and opportunities for cross-collaboration.

81

Recommendations The following recommendations are offered to improve and scale the course. These are framed by the three implementation drivers necessary for the success of an intervention: competency, leadership and organization (NIRN, 2014).

Competency Drivers

To develop competency, training is a key way to ignite behavioural change.

By supporting staff training through course delivery, design, and content, future trainings and evaluation are elements of implementing this training that can help facilitate change.

Course delivery 1. Based on challenges accessing training due to cost and distance, offering this training free to organizations across a wide range of sectors, including those in the pilot phase.

2. Offer the training online, which allows participants to complete it at their own pace and reaches a wider range of people (e.g., shift workers).

3. Allow access to course after upon completion to review material as some may prefer accessing the course online.

4. Consider developing summaries of modules with key take-away points so that participants have access to the information without having to log in if they prefer that.

5. To enhance learning, share facilitation guides with organizations and ensure that the guides provide discussion questions so that organizations can take the course together, or reflect/discuss topics together.

6. Provide guidance on how the course could be shared with clients and families so it is not overwhelming or re-traumatizing, and service 82

providers can answer questions or direct them to the necessary resource.

Course design and content 7. Maintain variety of learning elements (various media, quizzes, personal stories, examples, case studies) as these helped in engaging participants.

8. Consider developing exercises to be more challenging.

9. Maintain the simplicity, clarity and informativeness of content, as well as build-up from Modules 1 to 5.

10. Maintain flow of content, but consider improving transitions between conversations with characters to videos of real-life stories.

11. When collecting written information, provide transparency regarding confidentiality. This could include redeveloping or reframing these activities as personal journals that are exportable (if such function can be developed).

12. While the Sheldon Kennedy videos were deemed impactful, their inclusion of some offensive language upset some viewers. Consider warnings, removal or replacement.

13. Set expectations regarding the time for completion, as it exceeded the advertised 90 minutes.

14. Provide timing for each module and a progress bar, so as learners progress they are aware of how much time is expected and how much time is left. Communicate ahead of time that learners will not be able to pick up where they left off, if they must stop in the middle of a module.

15. Enable options to rewind parts of modules to review concepts that were missed or hard to follow given the diversity of learning styles.

With respect to skipping sections, it is recommended that caution be taken with such an option as learners may skip ahead and hamper their learning.

83

16. Provide closed captioning and/or connect to Google Translate to view content in another language to aid diversity of learners, potentially those whose primary language may not be English or those who may have accessibility needs.

Future trainings 17. Consider developing courses that extend from this initial training through advanced training opportunities, particularly intergenerational trauma.

18. Consider offering trainings on the following trauma-related topics: a. Loss and grief, loss of culture b. Culture/community specific trauma c. Impact of post-migration on previous traumas among newcomers d. Trauma experienced by Indigenous people and the integration with the Truth and Reconciliation Commission report e. Impact of trauma for LGBTQ children/youth 19. Ensure trainings do not duplicate efforts and are integrated into current professional development plans, given workload issues and compassion fatigue.

Evaluation 20. Integrate pre- and post-module surveys and completion surveys for future evaluation purposes, and to provide participants with an opportunity to check-in with their learning.

21. Consider providing participants with immediate feedback to be used as a learning tool and to prompt review of concepts not easily understood.

22. Provide a summary of the project, including high-level evaluation findings, to participants who provided their data. Engage participants in future developments, with next steps and updates.

84

23. If follow-up evaluation will be included in future iterations, consider engaging participants through incentives or direct engagement to increase response rate in follow-up activities.

Leadership Drivers

Competent leadership is required to facilitate change. As a key implementation driver, leaders must be able to respond to issues in certain environments, and also in environments that are more complex and uncertain.

Broad engagement 24. Train staff at all levels, including supervisors, management, administrative staff, within and across organizations to ensure a common and consistent language applied.

Time 25. Learners require protected time to complete trainings such as these.

SKCAC should promote the need for protected time when connecting with organizations.

Awareness of trauma 26. Train leadership on course content as they are positions to facilitate change processes. Given that trauma is not a mainstream issue, having leadership across sectors on the same page could propel action. A topdriven approach may tackle issues related to staff resistance to change.

27. Provide opportunity (e.g., through administrative and team meetings) to discuss trauma-informed practices.

85

Systems integration 28. Post-secondary educational programs should consider opportunities to embed trauma-aware principles in their curriculum and facilitate cross-sectoral integration as early as possible. By having a shared understanding of trauma-aware principles guiding their work, there are increased opportunities to explore cross-sectoral integration and collaboration.

29. Provide opportunities for staff across sectors to train together in future training courses, with active opportunities for interprofessional and cross-sectoral collaboration.

30. Consider building awareness in the community, perhaps through a collaboratively-developed awareness campaign on trauma and its effects.

Organization Drivers

To build capacity within and across organizations, staff need the support of their organization in reducing barriers to implementation, championing systems interventions and supporting data-driven decision-making.

Buy-in 31. This set of recommendations require that leadership and staff agree on the importance of being trauma-aware in their practice. Direct engagement activities should be conducted to develop buy-in.

32. In scaling the course, SKCAC should use champions from the pilot phase representing a variety of organizations and sectors, which could promote buy-in from all levels. This should include leadershiplevel champions as well as staff-level champions to reduce resistance to change.

33. Use champions from the pilot phase to help in developing necessary material (as outlined in course delivery recommendations). This could 86

take the form of a train-the-trainer model to enhance the application of the content across a variety of contexts and to increase participant engagement and recruitment across sectors and levels.

Marketing 34. When scaling the program, ensure a multi-faceted marketing strategy is in place to reduce the gap between the availability of training and awareness that such training exists.

Supporting policies and procedures 35. Trauma-informed policies and procedures need to be developed and in place to support staff in applying course content to practice. SKCAC should follow-up with representatives from organizations three to nine months after project completion (i.e., at the six month or one year mark) to understand changes (if any, as well as barriers) and provide consultation policy development.

Evaluation 36. Future scaling should emphasize longitudinal evaluation (e.g., six months to a year) to determine if and how organizations have changed. This may require partnership and investment of participating organizations in these shared evaluation efforts.

Conclusion The value of â&#x20AC;&#x153;Being trauma-awareâ&#x20AC;? to participants cannot be understated.

The course was very well-received as a comprehensive, foundational tool to provide learners across sectors with a common language. With improvements in technical elements, clarity in activities and language, the course is well-positioned as an educational tool. As a starting point to change, organizational responses and systematic structures must be in place to support cross-sectoral collaboration and consistent practice change.

87

Acknowledgements The evaluation team would like to thank the following people who made this project possible:  The team at Respect Group Inc., led by Garth Stonier, with whom data collection was possible.  SKCAC Evaluation Coordinator Kristie McCann and Administrative Assistant Kate Bowers, who facilitated focus groups in Calgary.  The Performance Group for their early input on the evaluation framework.  The tremendous dedication of Detective Brenda Neis, Project Coordinator to evaluation and also the project at large.

Thank you!

88

References Fixsen, D., Naoom, S., Blase, K., Friedman, R. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Louis de la Parte Florida Mental Health Institute Publication #231: Tampa, Florida.

Kirkpatrick, D.L., Kirkpatrick, J.D. & Kirkaptrick, W.K. (2016). The New World Kirkpatrick Model. Retrieved from: http://www.kirkpatrickpartners.com/OurPhilosophy/TheNewWorldKirkpatri ckModel/tabid/303/Default.aspx.

National Implementation Science Network (NIRN) (2015). Implementation Drivers: Assessing Best Practices. Retrieved from: http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/ NIRN-ImplementationDriversAssessingBestPractices.pdf.

89

Appendix A: Knowledge, awareness, confidence Statement

Baseline

3-month follow-up

Z statistic, p-value

Z statistic, p-value

I have good understanding of the contributors to child trauma.

-11.892, p < 0.001

-2.324, p = 0.020

I have good understanding of trauma.

-12.222, p < 0.001

-2.324, p = 0.020

I have a clear understanding of what trauma-aware means in my professional role.

-12.112, p < 0.001

-2.398, p = 0.016

I feel open to trying a trauma-aware approach with children and families.

-5.234, p < 0.001

-2.000, p = 0.046

I have good understanding of the impacts of trauma on children and youth.

-11.668, p < 0.001

-3.500, p < 0.001

90

I have good -11.966, p < 0.001 understanding of the impacts of trauma on the community.

-1.500, p = 0.134

I am aware of the importance of using a trauma lens.

-11.783, p < 0.001

-1.941, p = 0.052

I am confident in -12.342, p < 0.001 addressing issues of child maltreatment and trauma.

-2.400, p = 0.016

I have a clear understanding of what trauma informed practice means in my professional role.

-2.308, p = 0.021

-13.300, p < 0.001

91

Appendix B: Module-based measures Module-based knowledge

Baseline

3-month follow-up

T statistic, p-value

T statistic, p-value

N/A

N/A

1.374, p = 0.170

-1.000, p = 0.324

1.607, p = 0.109

0.000, p = 1.000

-1.96, p = 0.050

-1.434, p = 0.160

The brainâ&#x20AC;&#x2122;s ability to change in response to the environment is calledâ&#x20AC;Ś 2.291, p = 0.023

0.813, p = 0.422

Definition of trauma in child advocacy centres Complex trauma refers to an individual being exposed to multiple and extensive traumatic events.

The part of the brain that is responsible for our ability to make plans, prioritize tasks and problem solve is the prefrontal cortex.

Positive stress and toxic stress impact the brain architecture

92

Adverse childhood experiences

-1.570, p = 0.117

0.572, p = 0.571

Childhood maltreatment is a risk factor for early onset of substance use later in life.

1.416, p = 0.158

N/A

There are greater rates of alcohol, drug use, suicide, attempted suicide among adolescents who experience childhood maltreatment than those who do not.

0.816, p = 0.415

N/A

Domains of impairment

N/A

N/A

Potential impacts of adverse childhood events in adolescence and adulthood 2.121, p = 0.35

N/A

Intergenerational trauma -23.043, p<0.001

8.152, p<0.001

Trauma-informed

12.699, p = 0.000

-1.639, p = 0.110

Trauma-aware

6.789, p = 0.00

-1.641, p = 0.109

93

1.549, p = 0.122 Trauma-responsive

0.572, p = 0.571

Trauma-sensitive

-0.503, p = 0.615

1.00, p = 0.324

The feeling that past trauma is reoccurring or if current experience of someone trying to help is as unsafe as past trauma refers to retraumatization.

0.572, p = 0.571

1.00, p = 0.324

When a child disclosed abuse to you, what steps can be taken to make children feel physically and emotionally safe?

N/A

N/A

Resilience

N/A

N/A

The single most common factor to foster resilience is at least one stable and committed relationship with a supportive adult.

3.405, p = 0.001

N/A

1.343, p = 0.180

1.00, p = 0.324

An individualized selfcare plan is different for

94

everyone and includes things that we can do take care of ourselves in efforts to prevent the effects of vicarious trauma.

95

Appendix C: Beliefs Beliefs

Baseline

3-month follow-up

Z statistic, p-value

Z statistic, p-value

Workers should receive cross-training with other child-serving systems (e.g., courts, mental health, schools, etc.).

-3.836, p < 0.001

0.000 p = 1.000

Service programs should work together to provide integrated care for each child and family.

-2.676, p = 0.007

-0.513, p = 0.608

Direct care providers should be encouraged to have regular contact (with proper consent) with other service providers working with the same child/family.

-5.406, p < 0.001

-0.980, p = 0.327

The impact of complex trauma extends to multiple sectors such as education and health.

-1.544, p = 0.123

-1.040, p = 0.298

96