Special Education Research, Policy & Practice - 2021 Edition by Hofstra University

SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE (2021 Edition)

2021 Edition Volume 5 Table of Contents

Page

Editorial Board of Reviewers

Promotion of Positive Partnerships with Culturally and Linguistically Diverse Parents of General Education and Students with Special Needs Nicole Manly and Darlene Desbrow

Characteristics of Rural STEM Clubs and Implications for Students with Disabilities Karin M. Fisher, Peggy Shannon-Baker, Kelly Brooksher, and Kania Greer

Increasing Teacher Education Candidate Collaboration Knowledge and Skill Through Approximation of Practice Cathy L. Eschete, Renée E. Lastrapes, and Paul Mooney

Preparing Preservice Special Educators for Culturally and Linguistically Diverse Classrooms: A Systematic Review of Teacher Preparation Programs Krystal Lewis-Pratl, Yojanna Cuenca-Carlino, and April Mustian

Self-Monitoring Among Secondary and Post-Secondary Students with Intellectual and Developmental Disabilities: A Systematic Review Melvin S. Marsh and Stephanie M. Devine

Pre-Service Teachers’ Noticing of Struggling Students’ Mathematical Thinking Emily N. Rutherford and Dittika Gupta

Intellectual Development and Specific Learning Disability: The Role of Norm-Referenced Tests Edward K. Schultz, Emily Rutherford, and Dennis Cavitt

102

Training Preservice Special Educators for Professional Collaboration: What Teacher Educators Think and Do Laurie A. Sharp and Michelle Simmons

116

3D Printing as a Teaching Tool for People who are Blind and Visually Impaired Julia VanderMolen and Jennifer Fortuna

128

A Comparison of Adolescents’ Goals for One and Five Years After High School Graduation Kendra Williams-Diehm, Penny Cantwell, Kylie Lyons, Malarie Deardorff, and Patricia Lynch

141

Author Guidelines

163

Publishing Process

164

165

3 Editorial Board of Reviewers All members of the Hofstra University Special Education Department will sit on the Editorial Board for the SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE. Each of the faculty will reach out to professionals in the field whom he/she knows to start the process of building a list of peer reviewers for specific types of articles. Reviewer selection is critical to the publication process, and we will base our choice on many factors, including expertise, reputation, specific recommendations and previous experience of a reviewer. Editor George Giuliani, J.D., Psy.D., Hofstra University Hofstra University Special Education Faculty Elfreda Blue, Ph.D. Stephen Hernandez, Ed.D. Gloria Lodato Wilson, Ph.D. Mary McDonald, Ph.D., BCBA Darra Pace, Ed.D. Diane Schwartz, Ed.D. Editorial Board Mohammed Alzyoudi, Ph.D., American University in the Emirates. Dubai. UAE Faith Andreasen, Ph.D. Vance L. Austin, Ph.D., Manhattanville College Amy Balin, Ph.D., Simmons College Dana Battaglia, Ph.D., Adelphi University Brooke Blanks, Ph.D., Radford University Kathleen Boothe, Ph.D., Southeastern Oklahoma State University Nicholas Catania, Ph.D. Candidate, University of South Florida Lindsey A. Chapman, Ph.D., University of Florida Morgan Chitiyo, Ph.D., University of North Carolina at Greensboro Jonathan Chitiyo, Ph.D., University of Pittsburgh at Bradford Heidi Cornell, Ph.D., Wichita State University Lesley Craig-Unkefer, Ed.D., Middle Tennessee State University Amy Davies Lackey, Ph.D., BCBA-D Josh Del Viscovo, MS, BCSE, Northcentral University Darlene Desbrow, Ph.D., United States University Janet R. DeSimone, Ed.D., Lehman College, The City University of New York Lisa Dille, Ed.D., BCBA, Georgian Court University William Dorfman, B.A. (MA in progress), Florida International University Brandi Eley, Ph.D. Tracey Falardeau MA, MS Genesee Intermediate School District Danielle Feeney, Ph.D. Candidate, University of Nevada, Las Vegas Lisa Fleisher, Ph.D., New York University Neil O. Friesland, Ed.D., MidAmerica Nazarene University

4 Theresa Garfield, Ed.D., Texas A&M University-San Antonio Leigh Gates, Ed.D., University of North Carolina Wilmington Sean Green, Ph.D. Deborah W. Hartman, M.S., Cedar Crest College Shawnna Helf, Ph.D., Winthrop University Nicole Irish, Ed.D., University of the Cumberlands Randa G. Keeley, PhD, Texas Woman's University Hyun Uk Kim, Ph.D., Eastern Oregon University Louisa Kramer-Vida, Ed.D., Long Island University Nai-Cheng Kuo, PhD., BCBA, Augusta University Renée E. Lastrapes, Ph.D., University of Houston-Clear Lake Debra Leach, Ed.D., BCBA, Winthrop University Marla J. Lohmann, Ph.D., Colorado Christian University Mary Lombardo-Graves, Ed.D., University of Evansville Pamela E. Lowry, Ed.D., Georgian Court University Matthew D. Lucas, Ed.D., Longwood University Jay R. Lucker, Ed.D., Howard University Jennifer N. Mahdavi, Ph.D., BCBA-D, Sonoma State University Alyson Martin, Ed.D., Fairfield University Krystle E. Merry, M.S. Ed., NBCT., Ph.D. Student, University of Arkansas Marcia Montague, Ph.D., Texas A&M University Chelsea T. Morris, Ph.D., University of West Georgia Gena Nelson, Ph.D. Candidate, American Institutes for Research Lawrence Nhemachena, MSc, Universidade Catolica de Mozambique Maria B. Peterson Ahmad, Ph.D., Western Oregon University Christine Powell. Ed.D., California Lutheran University Deborah Reed, Ph.D., University of Iowa Ken Reimer, Ph.D., University of Winnipeg Dana Reinecke, PhD, BCBA-D, Capella University Dana Reinecke, Ph.D., Long Island University-C.W. Post Denise Rich-Gross, Ph.D., University of Akron Benjamin Riden, ABD - Ph.D., Penn State Mary Runo, Ph.D., Kenyatta University Emily Smith, Ed.D., Midwestern State University Carrie Semmelroth, Ed.D.., Boise State University Pamela Mary Schmidt, M.S., Freeport High School Special Education Department Edward Schultz, Ph.D., Midwestern State University Mustafa Serdar Köksal, Ph.D., Hacettepe University, Turkey Emily R. Shamash, Ed.D., Teachers College, Columbia University Christopher E. Smith, PhD, BCBA-D, Long Island University Emily Smith, Ed.D., Midwestern State University Gregory W. Smith. Ph.D., University of Southern Mississippi Emily Sobeck, Ph.D., Franciscan University Ernest Solar, Ph.D., Mount St. Mary’s University Gretchen L. Stewart , Ph.D., University of South Florida Roben Taylor, Ed.D., Western Governors University

5 Jessie Sue Thacker-King, Arkansas State University Julia VanderMolen, Ph.D., Grand Valley State University Cindy Widner, Ed.D., Carson Newman University Kathleen G. Winterman, Ed.D., Xavier University Sara B. Woolf, Ed.D., Queens College, City University of New York Perry A. Zirkel, J.D., Ph.D., Lehigh University

6 Promotion of Positive Partnerships with Culturally and Linguistically Diverse Parents of General Education and Students with Special Needs Nicole Manly, M.S. Darlene Desbrow, Ph.D. Abstract Educators and parents work together to support the student’s academic and behavioral success in the general education and special education classroom. Parents of cultural and linguistic diversity often demonstrate limited collaboration with teachers due to lack of understanding the benefits of parental involvement in their child’s education. This lack of understanding may be related to language barriers and/or cultural differences. Educators play a crucial role in creating, building, and maintaining a successful collaborative relationship with parents. This article presents information related to strategies for building successful collaborative relationships with the culturally diverse family and discusses barriers which may negatively affect successful collaboration and the student’s overall performance. Keywords: individualized education program, barriers to educational partnerships, collaboration, cultural diversity, barriers, collaborative strategies Promotion of Positive Partnerships with Culturally and Linguistically Diverse Parents of General Education and Students with Special Needs This literature review was conducted in response to the current limit of literature on the positive outcomes of overcoming barriers that prevent the culturally linguistic parents from collaborating with teachers for student success. Parents collaborating with the educator has been shown to have positive outcomes for student achievement regardless of family demographics (Grant & Ray, 2019; Staples and Diliberto, 2010). As well, parent participation in the IEP process is an integral part of the Individuals with Disabilities Education Act (IDEA). While the concept of parent-school collaboration as a positive fundamental element in general education and special education is agreed upon by both researchers and practitioners ( D’Haem & Griswold, 2017), the partnership between culturally and linguistically diverse families and the educator is too often ignored or poorly applied for social and behavioral issues and when developing individualized education programs (IEPs) despite the benefit to the student (Tran, 2014; Coots, 2007; and Ditrano & Silverstein, 2006). By adopting strategies to build rapport, provide access to information, and attend to family routines, educators can overcome several barriers and establish a climate of successful collaboration with culturally and linguistically diverse (CLD) families in the interest of student achievement (Haem & Griswold, 2017; Tran, 2014; Coots, 2007; Ditrano & Silverstein, 2006; Harry, 2008; Blue-Banning et al, 2004; Bernheimer & Keogh, 1995;). The following terms are used as defined in the study: 1. Individualized Education Program (IEP)- A written individualized program that includes the child’s present levels of performance, goals, objectives and services provided (Kauffman, Hallahan, Pullen, Badar, 2018).

7 2. 3. 4. 5.

Barriers- Educational barriers can be physical, technological, systemic, attitudinal, emotional blocks that can prevent a child from having educational success (Minor, 2016). Collaboration- The process of working together to build success (Harry, 2008). Cultural Diversity- A mix of cultures and ethnics groups within a society (Harry, 2008). Holistic- Holistic education is self-knowing and provides a connection between a person and his or her community, world and self-values (Kolb & Kolb, 2009). Collaboration and Involvement

The first step to building positive partnerships with culturally-diverse parents is to understand the meaning of the term collaboration. Researchers suggest that while educators understand the importance of collaborative partnerships, they do not necessarily understand the essential components such a relationship requires (Tran 2014; Blue-Banning et al., 2004). Fundamental tenets of collaboration used for building successful parent/teacher relationships include: (a) positive, understandable and respectful communication; (b) commitment to the child, family and goals; (c) equality in the decision-making process; (d) perception and demonstration of competence; (e) mutual trust; and (f) mutual respect (Grant & Ray, 2019; Tran, 2014; Walker et.al., 2011; Blue-Banning et al., 2004). Epstein (2018) furthers these tenets with idea that the family and various service providers have complementary expertise working towards a shared goal for the student. The lack of understanding by school personnel in how to cultivate parent partnerships is furthered by varied understandings of the term parent involvement. Staples and Diliberto (2010) define parental involvement as school-based and home-based. Home-based is the parent’s involvement with the child’s homework, projects, affirming the value of education, and enrichment activities. School-based is the parent’s involvement with conferences, events, functions, and outings. Often, the absence of traditional parent involvement is often perceived as indifference instead of being seen as a barrier that the school system needs to resolve internally (Olivos, Gallagher & Aguilar, 2010). Parental Involvement As an educator in today’s public school system, finding ways to include CLD parents is the first step to increasing involvement. You have to ask these questions:  Do schools have proper plans in place to include CLD parents and all parents in activities and events?  Is parent involvement included in the school’s mission statement?  Is the school climate welcoming to parents?  Are the proper communication systems of today (telephone message recordings of upcoming events, emails, flyers, newsletters, school web announcements) used to keep parents aware school happenings? And if so, is material available in the language of the intended families? (Grant & Ray, 2019). Parental engagement in the education of their child yields higher levels of student involvement in academics, improved attendance, higher achievements rates, and increased graduations rates (Lasky and Karge, 2011). As well, student attitudes towards education are positive and decreased drop-out rates are observed (Tran, 2014). Parent involvement can and should happen though a

8 range of methods. Epstein (2018) emphasizes the need to have various opportunities for parents to demonstrate involvement. The researchers shows that these should include opportunities through parenting that supports the child as a student; communication between families and the school; volunteering at or for the school; learning at home; participating in school-based decisions; and liaising with the community (Grant & Ray, 2019; D’Haem & Griswold, 2016; Epstein, 2018). Parent involvement should also include parents as partners, collaborators, supporters, advisors and co-decision makers (Karge & Karge, 2011). These different aspects of parent involvement are important when working with CLD families to divert from the stereotypes that paint low-income families as uninvolved (Olivos, Gallagher & Aguilar, 2010, Lightfoot, 2004). Collaboration and provision of multiple types of involvement should lead to outcomes that respect all parties, address diverse needs, and support student achievement. Despite the noted benefits to successful collaboration, there are barriers that may prevent progress. Barriers Deficit Views Misunderstanding or lack of knowledge regarding key terms such as collaboration and parental involvement is only one barrier in reaching the desired outcome of positive parent partnerships when working with CLD families. Isolation and segregation are historical features of our nation’s public school system which lead to negative imagery and deficit-based stereotypes when discussing collaboration with CLD families both on school sites and in the research today (Harry, 2008; Lightfoot, 2004). Low-income families are seen as not having much to give and being in a position of need rather than middle- and upper-class families who are seen as giving, sometimes even in excess (Lightfoot, 2004). As well, cultural groups such as Mexican immigrant families are believed to not value education therefore not involving themselves in the education process (Lasky & Karge, 2011). Many times these deficit views are based on biases and unsubstantiated bits of information (Harry, Klingner, & Hart, 2005). Furthermore, a lack of respect leading to feelings of mistrust is a prevalent throughout research of CLD families and their struggles with the public school system (Harry, 2008). Cross-Cultural Differences A school’s push for conformity with the mainstream dominant culture instead of consideration of the family’s unique composition is another barrier when building partnerships with CLD families. This occurs throughout multiple aspects of the educational system from the expectation of parental involvement to IEP meeting times to the definition of disability (Grant & Ray, 2019; Harry, 2008; Lightfoot, 2004; Minke & Anderson, 2005). Research has found that parents of diverse backgrounds may be confused by or disagree with special education labels of disability. This discontinuity can lead to friction in the partnership and cross-cultural miscommunication (Harry, 2008). Harry (2008) states that this conflict based on cultural views can also permeate into goal setting, transition planning, and caregiver roles. In a study conducted by Walker, Ice, Hoover-Dempsey & Sandler (2011) Latino parents do hold high expectations on their children regarding education, but believe it is the schools job to successfully educate the students. Many Latino parents believe it is disrespectful to interrupt the teacher’s expertise and knowledge. Walker et al (2011) notes the importance of teachers educating Latino parents on the necessity of collaboration to support their child’s success. Tradition runs deep both in families and in the

9 school system. It is important that teachers are knowledgeable of various cultural beliefs to gain a deep understanding of parent views towards the education of their child (Tran, 2014). Lack of Communication Another barrier prevalent in the research is the lack of communication between school personnel and CLD families. While the lack of knowledge regarding parental rights, educational options and decision making may be due to language barriers or cultural views regarding the authority of schools, it disempowers families and separates them from the collaboration process (Olivos et al., 2010). Harry, Klinger and Hart (2005) found that even if communication was existent between families and school, it was, in some instances, filled with jargon, disingenuous, and disparaging. As a result of negative or nonexistent communication, CLD families and their children become submissive recipients of service rather than active collaborators perpetuating the aforementioned stereotypes (Ditrano & Silverstein, 2006; Lightfoot, 2004; Olivos et al., 2010). Schools are the gatekeepers to necessary information. If school personnel do not have open, clear defined frequent communications with families and communities, then these stakeholders are left out of the decision-making processes (Epstein, 2018). Survey Study A study was conducted to determine teacher knowledge of successful collaboration with the culturally and linguistically diverse parents. The population for this study was K-8 teachers from four elementary schools and one middle school from one urban, middle class district in the Orange County region of southern California. The Pie Chart shows this districts most recent school enrollment by race and ethnicity.

10 Survey Instrument The instrument used to collect data was an emailed survey entitled Connecting with Culturally and Linguistically Diverse Parents and (Table 1). This survey was created to determine teacher understanding of ways to successfully connect and collaborate with the CLD parent. Surveys were emailed to participants with instructions which noted that teacher responses reported within survey would be anonymous. Table 1. Survey Questions ___________________________________________________________________________ How do you ensure that CLD parents clearly understand your classroom procedures and standards, rules, expectations and/or IEP goals and objectives (if IEP's pertain to you?) 1.) How do you ensure that CLD parents feel connected to his/her child in your classroom? 2.) Are there any barriers that you feel interfere with CLD parental involvement? 3.) Would you benefit from literature that provides strategies for including CLD parents into the school climate to increase interaction and involvement in their child's education? ____________________________________________________________________________ Table 2. Survey Responses ____________________________________________________________________________ 1.) How do you ensure that CLD parents clearly understand your classroom procedures and standards, rules, expectations and/or IEP goals and objectives (if IEP's pertain to you?) ● Back to School Night meeting ● Welcome Letters sent to families ● Class Websites for information and assignments ● Parent Conferences ● Report cards and Progress reports  Use of Interpreters ● I do not have any CLD parents ● Unsure 2.) How do you ensure that CLD parents feel connected to his/her child in your classroom? ● ● ● ● ● ● ● ●

Open Communication Treat them as I would treat all parents Invite parents on field trips Invite parents to school wide activities Provide weekly/monthly agendas Provide a web page for information This is the job of administrators Unsure

11 3.) Are there any barriers that you feel interfere with CLD parental involvement? ● ● ● ● ● ●

Parents do not respond to emails and phone calls Parents do not attend school wide activities Parents do not attend conferences Parents are uninvolved/do not care Language barriers None

4.) Would you benefit from literature that provides strategies for including CLD parents into the school climate to increase interaction and involvement in their child's education? ● Yes, I would benefit from literature for support (44 responses) ● I would benefit from resources, not literature (1 response) ● No (1 response) ____________________________________________________________________________ Survey Results Surveys were emailed to 84 participants. Of the 84 surveys emailed, 46 surveys were returned completed. Results of the survey yielded similar results (Table 2). Survey results are a combination of middle school and elementary school results that were repeated throughout the survey. Results of this survey show that although teachers use typical school wide strategies to involve parents and collaborate academically and socially (Diliberto, 2010), they do not articulate how to make those activities understandable to the CLD parent. Use of translators is noted for the language barrier if the parent present, but if the parent does not respond to invites, ways to involve the parent for positive collaboration is unacknowledged. As well, according to Tran (2014) the school environment should also establish a sense of community by including families in tasks such as a school wide improvement plans, curriculum development, decision making regarding important school issues. Results of the survey also show responses of “I don’t have any CLD parents” although the current pie chart of enrollment by race and ethnicity shows less than half the population is white. Finally, 44 respondents noted that they would benefit from literature that would provide information to promote positive collaboration with the CLD parents. Strategies Building Rapport Despite the barriers, it is up to individual educators to build a positive partnership with the families they serve regardless of their culture, background or status. Finding shared goals among the school and the individual family is one way to begin a relationship of collaboration (Tran, 2014; Lasky & Karge, 2011; Harry et al., 2005). While educators and families may have differing ideas and methods to promote the student’s growth, all members of the educational team have a fundamental goal of progress for that student (Olivos et al., 2010). Since a collaborative partnership includes two-way communication, it is important to carefully listen, without passing judgment to a family’s wants and needs when determining goals and subsequent interventions (Minke & Anderson, 2005; Blue-Banning et al., 2004; Bernheimer & Keogh, 1995). This shared decision-making power not only builds rapport with the family, but it

12 provides opportunity for parent involvement where there may not have been before. Continued commitment towards shared values and goals is also crucial to the maintenance of the partnership built (Blue-Banning et al., 2004). In the spirit of true collaboration, respect and trust should be shown by the school to the families served; this will continue to strengthen the partnership and rapport with CLD families (BlueBanning et al., 2004). While Blue-Banning et al. (2004) assert that “common sense and ordinary human decency” can lead to positive partnerships with parents, it may not come easily. Educators should actively work to demonstrate respect in their interactions. Harry (2008) recommends that this be a part of pre-service and in-service preparation to foster “cross-cultural competence.” Often professionals are unaware of how their communication can be seen as disrespectful. Coursework and professional development in multicultural issues and communication will aid in the creation of unbiased, respectful practices. In addition to education on cultural view and differences, school personnel need to examine and understand their own personal biases (Olivos et al., 2010; Harry, 2008). Access to Information Since open, honest communication is part of a collaborative partnership, it is the duty of not only the school but also individual educators to communicate with all families regarding education progress and decisions on frequent basis and provide informational material in the language of the student (Harry, 2008; Blue-Banning et al., 2004). This communication, whether formal or informal, should be both culturally sensitive and understandable (Olivos et al., 2010). This practice should extend to any information regarding the student and decision-making. Ditrano and Silverstein (2006) found that when families had access to information, support from others, and an understanding of their rights, they had an increase in confidence. In turn, this allows for active and productive parent involvement. Access to information also empowers the family to be the student’s life-long advocate and promote positive educational outcomes (Minke & Anderson, 2005). Individualized Education Plans An Individualized Education Plan is a legal document created to support students with special needs. The IEP is developed as a team and parents are a crucial part of the decision making process (Lasky and Karge, 2011). According to Staples and Siliberto (2010) many parents may be uninvolved in the IEP process because they feel frustrated or unsure, as well, the IEP environment can be intimidating. Parents may also become overwhelmed by the large number of people present and the abundance of information (Teaching Exceptional Children, 2012). Getting parents involved in many school activities throughout the year will allow the parents to meet the team and gain an understanding that that the team members involved in the IEP meeting are working together to create a successful and nurturing learning environment for the child. IDEA requires that schools provide families with information regarding IEP proceedings as well as copies of the IEP and reports (Harry, 2008). To provide a successful collaborative environment with parents in an IEP, request a translator, if needed, and provide IEP information in the parent or guardians native language. Consideration of Routine

13 Families should be given the opportunity to express their own needs and preferences when participating in their child’s educational and future outcomes (Minke & Anderson, 2005). The research indicates that does not stop with the identification goals and values but should include social supports, such as transportation, child care and flexibility in scheduling, as well. Olivos et al. (2010) recommend using a “holistic” approach when considering the family’s needs and making the appropriate accommodations. Educators should heed the same advice and consider the routine and culture of a family when designing intervention to help ensure consistent implementation at home (Bernheimer & Keogh, 1995). If the intervention is not feasible then it cannot be sustained. Divergence from successful implementation is often seen as lack of commitment by the family, but it may be due to the lack logistical support needed for productive collaboration when establishing an intervention that could be maintained at home (Bernheimer & Keogh, 1995; Harry, 2008). While this process takes additional effort and careful listening, individualization is truly a cornerstone of IDEA (Coots, 2007). Conclusion Positive partnerships between educators and the culturally diverse parent is eminent to student success (Tran, 2014; Lasky & Karge, 2011; Coots, 2007). The promotion of a positive partnership benefits all stakeholders, the most central being the student (Olivos et al., 2010). Increased parent involvement, when done in a fashion that promotes a partnership with the school, improves delivery of services, educational outcomes, and a student’s overall quality of life (Blue-Banning et al., 2004; Coots, 2007). Achieving positive partnerships means overcoming barriers such as cultural bias, negative imagery, and stereotypes. Educators must elicit a sense of trust and respect and educate the families on the importance of collaboration for student achievement. Limitations The data and results presented in this paper include limitations. First, the study is based on data collected from a small sample of 46 elementary and middle school teachers from one Orange County school district teaching in a region with low percentage of culturally diverse families. Further study of understanding ways to successfully collaborate with CDL parents would be beneficial with a broader and more representative sample. As well, collecting data through survey and interview may elicit more honest results from participants. References Bernheimer, L., & Keogh, B. (1995). Weaving interventions into the fabric of everyday life: An approach to family assessment. Topics in Early Childhood Education, 15, 415-434. Blue-Banning, M., Summers, J.A., Frankland, H.C., Lord Nelson, L., & Beegle, G. (2004). Dimensions of family and professional partnerships: Constructive guidelines for collaboration. Exceptional Children, 70(2), 167-184. Chopra, R. V., Sandoval-Lucero, E., Aragon, L., Bernal, C., Berg de Balderas, H., & Carroll, D. (2004). The paraprofessional role of connector. Remedial and Special Education, 25(4), 219-231.

14 Coots, J. (2007). Building bridges with families: Honoring the mandates of IDEIA. Issues in Teacher Education, 16(2), 33-40. Ditrano, C. J. & Silverstein, L. B. (2006). Listening to Parents' Voices: Participatory action research in the schools. Professional Psychology: Research and Practice, 37(4), 359-366. D’Haem, J. and Griswold, P. (2016). Teacher Educators’ and Student Teachers’ Beliefs About Preparation for Working With Families Including Those From Diverse Socioeconomic and Cultural Backgrounds. Education and Urban Society, 49(1), pp.81-109. Epstein, J. L. (2002). School, family, and community partnerships: Caring for the children we share. In Epstein, J. L., Sanders, M. G., Simon, B. S., Salinas, K. C., Jansorn, N. R., Van Voorhis, F. L., School, family, and community partnerships: Your handbook for action (2nd ed., pp. 7-29). Thousand Oaks, CA: Corwin Press. Epstein, J. (2018). School, family, and community partnerships in teachers’ professional work. Journal of Education for Teaching, 44(3), pp.397-406. Grant, K. and Ray, J. (2019). Home, school, and community collaboration. Harry, B. (2008). Collaboration with Culturally and Linguistically Diverse Families: Ideal versus reality. Exceptional Children, 74(3), 372-388 Harry, B., Klingner, J. K., & Hart, J. (2005). African-American families under fire: Ethnographic views of family strengths. Remedial and Special Education, 26, 101-112. Lightfoot, D. (2004). “Some parents just don’t care”: Decoding the meanings of parental involvement in urban schools. Urban Education, 39(1), 91-107. Minke, K. M. & Anderson, K. J. (2005). Family-school collaboration and positive behavior support. Journal of Positive Behavior Interventions, 7(3), 181-185. Olivos, E., Gallagher, R., & Aguilar, J. (2010). Fostering collaboration with culturally and linguistically diverse families of children with moderate to severe disabilities. Journal of Educational & Psychological Consultation, 20(1), 28-40. Tran, Y. (2014). Addressing reciprocity between families and schools: Why these bridges are instrumental for students’ academic success. Improving Schools, 17(1), pp.18-29. Walker, J., Ice, C., Hoover-Dempsey, K. and Sandler, H. (2011). Latino Parents' Motivations for Involvement in Their Children's Schooling: An Exploratory Study. The Elementary School Journal, 111(3), pp.409-429. About the Authors Dr. Darlene Desbrow is a special education teacher in the Los Alamitos Unified School District located in Orange County, California. She also works as an adjunct professor of education for United States University. Dr. Desbrow has had over 20 years of experience working with mild to moderate learning disabled students at the elementary and middle school levels. She has recently earned AASEP Board Certification in Special Education and is passionate about the importance of continued growth and improvement within the field of special education. Nicole Manly, M.S., is a special education teacher in the Los Alamitos Unified School District located in Orange County, California. She has taught students with mild to severe learning, behavioral, functional and cognitive disabilities at both the elementary and middle school levels. She has a Masters of Science in special education and is passionate about the inclusion of individuals with disabilities in both schools and the greater community.

15 Characteristics of Rural STEM Clubs and Implications for Students with Disabilities Karin M. Fisher, PhD. Peggy Shannon-Baker, PhD. Kelly Brooksher, EdD. Kania Greer, EdD. Georgia Southern University Abstract There are many benefits for students to participate in extracurricular science, technology, engineering, and mathematics (STEM) activities or clubs. It is also known that students with disabilities (SWD) do not participate as much as their peers without disabilities (SWOD). However, there is a lack of research on SWD and their participation in STEM clubs. This sequential explanatory mixed method study explored teachers’ perceptions of the types and characteristics of STEM clubs and their participants, and their professional development (PD) to work with SWD in their clubs. Findings suggest a variety of STEM clubs are offered with an average of 20 students each but most participants did not know how many SWD were in their clubs. None had PD to work with SWD in informal environments. A discussion of findings include accommodations STEM club sponsors can use with SWD. Keywords: STEM, students with disabilities, clubs, extracurricular Activities, rural, teachers, mixed methods Characteristics of Rural STEM Clubs and Implications for Students with Disabilities Science, technology, engineering and mathematics (STEM) education is not limited to the formal P-12 classroom. In fact, children spend a little more than 1,200 hours (180 days by 7 hours a day) in school which is only 20% of their waking hours. All children are natural scientists at birth, as they discover and explore, trying to make sense of their worlds (Bers, 2008). Since children are learning all the time, informal learning programs like school based extracurricular activities, have the capability to engage, inspire, and stimulate interest by letting them experiment with STEM ideas in real-world situations. Extracurricular STEM programs “are increasingly recognized as playing a valuable role in improving...STEM education” (Afterschool Alliance, 2015, p. 9). Due to the ever-increasing demands of class time, many students without disabilities (SWOD) having their interest in STEM met after school through clubs and other extracurricular activities (Afterschool Alliance, 2015). Research is prevalent on the positive impact of extracurricular activities on student development (e.g., Darling et al., 2005; Fredericks & Eccles, 2006; McGuire & McDonnell, 2008; Shulruf et al., 2008). Several studies indicated inquiry-based afterschool STEM programs have similar gains (Lauer et al., 2006; Makitalo-Siegl et al., 2011). In 2009, the National Research Council published a report that examined informal STEM learning settings, like STEM clubs, and found learners experience excitement, interest, and motivation about STEM subjects and begin to think of themselves as science learners. Additionally, Cutucache

16 and colleagues (2018) found an increase in STEM content knowledge in participants of an afterschool STEM club. Indeed, students make gains when they participate in STEM clubs, however, fewer SWD participate in extracurricular activities than their peers without disabilities (U.S. Department of Education, 2013; Kleinert et al, 2007). Coster and colleagues (2012) found in their study that 62% of parents of SWD indicated their child never participated in teams, clubs, and organizations in their schools or communities. This is surprising given the specific directions for IEP teams to consider SWD participation in nonacademic activities by the Individual with Disabilities Education Act (IDEA; 2004). In fact, Power and colleagues (2005) reviewed the Individual Education Programs (IEPs) of almost 400 SWDs and found only 11% included information related to extracurricular involvement. Researchers also found barriers to traditional STEM education for SWD to include low expectations, limited exposure to prerequisite courses, and lack of role models and access to individualized supports (Alston et al., 2002; Lee, 2011; Rule et al., 2009). Another study by Eriksson and colleagues (2007) found lower participation rates of SWD in unstructured STEM related activities compared to their peers. Research also indicates students in rural areas have additional barriers related to participation in STEM clubs. Rural schools are often challenged when it comes to offering a variety of after-school activities and in preparing students to become STEM literate (Schafft & Jackson, 2011). Rural school districts often lack access to specialists, have few discretionary resources, and students tend to be geographically isolated (Feinberg et al., 2005). In addition, many rural students face transportation issues, have trouble accessing technological advances (Spencer, 2017), have limited access to advanced coursework in mathematics and science (National Science Board, 2014), and face economic barriers that impact educational opportunities for future employment (Lapan et al., 2007). For example, students in small rural schools are less likely than their urban counterparts to take algebra in the 8th Grade (Spielhagen, 2006) or calculus in high school (Kena et al., 2016). According to the National Center of Education Statistics (NCES; 2015), rural students are substantially less likely to attend college compared to their urban, or suburban counterparts. With these barriers, the potential lack of involvement and support of SWD in afterschool activities could further increase the achievement gap between SWOD and SWD in STEM content areas. The Afterschool Alliance (2016) found there is a strong demand for more quality afterschool and summer learning programs for rural children. However, Hott and colleagues (2019) found teachers in rural districts have difficulty accessing needed Professional Development (PD) and that lack of access may impact quality and quantity of STEM clubs. There is a distinct lack of research on the types and characteristics of extracurricular STEM activities and the characteristics of the participants in rural areas and what PD teachers/sponsors have received to work with SWD in their clubs. The rationale for the current study is to identify extracurricular STEM clubs in two rural districts and study the recruitment, retention, and PD/knowledge of working with SWD by the teacher/sponsor to add to the literature base of SWD and STEM clubs. The following research questions (RQ) guided this exploratory study:

17 RQ1: What types of STEM extracurricular activities are offered in the two rural districts? RQ2: What are the characteristics of the clubs and their participants? RQ3: What professional development has the teacher/sponsor had to work with SWD? Method The study focused on two rural counties in the Southeast United States (U.S.) where a convenience sample of 16 teachers/sponsors completed a survey and 69% (n =11) participated in follow-up interviews. According to the National Center of Educational Statistics (2019) the locale of district 1 is rural: distant and district 2 is rural: fringe. Table 1. District Locale District in study

Type

Miles from urban area Population

Rural: Distant

5-25

>50,000

Rural: Fringe

2.5-10

2500-50000

As noted in Table 1, rural: distant is defined as rural territory that is between 5 and 25 miles from an urbanized area (populations greater than 50,000) and between 2.5 and 10 miles from an urban cluster (populations between 2,500-50,000). Rural: fringe is defined as rural territory less than 5 miles from an urbanized area and less than 2.5 miles from an urban cluster. Next, the researchers used National Center for Education Statistics (NCES; 2019) data to define the locale for each school that had a sponsor participate in the study. Table 2 describes each school. Table 2. School Locale Definitions School

Number of Participants

Locale

Number of Students (17-18)

Elementary School 1

Suburb: Large

669

Elementary School 2

Rural: Distant

452

Elementary School 3

Rural: Distant

384

Elementary School 4

Rural: Fringe

1,225

Middle School 1

Rural: Fringe

710

Middle School 2

Rural: Fringe

1,775

Middle School 3

Rural: Distant

812

Middle/High School

Rural: Distant

441

18 High School 1

Suburb: Large

2,722

High School 2

Town: Distant

1,640

High School 3

Rural: Distant

563

Schools Elementary school 1 is a large suburban school with 669 students in the 2019-2020 academic year. White students represent 67% of the student population, 13% are black, 10% are Hispanic, and 6% are multi-racial. Economically disadvantaged students represent 13% of the population and 8% have disabilities. The researchers included this school in the study even though it is located in a suburban area because it is located in a predominantly rural county. Elementary school 2 is a rural school with 452 students. White students represent 69% of its population, 22% are black, and 5% are multi-racial. Economically disadvantaged students make up 37% of its population and 17% are identified as having a disability. Elementary school 3 is a rural school with 384 students in the 2019-2020 school year. Its student population is 59% white, 26% black, 8% Hispanic and 7% multi-racial. Over 50% of this school’s students are from low income families. Students with disabilities represent 13% of its student population and 4% are English Language Learners. Elementary school 4 is a large rural school with 1,225 students in the 2019-2020 school year. White students make up 79% of its population, 7% are Hispanic, 6% are black, and 5% are multi-racial. Only 3% of its students are economically disadvantaged and 8% are identified as having a disability. Middle school 1 is a rural school with 710 students. Black students represent 65% of its population, 21% is white and 10% is multiracial. Economically disadvantaged students represent 54% of the population and 19% are identified as having a disability. Additionally, 5% are English language learners. Middle school 2 is a rural school with 1,775 students. White students represent 67% of its population, 14% are black students, 10% are Hispanic, and 5% are multiracial. Economically disadvantaged students represent 9% of its student population and 10% are identified as having a disability. Middle school 3 is a rural school with 812 students in 2019-2020. White students represent 80% of its population, 11% are black, and 4% are multiracial. Economically disadvantaged students represent 22% of its student population and 15% are identified as having a disability. One of the schools represented in our population was a combination middle and high school (grades 6-12) representing 441 students. White students represent 58% of the student population while 32% are black. Hispanic students represent 6% of the population. Economically disadvantaged students represent 35% and 12% have disabilities.

19 High school 1 is a large suburban school with 2,272 students. White students make up 64% of its population and 16% are black. Hispanic students represent 10% and 6% are considered multiracial. Only 7% are economically disadvantaged and 9% have a disability. This high school has a 91% graduation rate. The participants from this school were included in our study because this suburban high school is located in a rural county. High school 2 has 1,640 students in the 2019-2020 school year. Black students represent 49% of the population, 38% are white, and 8% are Hispanic. Economically disadvantaged students represent 34% of the population and 14% are identified as a SWD. This high school was included in our study because it is located in a rural county. High school 3 is a small rural school with 563 students. White students represent 67% of the population while 24% are black. Hispanic students represent 3% and 26% are considered economically disadvantaged. Students with disabilities make up 16% of the population. Participants Of the 27 potential teachers on the initial list, 18 teacher/sponsors responded and 16 completed the survey for a response rate of 59%. Table 3 displays the demographics of the 16 survey participants. The participants were half male and half female. They were predominantly white (94%). Most of the participants (38%) represented elementary schools and 25% of the participants in the survey did not state what school they represented. The surveys were sent at the beginning of April and all 16 responses were received by the end of May. Of the 16 participants in the survey, 14 (88%) identified where their clubs meet and represent 11 different schools. As seen in Table 2, 73% (n = 8) of the schools were identified as being in rural areas. However, 42% (n = 6) of the 14 participants who answered the question on the survey did not teach at schools identified as rural because the two schools from suburban areas had multiple teachers participate in the survey. Table 3. Descriptive Statistics of Survey Participants N = 16

% of N

Male

50%

Female

50%

Caucasian

94%

Hispanic

56%

Gender

Ethnicity

District District 1

20 District 2

44%

Elementary

38%

Middle

Middle/High combined

High

25%

No response

25%

School Level

The researchers used a sequential explanatory mixed methods design (Creswell & Plano Clark, 2017). The design typically begins with a quantitative phase followed by a qualitative phase used to help explain, extend, and/or complexify the findings from the quantitative phase (Griffin, 2012; Plano Clark & Ivankova, 2016). These designs also typically use the findings or sample from the quantitative phase to identify participants for the qualitative phase (Creswell & Plano Clark, 2017). Multimethod and mixed methods research approaches have been shown to enhance the interpretations of and provide a richer picture of special education processes (e.g., Collins et al., 2006; Hott et al., 2019). Data were collected in two phases for this study: Phase I entailed an online survey of teachers/coaches who lead STEM extracurricular activities, and Phase II consisted of individual semi-structured interviews with survey participants. Phase I: Quantitative Survey Implementation and Analysis Procedures The survey sent to teachers/sponsors was modified from a previous study about after school STEM activities in another southeast state (Fisher, 2016). The purpose of that research study was to identify the types of afterschool STEM activities, inclusion of SWD in STEM activities, types of SWD, number of SWD who participated in the activities and then compared district STEM offerings with standardized 8th grade science scores of SWD. The results of that study were that most districts do not track the number of SWD in their STEM clubs, the largest category of SWD who do participate were students with learning disabilities (SLDs), and there was a small positive correlation between the number of STEM clubs offered in a district and the results of the 8th Grade standardized science scores for SWD. Two experts in the field of STEM education reviewed the questionnaire for clarity, coverage, face, content, and construct validity. The survey was revised based on their feedback and approved by the Institutional Review Board (IRB). The survey consisted of 19 questions and is available in Appendix A. In order to increase the response rate, the length of the survey was purposefully kept short (Dillman, 2000; Mavletova, 2013). The overall survey focus was to determine the types of STEM extracurricular activities offered, the characteristics of the clubs and their participants, and the knowledge/PD of the teacher/sponsor to work with SWDs. The first section contained questions aimed at collecting basic information about the clubs and teacher/sponsors, such as the types of clubs offered, which club(s) the participant coached, and their experiences leading clubs. The third section contained questions about the number of

21 students who participated in the club, their age ranges, and information about any SWDs known to the teacher/sponsor in the club (e.g., type of disabilities, number of students). The survey also asked if the teacher/sponsor ever received PD to work with SWD in extracurricular activities, as well as their experiences recruiting SWD. The questionnaire consisted of several short answer response questions for expansion of initial answers. The survey was prepared in Qualtrics and shared via email with an anonymized link. Descriptive statistics were used to evaluate the results of the survey. The researchers independently coded the open-ended responses. Lastly, the results were analyzed qualitatively by the school level (e.g., elementary, middle, high school) of the participants. Results Phase I Survey question 5 asked about the type of STEM club sponsored by the participants and multiple clubs could be selected. Respondents selected 30 different clubs which means most participants sponsor multiple STEM clubs. Indeed, each participant led an average of 2 clubs. Robotics was the most common selected (23%) with STEM Club and Coding Club coming in second at 16% each. Several other types of clubs were offered such as Science Club, Engineering Club, and Tech Clubs. See Table 4 for a list of specific STEM clubs by district. Table 4. Clubs Reported at Schools by Participants SQ Type

What type(s) of STEM* clubs do you sponsor or coach (select all that apply)?

District 1

District 2

Total

Robotics

3 (43%)

4 (57%)

SECME**

3 (100%)

0 (0%)

Science Club

0 (0%)

2 (100%)

STEM Club

3 (60%)

2 (40%)

Coding Club

1 (20%)

4 (80%)

Engineering Club

0 (0%)

1 (100%)

Other

3 (43%)

4 (57%)

Total 13 (43%) 17 (57%) 30 *STEM Clubs are all encompassing for science, technology, engineering, and math type clubs. The name of the club may be STEM or it may be Robotics but it is still a STEM club. **SECME was founded as the Southeastern Consortium of Minorities in Engineering but now most commonly just goes by SECME. Participants also selected other STEM clubs offered at their schools. The following clubs were selected: Robotics, SECME, Science Olympiad, Science Club, STEM Club, Coding Club, Mathletics, Engineering Club, Math Olympiad, and Video Game Club. Robotics and Science Olympiad had the highest results with 17% of the participants selecting those clubs. Participants were asked how long they served as teacher/sponsors of extracurricular STEM activities. The results were that 75% of the participants only sponsored clubs from 1-3 years. See Table 4 for

22 the number of students whose teacher/sponsors stated regularly participate in their STEM clubs. Indeed, over 40% of the teacher/sponsors had clubs with a regular attendance rate of over 20 students. Table 5. How Many Students Regularly Participate Number of students 6-10 10-15 15-20 20+

n 2 4 3 7

% 13% 25% 19% 44%

When asked how many students in their clubs had disabilities, 14 of the 16 respondents who selected yes and unknown as seen in Table 6. Most (n = 6, 43%) selected they didn’t know how many had disabilities. When asked if they had received any PD to work with SWDs in extracurricular settings, only three participants (19%) stated yes. Two teachers responded to our request for the type of PD. One wrote a “workshop” and the other stated “I have taken PD in the past for developmentally delayed children”. Table 6. Survey Question Results on SWD Participation and PD Survey Question

Unknown

Yes

Are students with disabilities participating in your clubs?

6 (43%)

8 (35%)*

2 (13%)

Have you received PD to work with SWDs in informal learning environments?

3 (19%)

13 (81%)

*There were varying numbers reported by teachers: three said at least 1, two stated more than 5, with one participating selecting 2, 3, and 5. As noted in Table 7, three (21%) selected one SWD and two (14%) stated more than five SWDs participated in their clubs. One participant each selected two, three, and five SWDs participated in their clubs. Table 7. Survey Question Results on Number of Participating SWDs Survey Question

How many SWDs participate in your clubs?

3 1 1 (21%) (7%) (7%)

Unknown

1 (7%)

2 (14%)

6 (38%)

When asked what types of disabilities were represented in their clubs, the respondents could select more than one category of disabilities. As noted in Table 8, the most frequent response (n = 8) was emotional behavior disorder (EBD). The next most frequent response was SLD (n = 7). Three respondents selected autism spectrum disorders (ASD) and one participant each (not

23 necessarily the same participant) selected developmental delay (DD), physical disability, and hearing impaired (HI). Table 8. Survey Question Results of Type of Disabilities in Clubs N = 12 Survey Question

EBD

SLD

ASD

What type of disabilities were represented in your clubs?

8 7 3 (67%) (58%) (25%)

Physical

1 1 (8%) (8%)

HI 1 (8%)

Respondents were asked if they thought SWD participated less than their nondisabled peers and 14 answered the question as shown in Table 9. Five (31%) responded yes, seven (44%) said maybe, and two selected no. When asked if they had access to the IEPs of SWDs, only 12 of the 16 participants responded to the question. Most of the participants (n = 9, 56%) stated yes, one said no, and two selected other. Both respondents who stated “other” wrote they only have access if the student is also in one of their classes during the school day. Table 9. Survey Responses on Participation of SWDs and Access to IEPs. Survey Question

Maybe/Other

Yes

Do SWDs participate less than their peers?

7 (44%)

5 (31%) 2 (14%)

Do you have access to SWDs IEPs?

2 (17%)

9 (75%) 1 (8%)

Teacher/sponsors were almost evenly split on the survey with slightly more than half (56%, n = 9) indicating they actively recruit SWD. Interesting to note is that teachers/sponsors indicated on the survey they have access to IEP information for students in their clubs. Indeed, 75% of the respondents (n = 9) said yes, they did have access to this information. However, in the follow up interviews only two of the 11 teacher/sponsors (18%) said they had IEP access but only because the SWDs were in their classes during the school day. Phase II: Interview Procedures and Analysis In the Phase I survey, participants were asked if they would be interested in participating in a follow-up interview. The purpose of the interviews was to provide additional information, clarifications, and details regarding the survey data as a whole and their individual experiences as a teacher sponsor for STEM clubs. The researchers designed the semi-structured interview protocol (Appendix B) which contained 15 questions. The initial set of questions focused on basic information about the STEM clubs to contextualize their own experiences and allow the teachers to feel comfortable with the interviewer(s). The protocol also included questions about the number of students in the club with IEPs, as well as the teacher/sponsors’ PD regarding working with SWDs in an informal learning environment, like clubs. Questions asked included participant recruiting practices and requirements of SWDs. The protocol ended with general

24 questions asking the teacher/sponsor to share anything else that would be helpful and if they had questions about the research. The research team traveled in pairs to conduct the interviews where one team member generally led the interview, and both took notes. The interviews were conducted with the teacher/sponsors at their own school at a time and date that best fit their schedules. The interviews were audio recorded, and later transcribed verbatim. After reading the transcripts, all members of the research team independently completed open coding resulting in three themes: (a) recruitment, (b) retention, and (c) knowledge/PD. One person from the research team then collated the coded passages to check for interrater reliability of the interviews. There was a strong internal consistency among researchers in the coding of interviews. Each interview participant received a $25 gift card. Member checking was conducted by two volunteer participants. A draft of the results was sent to the members and their feedback was incorporated. Member checking participants each received an additional $25 gift card. Both stated the draft manuscript is representative of what they experienced in terms of the survey and follow up interviews. After the three themes were determined, the researchers operationally defined them. Recruitment was operationally defined as the act of enlisting new students to join an extracurricular STEM club. Retention was defined as the continued engagement of students in an extracurricular STEM club. Professional development was defined as activities formally organized to improve the requisite knowledge, skills, pedagogies, and effectiveness of the teacher sponsoring STEM clubs. Lastly, knowledge was operationally defined as facts, information, procedures and pedagogies related to the strengths and needs of SWD and their participation in STEM clubs. Knowledge includes information about types of SWD, which students were identified as having a disability, and what is included in IEPs including accommodations. To test the operational definitions, an expert in the area of SWD and STEM was asked to review the definitions for clarity and agreed the definitions operationally define each variable. The instruments used to measure recruitment, retention, and PD/knowledge are the survey openended questions and the transcripts from the interviews. The method used to measure the constructs were visual checks and colored highlighters. Four different researchers independently highlighted the transcripts, researcher interview notes, and written responses on the surveys for the three themes. Next the marked-up transcripts were compared to determine consensus of color coding of the three different themes. Differences were discussed and resolved when all researchers agreed on the data found in the transcripts, notes, and survey data. Results Phase II Recruitment. The types of clubs offered and the teacher/sponsors who run them can affect the recruitment of SWD into clubs at all levels. In addition, perceived or real barriers to recruitment practices can have an impact on whether students enroll in clubs. For the most part, high school clubs met in the afternoon while many elementary and middle schools’ clubs met in the mornings. When asked if they actively recruit SWD on the survey, 56 percent (n = 9) said yes.

25 Of those, three teacher/sponsors were at the high school level, one at the middle school level, and five were elementary school club coaches or teacher sponsors. Teachers/sponsors in the high school indicated “every student who enrolls in the club participates” and they “work closely with the special education teachers” when it comes to student participation. Typically, high school teachers/sponsors reported being “pretty relaxed about how [they] recruit” and that they have an “open house” at the beginning of the year. Otherwise, it is up to students “to be interested” enough to pursue joining the club. Middle school teachers/sponsors indicated they “don’t often have to recruit” because students want to get into the STEM Clubs and they “come to me”. However, the teacher/sponsors said some “general announcements” are made to the school. One teacher did mention they recruit through “word of mouth” but they feel you have to “show them interest and ask [SWD]” to join, pointing out “you don’t know what they [can] handle until you try them.” For elementary school teachers/sponsors, recruitment usually involves sending fliers home and students sign up “on interest.” Most respondents said they did not think about “actively recruiting SWD,” but can see how it would be a benefit stating “just because they have a disability doesn’t mean they can’t participate” and that for some students “[who] may not connect academically…[this] gives them some passions to kind [of] [want to] come to school.” During the interviews, all grade level teacher/sponsors indicated SWD may have barriers to joining that other students do not have or have to the same degree. Response examples include “transportation” and “parents who aren’t comfortable signing up their child.” More responses included “their disability will be a hindrance,” and “club meeting times” conflicted with remediation courses for SWD when clubs met during the school day. Retention. Once SWDs join a club, keeping them becomes a priority especially for those clubs participating in STEM competitions. Being aware of what keeps SWD involved and their barriers to continued participation are important when thinking about how to retain SWD in the clubs/organizations. When asked on the survey if respondents (N = 16) felt SWD participated less than their peers five said yes, two said no, and seven said maybe. One common thread among all teachers/sponsors was a lack of awareness of accommodations for SWD participating in STEM activities. During the interviews, high school teacher/sponsors stated the level of “competition is pretty high,” and they speculate that many SWD feel they cannot compete because of a “[lack of] confidence,” “feeling like the club is unapproachable,” and/or lack of “advocacy skills” at this level since many struggle socially and academically. However, these teachers/sponsors also attempt to “differentiate and help them understand the material.” Another teacher/sponsor felt SWD are drawn to “fine arts” programs like chorus where students can participate without the competitive academic pressures often associated with STEM Clubs. Many of the high school teacher/sponsors acknowledged not really understanding why SWD do not participate more and speculated that “[lack of] confidence,” “feeling like the club is unapproachable,” and/or lack of “advocacy skills” could be barriers to students continuing to participate in these clubs.

26 For middle school teachers/sponsors, only one respondent said he/she felt SWD participate less than their peers while two others said maybe. One of the main retention issues mentioned during the interviews was SWD struggling in reading or mathematics. Some middle school teachers/sponsors found it easier to be “familiar with all the special ed teachers”. Additionally they stated they find out “what students need''. However, other teachers/sponsors felt students “lose interest or [the club] is not what they thought it was going to be.” Some teachers/sponsors stated the students may process things more slowly and that makes “[them] uncomfortable”. At the elementary level, five respondents felt SWD participated less than their peers, one respondent felt SWD did participate more than their peers, and one selected maybe on the survey. When interviewed, elementary teachers/sponsors expressed a willingness to speak to “other teachers” about students so they can provide needed support. In some instances, however, teacher/sponsors stated coming to an afterschool program is a “privilege” (for students) and this could make it difficult for SWD (especially EBD). Many teachers/sponsors stated transportation is one of the largest barriers to SWD retention in clubs. Lastly, teacher/sponsors felt SWD may “feel intimidated” being in clubs with students who they see as smarter and more able. Knowledge/PD. Teachers/sponsors have knowledge of working with SWD in general and that SWD have specific IEP accommodations to make material accessible to them. Indeed, nine survey participants (56%) said they had access to IEPs. However, none of the teacher/sponsors interviewed knew the IEP accommodations or information on students who joined their clubs unless that student was already in their classroom. In fact, in many cases the teacher/sponsors were only speculating on the number of SWD’s in their clubs, because they were not officially informed by the special education case manager or parent. Most of the elementary teachers/sponsors expressed a willingness and an effort to go and speak with classroom teachers if they knew a student needed special modifications. Most of the middle and high school teachers/sponsors in the study “figured out what they need” “on the job”. Knowledge/Development. On the survey, 13 of the 16 respondents indicated they have not received any PD to work with SWD in informal learning environments. Three respondents, one at each level (elementary, middle, and high) selected yes, they have received some PD to work with SWD in their clubs. Conversely, during the interviews all teachers/sponsors (n = 11) indicated there was a lack of information and PD on how to work with SWD in informal settings. Many even went so far as to say it was all “on the job training” and that “you learn as you go through”. Teacher/sponsors indicated they had workshops and college classes on disabilities, but none of the teacher/sponsors interviewed were aware that IDEA required IEPs address extracurricular needs or that accommodations were required in extracurricular settings as a related service. However, all the teachers/sponsors indicated they would “meet the students where they are” and help them as they could regardless of any formal training to do so. All admitted to understanding basic accommodations, but several admitted they “don’t face the same problems that traditional classroom teachers face” and most of what is learned is learned “informally”. Discussion The purpose of this study was to answer the following research questions:

27 RQ1: What types of STEM extracurricular activities are offered in the two rural districts? RQ2: What are the characteristics of the clubs and their participants? RQ3: What professional development has the teacher/sponsor had to work with SWD? Results showed SWDs in rural areas do have access to STEM clubs (see Table 4), however, the variety of these clubs, within the rural schools and districts, are often limited due to lack of resources (Afterschool Alliance, 2016; Afterschool Alliance, 2014). The STEM clubs were reported to have many students (see Table 5) and consist mostly of SWOD. These responses are consistent with the research that SWD participated less than SWOD (Coster et al., 2012; Kleinert et al., 2007; Power et al., 2005). Most of the participants indicated they did not know how many SWDs they had in their clubs. The finding is similar to those of the Fisher (2016) study in another state. It stands to reason if teachers/sponsors do not know which students have disabilities, they would not know how to accommodate them. As a result, districts should set up a mechanism for extracurricular teachers/sponsors to have access to information on disability status and IEP’s of SWD who participate in their clubs. Many SWD often need instructional support with subject matter and based on club schedules, lack of support could impact SWD participation even if they were interested. Support during the school day includes specially designed instruction in a content area or in behavior/social skills. It is well known SWDs often struggle in STEM content areas (i.e., Basham & Marino, 2013; McFarland et al., 2017; National Science Foundation, 2019). That struggle and lack of support could impact SWD participation and retention in extracurricular STEM activities. However, the teachers/sponsors who facilitate the clubs in this study appear to accept everyone in their clubs and ensure each student has an opportunity to succeed. STEM Club sponsors reported retaining SWD was difficult due to the lack of knowledge of who are SWDs, as well as what accommodations they should be making for them. While it is hoped students would advocate for themselves in these situations, it often falls on the club teacher/sponsor to ensure all students have equal access to the materials. For example, a student with dyslexia can benefit by having instructions read to them. A student with autism or general anxiety disorder may need competition-based projects broken down into checklists. If these accommodations are offered during the school day, they need to be offered in school sponsored clubs as well (IDEA, 2004). However, the results of this study show that most of the teachers/sponsors are not aware of the needed accommodations in order to make them. Implications for practice include adding specific accommodations to include in the IEP that teachers/coaches could use are read-aloud, breaking down large assignments into smaller chunks, and checking for understanding. For STEM competitions, accommodations could include the use of headphones and a sensory area where students with sensory impairments can find a quiet place to regroup (Fisher, 2019). The use of a buddy system or peer tutors could also be used for SWD to be included in afterschool activities and STEM competitions. Districts should also consider incentivizing special education teachers to support club sponsors.

Additionally, participants indicated a lack of PD offered on SWD in extracurricular environments (see Table 6). The teacher/sponsors took required courses in their college programs. Indeed, teacher preparation programs often require general educators to take one or more special education courses (Mader, 2017; Tait & Purdie, 2000). These introductory courses, however, are often considered ineffective for new teachers to educate SWD within their classes (Young, 2011), let alone their clubs. Professional development could include how to best provide accommodations for SWDs in informal learning environments. Offering an opportunity for these teachers/sponsors to be successful in mentoring and coaching SWDs in non-academic environments may allow students to feel more successful within classrooms, especially in STEM content areas. As one teacher said, “these are my kids who can persevere and problem solve and think through…they kind of have to do that in the academic setting…where my gifted learners are the first ones to shut down because they didn’t get it right the first time.” The last implication for practice is for STEM club sponsors to specifically identify and recruit SWDs into their clubs. The STEM workforce needs more diverse workers because these students are known to think outside of the box and can help fuel innovation in our ever changing society. If sponsors specifically recruit SWD into their clubs, the sponsor, school, and/or district could apply for grants specified in the Every Student Succeeds Act (ESSA). Grants under ESSA are for marginalized students like SWD and can help the sponsor, school, and district fund more of these STEM based extracurricular programs. Limitations The limitations of the study include teacher perceptions, sample size, accessibility to information, and timing of the study. Teacher perceptions. First, the teacher surveys and interviews used in this research were selfreported teacher perceptions. Participants may have over- or under-represented their knowledge. The information is speculative of the teacher/sponsors’ knowledge and recollection of SWD participation in extracurricular STEM activities. Findings indicated most club teachers/sponsors did not have an actual list of SWD in their clubs. Without a list of identified SWD, this allowed for a great deal of subjectivity on the part of the club teacher/sponsors. During interviews, it was apparent teachers/sponsors most often guessed about disability categories. The discrepancy of knowledge of the IEP is also a limitation in the study. In the survey, most of the participants (56%) stated they had access to the IEPs of the SWDs in their clubs but in the follow up interviews, none of the participants stated they had access. Sample size. The second limitation of the study is the sample size. As with all survey research, it was difficult to determine if a representative sample was obtained. Of the 27 teacher/sponsors gathered from websites and names generated by local school principals, 16 completed the survey. Additionally, a convenience sample was used and nonresponse error was not calculated. The qualitative analysis was intended to clarify and support the survey results and lends credibility to the findings. Although there was representation from elementary, middle, and high

29 school teachers/sponsors in both districts, it is plausible responses may not be indicative of true population norms. Accessibility to information. A third limitation was the lack of STEM clubs and activities listed on school websites. Schools’ websites varied greatly in the amount of information presented. Some contained a list of clubs with teacher/sponsors’ names and contact information while others had no clubs listed at all. As a result, the population of the number of STEM club teachers/sponsors was not determined. Furthermore, the researchers' agreement with each district stated the principal at each school would be contacted first. This is important because not all principals responded to our requests to conduct research at their school. Indeed, 29% (n = 5) of the principals in district one and 1% (n = 1) of the principals in district 2 did not respond to our email requests. Timing of the study. Researchers attempted to gather a representative sample of STEM club sponsors in the two participating rural school systems. Permission was asked of principals and the surveys were sent electronically to teacher/sponsors near the end of the school year (April). The timing of the surveys may have limited teacher/sponsor ability to devote time to completing surveys. While the survey response rate (56%) was positive, perhaps a different time of the year would have had a better response rate. Future Research Just as the findings presented here, there are several areas in which future research may provide a better understanding. These include recruitment and retention of SWD and PD of teacher/sponsors supporting SWD in clubs. With recruitment and retention, data needs to be collected in more areas to gain a better understanding. As reported, recruitment and retention data are all teacher/sponsor perceptions. Further research could investigate SWD and/or their parents’ perceptions regarding level of comfort and barriers. Additionally, since many teachers/sponsors did not know whether students in their clubs had or did not have disabilities, research could be conducted on the knowledge of club teachers/sponsors and the implementation of a system to notify the teacher/sponsors of the individual needs of SWD. The research on IEPs conducted by Power and colleagues (2005) could be replicated where IEPs are analyzed for information about extracurricular activities as well as the impact and of PD on the IEP requirements for club teachers/sponsors. Further research could also focus on accommodations of SWD who participate in afterschool clubs and how they compare to accommodations used during the school day. In this study, teacher/sponsors speculated comfort levels of SWD dropped when the rigor increased impacting the retention of SWD. Actual student data would confirm or dispute teacher/sponsor perceptions reported in this study. Additionally, future research would be able to determine the attrition of SWD from STEM clubs versus barriers of the disability impacting their performance. Student interviews or surveys may also be able to uncover why students leave STEM clubs. Additionally, student interviews would also reveal why SWD potentially select other clubs instead of STEM clubs.

Another facet of research would be to investigate the PD needs and support of rural teacher/sponsors sponsoring STEM clubs. Research on the impact of access to and implementation of IEP accommodations on the participation of SWD in their clubs should be investigated. The study by Hott and colleagues (2019) could be replicated using STEM teachers/coaches. If teacher/sponsors received and understood the accommodations set forth in academic environments, SWD may participate more in STEM clubs. Additionally, PD could impact the recruitment and retention of SWD in STEM clubs. To gain a deeper understanding of the effects of SWD’s participation in STEM-related clubs, research should be broadened to gain a more cohesive picture of the impacts of recruitment, retention, and knowledge of teachers who sponsor the clubs. Future research should include an expansion of this study to look at a broader spectrum of schools and districts to see how a rural setting compares with other settings across the U.S. References Afterschool Alliance. (2016). America after 3pm special report: The growing importance of Afterschool in Rural Communities, 1-49. Afterschool Alliance. (2015). Defining youth outcomes for STEM learning afterschool, 1-35. Afterschool Alliance. (2014). America after 3pm: Afterschool programs in demand. Washington, DC. Alston, R., Bell, T., & Hampton, J. (2002). Learning disability and career entry into the sciences: A critical analysis of attitudinal factors. Journal of Career Development, 28(4), 263-275. Basham, J. D., & Marino, M. T. (2013). Understanding STEM education and supporting students through Universal Design for Learning. Teaching Exceptional Children. 45(4), 8-15. Bers, M. U. (2008). Blocks, robots and computers: Learning about technology in early childhood. Columbia University Teacher’s College Press, New York, NY. Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. (2006). A model incorporating the rationale and purpose for conducting mixed-methods research in special education and beyond. Learning Disabilities – A Contemporary Journal, 4(1), 67-100. Coster, W., Law, M., Debell, G., Liljenquist, K., Kao, Y. C., Khetani, M., & Teplicky, R. (2013). School participation supports and barriers of students with and without disabilities. Child: Care, Health and Development, 39(4), 535-543. Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage. Cutucache, C., Boham, T., Luhr, J, Sommers, A., Stevenson, N., Sointu, E., & Tapprich, W. (2018). NE STEM 4U afterschool intervention leads to gains in STEM content knowledge for middle school youth. Cogent Education, 5(1), 1-12. Darling, N., Caldwell, L. L., & Smith, R. (2005). Participation in school-based extracurricular activities and adolescent adjustment. Journal of Leisure Research, 37, 51-76. Dillman, D. A. (2000). Mail and internet surveys: The tailored design method. New York: John Wiley and Sons. Eriksson, L., Welander, J., & Granlund, M. (2007). Participation in everyday school activities for children with and without disabilities. Journal of Developmental and Physical Disabilities, 19(5), 485-502. Feinberg, T., Nuijens, K., & Canter, A. (2005). Workload vs. case- load: There’s more to school psychology than numbers. NASP Communiqué, 33.

31 Fisher, K. (2019). ESSA, students with disabilities, and robotics. Technology and Engineering Teacher, 28-32. Fisher, K. (2016). The relationship between extracurricular STEM activities and performance on the Florida science assessment. Electronic Theses and dissertations, 5076. Fredericks, A. J., & Eccles, S. J. (2006). Is extracurricular participation associated with beneficial outcomes? Concurrent and longitudinal relations. Development Psychology, 42(4), 698-713. Griffin, K. A. (2012). Learning to mentor: A mixed methods study of the nature and influence of Black professors’ socialization into their roles as mentors. Journal of the Professoriate, 6(2), 27-58. Retrieved from https://works.bepress.com/kimberly_griffin/5/download/ Hott, B. L., Dibbs, R. A., Naizer, G., Raymond, L., Reid, C. C., & Martin, A. (2019). Practitioner perceptions of algebra strategy and intervention use to support students with mathematics difficulty or disability in rural Texas. Rural Special Education Quarterly, 38(1), 3-14. DOI: 10.1177/8756870518795494 Individuals with Disability Education Act, 34 C.F.R. § 300.117 (2004). Kena, G., Hussar, W., McFarland, J., de Brey, C., Musu-Gillette, L., Wang, X., . . . Barmer, A. (2016). The condition of education 2016 (NCES 2016-144). Washington, DC: National Center for Education Statistics. Kleinert, H. L., Miracle, S. A., & Sheppard-Jones, K. (2007). Including students with moderate and severe disabilities in extracurricular and community activities: Steps to Success. Intellectual and Developmental Disabilities, 45(1), 33-38. Lapan, R., Aoyagi, M., & Kayson, M. (2007). Helping rural adolescents make successful postsecondary transitions: A longitudinal study. Professional School Counseling, 10, 266-272. doi:10.5330/prsc.10.3.u6j3j64h48p27w25 Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of-school time programs: A metanalysis of effects for at-risk students. Review of Educational Research, 76, 275-313. Lee, A. (2011). Postsecondary science, engineering, and mathematics (STEM) enrollment comparisons for students with and without disabilities. Career Development for Exceptional Individuals, 34(2), 72-82. Mader, J. (2017). How teacher training hinders special-needs students. The Atlantic. Retrieved from https://www.theatlantic.com/education/archive/2017/03/how-teacher-traininghinders-special-needs-students/518286/ Makitalo-Siegl, K., Kohnle, C., & Fischer, F. (2011). Computer-supported collaborative inquiry learning and classroom scripts: Effects on helping-seeking processes and learning outcomes. Learning and Instruction, 21(2), 257-266. Mavletova, A. (2013). Data quality in PC and mobile web surveys. Social Science Computer Review, 31(6), 725-743. McGuire, J., & McDonnell, J. (2008). Relationships between recreation levels of selfdetermination for adolescents and young adults with disabilities. Career Development for Exceptional Individuals, 31, 154-163. McFarland, J., Hussar, B., de Brey, C., Snyder, T., Wang, X., Wilkinson-Flicker, S., Gebrekristos, S., Zhang, J., Rathbun, A., Barmer, A., Bullock Mann, F., & Hinz, S. (2017). The condition of education 2017. (NCES 2017-144). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from https://nces.ed.gov/pubs2017/2017144.pdf

32 National Center for Education Statistics. (2015). Percentage of persons ages 18-29 enrolled in colleges or universities, by age group, 4-category local, and sex [data file]. Retrieved from https://nces.ed.gov/surveys/ruraled/tables/b.3.b.-1.asp National Science Board. (2014, February). Science and engineering indicators 2014 (NAB 14-01). Arlington, VA: Author. National Science Foundation (2019). Women, minorities, and persons with disabilities in science and engineering: 2019. Special Report (NSF 13-304). Arlington, VA. Retrieved from http://www.nsf.gov/statistics/wmpd National Research Council (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press. Plano Clark, V. L., & Ivankova, N. V. (2016). Mixed methods research: A guide to the field. Thousand Oaks, CA: SAGE. Power, K. M., Gil-Kashiwabara, E., Geenan, S. J., Powers, L., Balandran, J., & Palmer, C. (2005). Mandates and effective transition planning practices reflected in IEPs. Career Development for Exceptional Individuals, 28, 47-59. Rule, A., Stefanich, G., Hadelhuhn, C., & Peiffer, B. (2009). A working conference on students with disabilities in STEM coursework and career. Iowa City: College of Education, University of Iowa. Schafft, K. A., & Jackson, A. Y. (Eds.). (2011). Rural education for the twenty-first century: Identity, place, and community in a globalized world. University Park: Pennsylvania State University. Shulruf, B., Tumen, S., & Tolley, H. (2008). Extracurricular activities in school, do they matter? Children and Youth Services Review, 30(4), 418-426. Spencer, K. (2017, January 22). Not all towns are created equal, digitally: How a Colorado school district struggles to give its students a technology boost (The Hechinger Report). New York, NY: Teachers College at Columbia University. Retrieved from http://hechingerreport.org/not-all-towns-are- created-equal-digitally/ Spielhagen, F. R. (2006). Closing the achievement gap in math: Considering eighth grade algebra for all students. American Secondary Education, 34(3), 29-41. Tait, K., & Purdie, N. (2000). Attitudes toward disability: Teacher education for inclusive environments in an Australian university. International Journal of Disability, Development and Education, 47(1), 25-38. U.S. Department of Education, Office for Civil Rights. (2013). Students with disabilities in extracurricular athletics. Retrieved from www2.ed.gov/about/offices/list/ocr/letters/colleague-201301-504.pdf Young, K. S. (2011). Institutional separation in schools of education: Understanding the functions of space in general and special education teacher preparation. Teaching and Teacher Education, 27(2), 483-493. About the Authors Karin M. Fisher, PhD, is an assistant professor of special education at Georgia Southern University. Her research interests include students with disabilities, STEM education, extracurricular activities, and special education law. In addition, Fisher serves as President Elect of the Georgia Council for Exceptional Children (CEC) and President of the Early Career Special Interest Group of the Teacher Education Division of CEC.

Peggy Shannon-Baker, PhD, is an assistant professor in the Department of Curriculum, Foundations and Reading and affiliate faculty member of the Women’s, Gender, and Sexuality Studies Program at Georgia Southern University. Shannon-Baker’s research bridges two areas: critical, international approaches to teacher education and the use of social justice-informed artsbased, qualitative, and mixed methods research approaches. Kelly S. Brooksher, EdD, is an Assistant Professor in the Department of Elementary and Special Education at Georgia Southern University. Brooksher has been in education for over two decades, and she has taught general and special education. She has also been a school administrator. Her research interests include: co-teaching, grading and assessment, and higherorder thinking strategies. Kania Greer, EdD, is the Coordinator for the Institute for Interdisciplinary STEM Education in the College of Education at Georgia Southern University. Greer has worked with Students with Disabilities as a Vocational Rehabilitation Counselor and in informal education settings such as camps. Her research interests include science communication and the intersection of formal and informal STEM education.

Appendix A* SURVEY Are you a sponsor of a school-based extracurricular STEM club? The definition of club sponsor is an individual that provides support to the club and is the adult responsible for the students involved in the club. STEM Clubs include any extracurricular activity that is based on improving science, technology, engineering, and/or math skills. Examples include Robotics, Science Olympiad, Coding Club, etc. o Yes o No o Other ________________________________________________ How many years have you been a coach for extracurricular STEM activities? o 1-3 years o 4-6 years o 7-10 years o 10 + years In which district does your club meet? o District 1 o District 2 o District 3 o Other ________________________________________________ What is the name of the school where your club meets? ________________________________________________________________ What type(s) of STEM clubs do you sponsor or coach (select all that apply)? ▢ Robotics ▢ Modeling and Simulation Club ▢ SECME ▢ Science Olympiad ▢ Science club ▢ STEM Club ▢ Math Counts ▢ Coding Club ▢ Science bowl ▢ Math-letics ▢ Engineering club ▢ Math Olympiad ▢ Video Game Club ▢ Other ________________________________________________

35 What other type(s) of STEM clubs are offered at the school where you meet (select all that apply)? ▢ Robotics ▢ Modeling and Simulation Club ▢ SECME ▢ Science Olympiad ▢ Science club ▢ STEM Club ▢ Math Counts ▢ Coding Club ▢ Science bowl ▢ Math-letics ▢ Engineering club ▢ Math Olympiad ▢ Video Game Club ▢ Other ________________________________________________ ▢ None How many students regularly participate in your STEM club? If you coach more than one club, please provide the average number of participants in both clubs. o 0-5 o 6-10 o 10-15 o 15-20 o 20 or more What is the age range of students in your club? (select all that apply) ▢ 5-7 ▢ 8-10 ▢ 11-13 ▢ 14-16 ▢ 17 and older</u> ▢ Unknown</u> Have you received any professional development or training to work with students with disabilities in an extracurricular environment? o Yes o No o Other If yes, what kind of professional development or training have you received to work with students with disabilities in an extracurricular environment? Please provide as much information as possible. ________________________________________________________________

36 Do any of your student participants have disabilities? o Yes o No o Unknown If known, how many of your student participants have disabilities? o 0</u> o 1 o 2 o 3 o 4 o 5 o More than 5 o unknown What kind of disabilities do they have? (select all that apply) ▢ Autism ▢ Learning Disabilities ▢ Speech Language Impairment ▢ Emotional Behavioral Disorder ▢ Developmental Delay ▢ Physical Disability ▢ Visually Impaired ▢ Hearing Impaired ▢ Other ________________________________________________ Do you think students with disabilities participate less in your club than students without disabilities? o Yes o Maybe o No If yes, why do you think they do not participate as much as their peers? ________________________________________________________________

Do you have access to the individual education programs (IEPs) for any students with disabilities who are participating in your club? o Yes o No o Other ________________________________________________ Do you actively recruit students with disabilities? o Yes o No Choose one or more races that you consider yourself to be: ▢ White ▢ Black or African American ▢ American Indian or Alaska Native ▢ Asian ▢ Native Hawaiian or Pacific Islander ▢ Other ________________________________________________ What is your sex? o Male o Female o Other ________________________________________________

38 Appendix B Qualitative Interview Questions Interviewee [pseudonym]: Interviewer(s): Date: ___________ Time: __________

Place: _________________________ [pseudonym]

Consent and Introduction: ● Introduce interviewer(s) ● The purpose of the study: ○ The purpose of our study is to learn more about the STEM extracurricular activities/clubs being offered at schools in the area, and how these activities/clubs include students with disabilities. ● The purpose of the interview: ○ The purpose of the interview is to help us follow-up with more details after the survey you took online with us. The survey was anonymous, but the interview will give us a better understanding of your STEM club(s) and activities. ● The general plan and expected length of the interview: ○ We have some basic questions about the STEM activities/clubs at your school. ○ The interview should take about 20-30 minutes. ● Plan for using the results from the interview: ○ We plan to analyze all the interviews and surveys together to help us get a better sense of the STEM activities/clubs in the area. ○ Do you want us to share the final report with you? ● Any follow-up after the interview: ○ We may follow-up via email if we have any questions. You can also ask for more information later. ● We want to get your permission to audio record the interview so that we have the best notes to use later. Is that ok? (If not, can we just take written notes?) Interview questions ● What STEM clubs or activities are offered at your school? ○ What grade levels do these cover? ● Do you sponsor/coach an extracurricular STEM club?

39 ○ What type of STEM club(s) do you coach? ○ Where does your club meet and how often? ○ How many students come to your meetings on average? ● Do any of the students in your club(s) have IEPs? Are you aware of and follow their accommodations? ○ If so, do you know what kind of disabilities they have? ● What kind of professional development have you had to work with students with disabilities (SWD) in an informal learning environment? ● How do you recruit students typically to your club(s)? ○ Do you recruit SWD in particular? ● Do you feel prepared to recruit and retain SWD in your club? ○ If not, what do you think you need? What would help you recruit and retain SWD in your club? General final questions ● Is there anything else we should know to help you recruit and retain SWD in your club? ● Is there anyone else we should contact to participate in our study? ● Do you have any questions for me/us? Don’t forget to thank them and give them the gift card or ask for how to send them the gift card.

40 Increasing Teacher Education Candidate Collaboration Knowledge and Skill Through Approximation of Practice Cathy L. Eschete, PhD Belmont University Renée E. Lastrapes, PhD University of Houston-Clear Lake Paul Mooney, PhD Louisiana State University Abstract This study utilized an approximation of practice framework to the teaching of collaboration skills in a teacher preparation program. Researchers questioned whether teacher candidate achievement and engagement would differ across six models of collaborative (co-) teaching instructional delivery systems. A single subject alternating treatments design was implemented to ascertain levels, variability, and trends of total engagement for the six models. Overall findings indicated that, first, engagement levels and content achievement of students were higher in co-teaching models that reduced the teacher to student ratio (station, parallel, and alternate teaching models). Second, station teaching and teaming were the most preferred models. Keywords: Approximation of Practice, Collaboration, Co-teaching, Outcomes, Teacher Education Increasing Teacher Education Candidate Collaboration Knowledge and Skill Through Approximation of Practice Teacher preparation as a whole has received a tremendous amount of criticism in recent years. Opponents of traditional teacher education have been concerned with the length and cost of such programs as well as a belief that they are not focused on what is perceived to be the critical component of developing teacher subject matter knowledge (Leko, Brownell, Sindelar, & Murphy, 2012). Contributing to the concern is a realization that research demonstrating a link between teacher training, teacher practice, and student outcomes has been difficult to locate and communicate clearly (Goe, 2006), thereby corroborating the concern regarding the length and cost associated with traditional teacher education programs. In special education teacher preparation, specifically, changes in how special education is conceptualized and organized over time have contributed to greater heterogeneity in preparation program structures. This expansion in program compositions and emphases prompted Brownell, Sindelar, Kiely, and Danielson (2010) to posit that there has been a loss of focus in teacher training programs. With these factors forming a backdrop, society’s need to systematically and scientifically strengthen teacher training continues to grow (Darling-Hammond, 2014). In the special education teacher preparation literature, of recent interest has been the emergence of collaboration as a target for inquiry. For purposes of this article, collaboration is defined as a

41 style of direct interpersonal interaction in which professionals who are considered equals voluntarily work toward shared goals (Friend & Cook, 2013). Leko and colleagues (2012) reviewed teacher and special education journals for research articles published from 2000 to 2011 in order to answer questions related to quality teacher preparation. One of their primary findings was that a growing number of research studies focused on two areas of inquiry, pedagogical content knowledge and collaboration. Regarding the knowledge and skills necessary for effective collaboration, Leko and colleagues (2012) indicated that some of the most promising strategies for developing collaborative skills involved direct teaching of collaboration skills strategies and providing opportunities for general and special education teachers to work together. Collaboration Emphasis in Teacher Preparation The recent emphasis on collaboration research that Leko et al. (2012) highlighted makes sense for a number of reasons. One is that collaboration is one of the seven major teacher preparation standards for initial special educators that have been developed by the Council for Exceptional Children (CEC; n.d.) and used as part of teacher candidate evaluation and teacher preparation accreditation. In its description of the collaboration standard, the CEC notes that a number of factors have converged to warrant a need for educator collaboration. Among those factors have been increasing diversity in student populations and increasing expectations for student success, including the academic performance of students with disabilities. For education professionals, the need for collaborative avenues continue to grow. For teachers, changing school organizational structures, whether it be the implementation of multi-tiered systems of support or TAP frameworks (National Institute for Excellence in Teaching)), requires a great deal of professional dialogue around curricular and behavioral issues. For administrators, there is a greater diversity in the types of observations of and feedback to teachers. For teacher educators, there is a greater need to, first, understand how to practice effectively in these new environments and, second, to teach candidates to manage—even thrive—amid such diversity. A second reason to highlight collaboration in teacher preparation relates to its inclusion as one of four critical areas of practice that define effective special education practice. In identifying a body of teaching “practices that are essential to effective teaching and fundamental to student learning” (McLeskey, Maheady, Billingsley, Brownell, & Lewis, 2019, p. vii), a CEC working group developed 22 high-leverage practices across the areas of collaboration, assessment, instruction, and social/emotional/behavioral development. Three collaboration practices focused on working effectively with professionals and parents to increase and support student learning and included leading effective meetings (McLeskey et al., 2019). A final reason to emphasize collaboration in teacher preparation relates to the rapid increase in the number of students with disabilities who are receiving the bulk of their instruction in the general education classroom. That is particularly true for students with high-incidence disabilities. McLeskey, Landers, Hoppey, and Williamson (2011) reported a 166% increase between the 1990–1991 and 2008–2009 school years in the number of students with learning disabilities who received 80% or more of their instruction in the general education classroom. For students with emotional disturbance, general education placement rates in that time span have also more than doubled (McLeskey, Landers, Williamson, & Hoppey, 2012). In 2011-

42 2012, 44.1% of students with emotional disturbance were taught primarily in the general classroom (U.S. Department of Education, 2014), with 40% of individual states reporting placement rates of 50% or higher (Jackson, Willis, Giles, Lastrapes, & Mooney, 2017). These changes in the makeup of general education classrooms require teachers to be aware not only of effective instructional practices for students with and without disabilities but also effective collaboration skills since it is likely that adult colleagues, be they special education teachers and/or paraprofessionals, are likely to share instructional responsibility for these students. Rationale for Continuing the Research on Collaboration in Teacher Preparation The Leko et al. (2012) review of special education teacher preparation literature indicated that the instructional approaches that held the most potential for positive impact were those that taught collaborative skills or a structure for working together. Features of effective teacher preparation included developing coursework that blended content knowledge with procedural or pedagogical knowledge, promoting active learning, aligning coursework and high-quality field experiences, and building opportunities for special and general education candidates to collaborate (Leko et al., 2012; Sindelar, Bishop, & Brownell, 2006). The present study builds on that base by evaluating the performance of teacher education candidates who were partnered together in a structure designed to promote collaborative instructional planning, implementation, and evaluation. The co-teaching model at the center of the present study has long been advocated as a logical way to facilitate inclusion of students with disabilities in general education classrooms (Friend, Cook, Hurley-Chamberlain, & Shamberger, 2010). An opportunity to extend the collaboration literature in special education teacher preparation arose out of a novel situation wherein teacher education candidates pursuing elementary/grades 1–5 and dual certification (elementary and special education/mild-moderate disabilities emphasis) at a southeastern United States research intensive university were enrolled together in a course designed to provide knowledge and skills in the areas of collaboration, consultation, and co-teaching practice. The study was designed to pair these two candidate groupings and require that each candidate plan, deliver, and evaluate two co-teaching lessons while also observing a series of their colleagues’ lessons within the class setting. Each lesson had a specific content to be taught, a particular co-teaching model to utilize, and a particular type of student with a highincidence disability to program for. Researchers videotaped the lessons for later determination of student engagement levels, developed and administered regular informal content assessments that were administered along with the co-taught lessons in order to evaluate student achievement, and surveyed candidates following the co-teaching to determine their perceptions related to coteaching practice and model preference. The present study aimed to develop the special education teacher preparation literature in a number of ways. First, the situation provided an opportunity to have students pursuing general and special education certifications learn and work together on course-related activities, a circumstance that has been both advocated for and supported in the literature (e.g., Hudson & Glomb, 1997; Leko et al., 2012). The study utilized the supportive setting of a university classroom to bring general and special education candidates together so that there could be a focus on pedagogical content knowledge and collaborative skill development. Focused on the practice of teaching, Grossman et al. (2009) described an approximation of practice model in

43 which novices are provided “greater freedom to experiment, falter, regroup, and reflect” (p. 2076). In theory, the model utilizes the university classroom setting rather than field-based settings to simulate aspects of practice in an environment that may be perceived by novices as friendlier than real classrooms. Such instructional settings also allow for greater use of and emphasis on pedagogical feedback, either individual comments targeted to a particular student or collective remarks aimed at groups of students. Feedback targets can be diverse and range from the steps in an instructional procedure to a novice’s thinking while practicing. Previous approximation of practice research had been undertaken with reading, math, and science subject matter. The present study extended approximation of practice research to collaboration knowledge and skills. Second, the study targeted a collaborative teaching methodology known as co-teaching. Cook and Friend (1995) have defined co-teaching as “two or more professionals delivering substantive instruction to a diverse, or blended, group of students in a single physical space” (p. 2). Similarly, Zigmond, Magiera, Simmons, and Volonino (2013) have defined co-teaching as two educators accommodating the needs of students with and without disabilities together. In special education over the past 20 years, legislative pressure has challenged educators to find efficient and effective ways to provide high-quality instruction including elements such as active engagement for students with disabilities within the general education setting (Solis, Vaughn, Swanson, & McCulley, 2012). In order to meet least restrictive environment requirements, many school districts have utilized inclusion settings for student placement (Friend et al., 2010). The co-teaching instructional routine has been advocated as a way to enhance instruction and facilitate learning for all students, but especially those with disabilities (Zigmond et al., 2013). What Friend and colleagues (2010) described as the complexity of implementation emerges, however, when the promotion of the practice is weighed against the fact that there is not a strong body of evidence supporting the practice’s positive impact on student outcomes (B. Cook, McDuffie-Landrum, Oshita, & Cook, 2011; Kauffman, Schumaker, Badar, & Hallenbeck, 2019; Murawski & Swanson, 2001). Moreover, it is likely difficult to uniformly implement coteaching models and/or practices given that there is very little guidance beyond narrative model descriptions in textbooks and the professional literature. In the present study, co-teaching teams collectively plan, implement, and evaluate co-taught lessons in a university classroom setting. It also provided an opportunity for teacher education candidates to provide input on their perceptions of co-teaching strengths and weaknesses as well as model preferences. Finally, from a scholarly impact standpoint, the study implementation provided teacher education program faculty at a research-intensive university the chance to meaningfully link teacher training, teacher practice, and teacher candidate outcomes, something Goe (2006) suggested is a rarity. Preservice teachers, in particular, were part of an experimental inquiry that allowed them to experience and evaluate different forms of an instructional service delivery model in a supported setting. Moreover, measures of their own engagement and learning were compared to help determine if different forms of the co-teaching service delivery model resulted in different engagement levels and achievement outcomes. In the present case, with transmission of content knowledge related to collaboration, co-teaching, and consultation being recognized as a critical component of the course, it was important to determine if any of the co-teaching models being practiced would prove to be a better

44 communicator of content for the participating candidates, at least as determined through scores of informal content assessments. Student grouping research in the special education field (e.g., Elbaum, Vaughn, Hughes, & Moody, 1999) has suggested that alternatives to whole grouping arrangements have produced positive effects on reading outcomes. The study was designed so that various models of co-teaching could be compared directly to each other to see if teacher candidate achievement differed by model. Further, by videotaping the candidate presentations created the ability to evaluate if there were differences in student engagement levels across co-teaching models. Engagement is an important instructional concept because of its association with student achievement (Parsons, Nuland, & Parsons, 2014). For the student, engagement in school plays a critical role in academic and life success (National Research Council and Institute of Medicine, 2004). For the teacher, increasing student engagement in academic instruction is vital to promoting positive student academic and behavioral outcomes (Hollo & Hirn, 2015). Engagement is also an instructional construct that can be considered an alterable variable (Bloom, 1980). That is, engagement practice can be controlled by the teacher through practices such as modeling desired academic/social behavior, providing opportunities to respond to curricular content, and/or providing academic/behavioral feedback (Harbour, Evanovich, Sweigart, & Hughes, 2015) and it can positively impact student achievement. By evaluating engagement levels and achievement across models, raised questions if definitive results might influence future special education teacher education instructional method course design and delivery. Given the preceding rationale, the following research questions were addressed: 1. Are there differences in patterns of engagement for teacher education candidates across six co-teaching models that are advanced in the professional literature? 2. Are there differences in academic outcomes for teacher education candidates as a result of implementation of six different co-teaching models? 3. After implementing and experiencing co-taught lessons in an approximation of practice framework, what is the social validity of the models for the teacher education candidates? 4. After implementing and experiencing co-taught lessons, what co-teaching model do teacher education candidates prefer? Method Setting and Participants The study utilized data that were collected during the spring semester of a teacher education course focused on collaborative practices for educators. The setting was a classroom that had seats for approximately 60 students on a university campus. All instructor presentations of content took place in the classroom. Each presentation was recorded. Consenting participants (N = 45) were generally female (n = 43). The majority of undergraduate students (n = 31) were grades 1–5 teacher candidates who were completing their final undergraduate semester and had forgone a one-semester student teaching experience in order to complete a fifth-year master’s program. None of these participants had taken any special education coursework beyond a single introduction to exceptionalities course. The remaining undergraduates (n = 9) were in the process of completing a dual certification, grades 1–5

45 program that would eventually allow them to be certified as general educators and special educators with mild-moderate (largely high-incidence) disabilities. All five graduate students were pursuing advanced coursework or certifications in special education. Design A single subject alternating treatments design was implemented to ascertain levels, variability, and trends of total (active and passive) engagement for the six models. An alternating treatments design is ideal to determine which of several interventions is the most effective. Additionally, the counterbalancing eliminates the threat of sequence effects (Alberto & Troutman, 2009). Independent Variable The independent variable in the present study was the presentation of one of the co-teaching models as described by Cook and Friend (2013). These are six unique models stated below, the first three can be considered large group, in that teaching is addressed to the class as a whole and the final three can be considered small group in that the class is divided into smaller groups. One teach/one assist. One teacher takes a clear lead in the delivery of instruction while the other circulates among the students observing and offering individual assistance as needed. One teach/one observe. One teacher leads the large-group instruction while the other gathers data on specific students or the class. The teacher observing may be seated strategically or moving about the classroom, so that he or she may observe a specific student or the class as a whole. Teaming. Both teachers co-lead large-group instruction. For example, you may see both teachers standing side by side alternating whole group discussion, or one may speak while the other demonstrates or models a concept or takes notes on a projection instrument. They also may model and role play, asking and responding to questions. Station teaching. Instruction is typically divided into several non-sequential parts, with students arranged in several groups. Students or teachers rotate from station to station. Two of the stations are traditionally teacher-led stations, and at the third station students work alone or with peers on a related project or assignment. The two teachers prepare and teach different material; however, instruction is repeated to each group as they circulate so that each group has an opportunity to hear the material that each teacher is presenting. Alternative teaching. One teacher provides instruction to most students (e.g., ¾ of the class) while the other works in a separate location (e.g., back of the classroom) with a small group for possible remediation, enrichment, assessment, preteaching, or another designated purpose. Both are delivering instruction simultaneously, but the instruction is different. Parallel teaching. Two teachers each take half the class and present the same or different material simultaneously (Murawski, 2010), for the primary purpose of instructional differentiation and increasing student participation by lowering the student to teacher ratio. Dependent Variables

46 Three dependent variables were utilized in the present study. The first was participants’ mean level of total (active and passive) engagement. Engagement was determined through use of the Behavioral Observation of Students in Schools (BOSS; Shapiro, 2004) behavioral assessment system. Engaged time was defined as those times when the student was actively attending to the assigned work (i.e., writing, reading aloud, raising hand, talking to the teacher about the assigned material, talking to a peer about the assigned material, looking up a word in a dictionary) or when the student was passively attending to assigned work (i.e., listening to a lecture, looking at the blackboard during teacher instruction, listening to a peer respond to a question) (Shapiro, 2004). The second dependent variable was quantitative and academic in nature and targeted gain scores for pre- to post-test on repeated multiple-choice quizzes that participants completed for each week of co-teaching model implementation (N = 5). Each quiz consisted of 30 items and included 5 questions per unit of content for each of the 6 lessons taught in a given week. Questions were developed by the first two authors who were the instructors of record for the course. There were no technical adequacy data available for the measures. Social validity was the third dependent variable which consisted of informal participant written responses to requests for co-teaching model preferences and perceptions. Data were collected on two occasions, with preference data collected from an examination question on the course final and perception data (relative to model strengths/benefits and weaknesses) collected from a postexperiment anonymous survey. Collectively, participants were asked to identify the co-teaching model they preferred and list strengths/benefits and weaknesses to co-teaching. Descriptive statistics were calculated related to the preference question and reported for the total population as well as for general education and special education sub-populations. Themes were identified relative to the strengths and weaknesses questions via a review and summarization of survey data. Fidelity of Implementation Fidelity data were collected for the first dependent measure utilizing a direct observation checklist and a second observer for 30% of the lessons implemented (n = 9). The checklist assessed the utilization of the characteristics of each of the co-teaching models as defined by Friend and Cook (2013). Agreement for the models was calculated by using an interval agreement formula. An agreement was scored when both observers indicated on their respective checklist that the characteristic was present in the video. Interobserver Agreement Interobserver agreement (IOA) procedures were administered across dependent variables. For engagement level, a second observer collected IOA data on the same nine lessons for which implementation fidelity data were collected. Agreement for engagement was calculated using an interval agreement formula in which an agreement was scored when both observers recorded engaged behavior during an interval. The number of agreements was divided by the number of agreements plus disagreements and multiplied by 100 (Alberto & Troutman, 2009). For achievement gain scores, independent observers scored each participant pre- and post-test score weekly.

47 Procedures As part of the course requirements, participants taught co-teaching lessons to a class of their peers utilizing the course curriculum. All lessons were videotaped by a third party, who was also one of the graduate students who graded the weekly pre- and post-tests. The camera was placed so that the non-teaching candidates could be seen and have their engagement assessed by the independent observers. Six co-teaching models were completed each week, with each participant engaging in two lesson implementations over the course of five weeks. Each group consisted of a minimum of two preservice teachers and was randomly assigned to a co-teaching model for each of their co-teaching lessons. The content of their lessons consisted of assigned course material from the course textbook, Interactions: Collaboration Skills for School Professionals (Friend & Cook, 2013). Each lesson was independent of the previous week; although it did follow the recommended sequence of the textbook. While all six co-teaching models were taught in one class period, the order of model implementation was randomly assigned. Candidates were notified which model was being utilized at the beginning of each lesson. Lesson lengths ranged from 20 to 30 minutes. Data Analysis Researchers used visual analysis to evaluate engagement level, trend, and variability of the data paths within and across the five weeks of co-teaching lessons. Visual analysis focused on each of the six models individually. Analysis was also conducted of the large group (one teach/one assist; one teach/one observe; teaming) and small group (station teaching; parallel teaching; alternative teaching) models. To evaluate the effect of the co-teaching method on achievement the pretest score was subtracted from the posttest score, to create 5 “gain scores” (for the 5 weeks of instruction) which were then analyzed with a repeated measures analysis of variance. Gain scores were chosen over analysis of covariance because a change in group means best addressed the research focus on the effect of co-teaching method (Fitzmaurice, Laird, & Ware, 2004). Descriptive statistics were computed for each teaching method. To evaluate social validity of the models, each student’s written response were read and themes were identified regarding engagement and efficacy. According to Ryan and Bernard (2003), themes come both from the data and from the investigator’s prior theoretical understanding of the material. All of the students’ responses were grouped into categories based on the similarity of their responses to others. For co-teaching model preference, descriptive statistics were calculated on the students’ responses to their preference of the six co-teaching models taken from a question on the final exam. Results Treatment Fidelity and Interobserver Agreement A direct observation checklist was used for implementation fidelity for the six co-teaching models. Direct observations were completed for each of the co-teaching lessons to determine the total percentage of characteristics included. The overall mean percentage of characteristics included was 95% (range 85%-100%) in all co-teaching lessons. A second observer viewed 30%

48 of the lessons. The mean percentage of interobserver reliability of the utilization of the characteristics of each of the co-teaching models was 98% (range 90%-100%). A direct observation measure was also utilized to evaluate the participant’s level of engagement during each of the co-teaching lessons. Direct observations were completed utilizing momentary time sampling in 15-second intervals for the length of each lesson. A second observer provided interobserver reliability for 30% of the lessons. The mean percentage of interobserver reliability of the dependent variable was 93% (range 89%–97%). Engagement Across Co-teaching Models Results are presented comparing the levels of student engagement across six co-teaching models for students enrolled in a teacher education course (see Table 1 and Figure 1). Visual analysis was used to evaluate level, trend, and variability of the engagement data paths within and across the five weeks of co-teaching lesson implementation. Engagement rates during one teach/one observe varied from 77% to 82%, with a mean of 79.2% across 5 lessons. Data were stable. The level of the data on the y-axis fell into a moderate to high level (Cooper, Heron, & Heward, 2007). The trend line for one teach/one observe when analyzed using a split middle trend yielded no trend. Table 1. Mean Percentage Engagement by Co-teaching Model Model

Average

Alternative

One Teach/One Assist

One Teach/One Observe

Parallel

100

Station

100

Teaming

Engagement rates during teaming ranged from 78% to 86% (M = 81.8%). There was little variability in the data, with a high level and a slightly upward trend. Engagement during parallel teaching varied from 89% to 100% (M = 94%). There was little variability, with a high level and a slightly upward trend. Engagement during station teaching varied from 90% to 100% (M = 95.6%), with little variability, a high level, and a slightly downward trend. Engagement during one teach/one assist ranged from 65% to 85% (M = 76.8%). There was variability in the data. Level was moderate to high. The trend was downward. Engagement during alternative teaching varied from 76% to 97% (M = 87.2%). There was also variability in these data, a high level, and a slightly downward trend.

49 100 95 90 85 80 75 70 65 60 55 50 0

One Teach/One Assist

One Teach/One Observe

Teaming

Alternative

Parallel

Station

Figure 1. Levels of engagement for each of the co-teaching models over the course of the 5 weeks. Figure 2 shows the mean engagement rate of students during co-teaching models that reduced the presentation student-teacher ratio, called here “Small Group” (i.e., station, parallel, and alternative), and those that did not, referred to as “Large Group” (i.e., one teach/one assist, one teach/one observe, teaming). Engagement when the ratio of student to teacher was reduced varied from 88% to 95% (M = 92%), with 78% to 80% rates (M = 79%) for the larger groupings. The data were stable. Visual inspection of the data indicated that the small group instruction consistently produced higher levels of engagement. 100 95 90 85 80 75 70 65 60 55 50 1

3 Large Group

Small Group

Figure 2. Levels of engagement when collapsed across large group (one teach/one assist, one teach/one observe, and teaming) and small group (parallel, station, and alternative) Mean Differences in Outcomes across Co-teaching Models and Groupings

50 Table 2 presents gain scores collected for each of the co-teaching models as well as the smallversus large-group combinations. A repeated measures ANOVA with a between subjects factor (co-teaching model) was conducted to compare the effect of co-teaching type on student achievement on a subject matter test given to students before and after being taught with one of six co-teaching models. There was a significant difference among the time periods, Wilks Lambda = .89, F(4, 133) = 4.27, p = .003, partial eta squared = .14 as well as for the interaction of time and co-teaching model, Wilks Lambda = .54, F(20, 544) = 4.49, p = <.001 partial eta squared = .14. Mauchly’s test of sphericity was not significant, indicating that there was homogeneity of variances. Post hoc tests were conducted using the Tukey method and only the station teaching method was significantly different than the other co-teaching methods at all time periods. Tests of within subject effects indicated a significant interaction between time and method F(20, 544) = 5.35, p = <.001, partial eta squared = .16. Tests of within subject polynomial contrasts indicated that the interaction between time and method were not linear, but rather cubic, F(5, 136) = 4.42, p = .003, partial eta squared = .14. The results suggested that there was a difference between station teaching and other types of co-teaching models, but with the interaction it depended upon what time the students were tested. Table 2 Gain Scores for Achievement by Co-teaching Model Individually and Collectively Model One Teach/One Assist

One Teach/One Observe

Teaming

Station

Parallel

.57

.19

.14

.00

.90

1.12

1.16

.35

1.26

1.81

.92

.36

.08

.04

.24

1.15

1.38

.57

1.30

1.05

.29

.43

.38

.48

.38

1.05

.92

1.20

.87

1.43

.68

2.88

1.80

.72

.16

.74

1.39

1.95

1.51

.98

.28

.40

.00

.08

.72

1.13

.81

.91

1.32

.93

51 Parallel

Small Group

Large Group

.28

.40

.00

.08

.72

1.13

.81

.91

1.32

.93

.69

1.34

.54

.50

.54

1.10

1.58

1.60

1.30

.93

.61

.32

.19

.16

.49

1.12

1.17

.78

1.17

1.44

For the comparison of small and large groupings, a repeated measures ANOVA with a between subjects factor (small vs. large group) was conducted. There was a significant difference among the groups, Wilks Lambda = .89, F(4, 137) = 4.39, p = .002, partial eta squared = .14, as well as for the interaction of time and small vs. large group models, Wilks Lambda = .92, F(4, 137) = 3.08, p = .02, partial eta squared = .08. Mauchly’s test of sphericity was significant, indicating that there was heterogeneity of variances. The Huyhn-Feldt correction for violations of sphericity was used. With only two groups, no post hoc tests were conducted. Examining the means and standard deviations in Table 2, it was evident that the small group showed larger gains at all time periods. Tests of within subject effects indicated a significant interaction between time and method F(3.85, 560) = 3.40, p = .01, partial eta squared = .03 (Huyhn-Feldt). Tests of within subject polynomial contrasts for small vs. large group also indicated that the interaction between time and model were cubic rather than linear F(1, 140) = 4.35, p = .038, partial eta squared = .03. These results suggested that there was a difference between coteaching models that utilized large and small groupings. Students receiving their instruction in small groups showed stronger gains than those taught in larger groups. Due to the significant interaction between time and model, however, differences depended on the time that students were tested. Student Perceptions of the Co-teaching Models Results are presented showing students’ preference of the co-teaching models following having taught and/or observed all six models on five occasions. Table 3 shows the number of coteaching model preferences for the general and special education teacher participants. Overall, the most preferred co-teaching model was teaming, followed closely by station teaching. One teach/one observe showed the lowest preference and was not chosen at all as a preferred model. Teaming and station teaching were also the most chosen method for general education teacher candidates, whereas special education teacher candidates and teachers preferred alternative and parallel teaching in addition to station teaching. Table 3 summarizes participant perceptions of co-teaching strengths and weaknesses following direct involvement in co-teaching implementation. Commonly identified benefits included increases in individual attention provided students, smaller group sizes, having someone to collaborate with, ability to cover more material, and utilization of a variety of methods during teaching. Several common themes noted in the participants’ identification of strengths and benefits of co-teaching related closely to station teaching as well. Common weaknesses

52 identified during co-teaching were noise level, unequal distribution of tasks, and time management. Noise level frustrations were commonly cited. One participant stated, “I like station teaching, but the noise level is very distracting.” Another participant said, “In parallel and station teaching, the teaching that was going on in the other part of the classroom was very distracting. Maybe one group could go to another class.” Table 3. Candidate Identified Strengths and Weaknesses Regarding Co-teaching Strengths Weaknesses Increases in individual student attention (N = 10)

Increased noise level (N = 20)

Smaller group sizes (N = 7)

Unequal distribution of tasks (N = 6)

Having someone to collaborate with (N = 13)

Time management (N = 9)

Ability to cover more material (N = 3) Utilization of a variety of teaching methods (N = 14) Discussion Collaboration has become a common target of inquiry in special education teacher preparation, with the desire for research to inform practice given the steady increases in students with disabilities served in general education classrooms and concomitant needs for cooperation and meaningful collective action by teaching professionals who are voluntarily or involuntarily thrust together in service to this population. The present inquiry evaluated differences in achievement and engagement levels for participating teacher candidates involved in a supported teacher education course targeting collaboration content. The outcomes warrant teacher educator consideration of the manner in which collaboration methodologies are presented and practiced. In the present study, when grouped according to presentation format, both engagement levels and academic achievement were stronger when candidates utilized smaller groupings. In other words, student engagement and achievement was better when the student-teacher ratio was reduced. Overall, visual analysis of engagement rates for each individual co-teaching model indicated generally high student engagement across the board for participants in a teacher education course. Across 30 lessons, the median level of engagement was 85%, with nearly twothirds of the observations (n = 19) above 80% and more than one-third (40%; n = 12) at or above 90% (see Table 1). Yet engagement rates were most distinct (see Figure 2) when the six models were grouped in terms of smallversus large-group format. Station teaching, parallel teaching,

53 and alternative teaching, which each reduced the student-to-teacher ratio during delivery of instruction, all evidenced higher levels of engagement. Present findings extended the teacher education collaboration literature that Leko et al. (2012) previously summarized. One of the Leko et al. findings was that successful outcomes were increased when collaborative skills or structures were taught to teacher education candidates or when general and special education candidates were placed together. In the present study, the collaboration knowledge base and skills were not only taught, but implemented as participants engaged in the implementation of co-teaching models. In other words, candidates were learning collaboration skills and applying that learning at the same time. The present findings also demonstrated that calls to provide training for participants in coteaching relationships (e.g., Cook et al., 2011; Hudson & Glomb, 1997) can be addressed within the teacher preparation context. That is, there was a demonstration that courses can effectively and efficiently bring general and special education teacher candidates together to build content and skill knowledge in a supportive setting. Second, coursework can be delivered in such a way as to demonstrate a direct link between teacher training, teacher practice, and candidate outcomes (Goe, 2006). The high levels of engagement indicated in the present findings were also noteworthy when viewed with engagement rates for public school classrooms. Findings were comparable to those of a large-scale systematic direct observation of public school teachers’ instructional behaviors. Scott, Hirn, and Alter (2014) reported overall engagement rates across 1,197 observations of 85%, with 95% reported for elementary grades observations and 82% for the more common middle/high school observations. These findings compare to our median of 85% engagement. Hollo and Hirn (2015) and Sweigart and Landrum (2015) also analyzed the observation data within grouping and co-teaching contexts, respectively. A major finding of Hollo and Hirn was that while whole-group instruction was the most frequently used format across elementary, middle, and high school settings, student engagement was greatest during small-group lessons, whether teacher- or peer-mediated. Sweigart and Landrum (2015) compared observations in which one adult versus more than one adult was involved, which may or may not have been coteaching depending upon who the adults in the multi-person grouping were. They also distinguished between elementary and middle/high school settings. Their findings indicated that, for elementary settings, while opportunity to respond, positive feedback, and negative feedback rates for students and small-group and one-to-one grouping percentages were higher for multiadult arrangements, the active student engagement percentages were higher in one adult settings. Combined, outcomes seem to suggest that instructional groupings that reduce the student-toteacher ratio were associated with higher student engagement. In the present study, differential student achievement was evidenced across co-teaching models, with outcomes better in models with smaller student-teacher ratios (i.e., station teaching, parallel teaching, alternative teaching). The pattern of findings was similar to that reported for engagement level. The literature is replete with calls for research targeting student outcomes in co-teaching. These results are believed to be the first direct comparison of achievement levels across co-teaching models.

A third interesting finding was that there were similarities and differences in preferences for participants pursuing general or special education certification routes. The present findings extended co-teaching perception literature to candidates. As hypothesized, teaming was the most preferred model for all participants, garnering 36.2% of the choices overall. However, differences in choice were evident by teacher type, with 44.1% of general educators but only 15.4% of special educators selecting teaming. One similar preference in co-teaching model across the co-teaching groups was station teaching, which was the second choice of each group and the second most preferred overall. Overall, special educators favored the smaller-group formats, with more than three-quarters of the selections (i.e., 77%; n = 10 of 13; see Table 1) targeting alternative, parallel, and station teaching. Interestingly, only 3 of 47 selections were for one teach/one assist, which research (e.g., Scruggs, Mastropieri, & McDuffie, 2007) has indicated is likely the most commonly implemented model in school-based settings. Limitations Study limitations were related to participants, setting, sample type, and implementation fidelity. First, participants in this study consisted largely of preservice teachers in a postsecondary setting. Therefore, study findings may not generalize beyond the postsecondary setting to a typical K–12 classroom where co-teaching is more frequently utilized. It is also limited in its ability to generalize to certified teachers and K–12 populations, due to the fact that the “teachers” that were co-teaching were preservice teachers and the students participating in the co-taught lesson were also preservice teachers. The population typically providing co-taught lessons is certified general and special education teachers, and the participants in co-taught lessons are generally K– 12 students. Second, in determining implementation fidelity for the six co-teaching models, observers were aware of the model type prior to watching the video for each of lesson presentations. The model name was stated at the beginning of each video, which could have resulted in bias. Without this prompt, the implementation fidelity findings may not have been as strong as was reported here. Implications With study findings and limitations identified, one major implication that spans the fields of teacher education and public school education should be emphasized. That is, it makes sense to continue to bring together professional efforts to strengthen the link between teacher training, teacher practice, and student outcomes (Leko et al., 2012) around the instructional routine known as co-teaching. In special education teacher training particularly, strengthening the link between teacher training, teacher practice, and outcomes—both candidate and public school student— might allow the profession to regain the meaningful focus that Brownell et al. (2010) suggest that has been lost. Some of that attention must be directed toward fostering instructional collaborative and inclusive practice that positively impacts achievement for students with highincidence disabilities. Building upon the approximation of practice (Grossman et al., 2009) may be a valid and valued way forward. Grossman (2011) described approximation of practice as an opportunity for candidates to engage in practice that is related and likely easier than that of practicing professionals. Approximations can target a single teaching behavior or collection of behavior, allowing the teacher educator to

55 simplify and/or shape the teaching demands. Approximations also foster use of targeted and global feedback during and/or after candidate simulations. Such interactions might even involve pausing candidate lesson presentations, providing specific feedback, and allowing candidates to redo the lesson or skill, thereby differentiating the learning experience from that engaged in field experiences or student teaching (Grossman, 2011). The approximation of practice process seems especially appropriate for teaching collaboration skills and, specifically, co-teaching models and practices. In the present research, informed by research on effective practice, special and general education teacher candidates interacted together, collectively planning, delivering, and evaluating lessons around content focused on effective collaboration. Candidates engaged in preliminary discussions related to developing measurable objectives, selecting appropriate ways to teach content and then assess learning, and differentiating instruction for a student with a high-incidence disability who might be included in the population of students in a general classroom setting. Candidates not only delivered lessons but served as students as colleagues delivered their own lessons. In this way, teaching and observing were part of the pedagogy. Once a series of lessons was complete, candidates also received individual group written and collective class oral feedback from the lead instructor. Such a context, then, allowed for active learning by teacher candidates as well as supported practice. Outcome data collected as a result of its implementation were promising, indicating that, one, learning took place, and, two, that smaller student grouping arrangements might be facilitators of student achievement. This focus on student outcomes and student outcome gain more specifically is vitally important in the context of scientific inquiry around co-teaching. As it stands today, advocacy for the instructional delivery approach continues to outweigh empirical support for the practice as an effective means to provide specially designed instruction for students with disabilities. Collective focus on instructional practice and meaningful student outcome gain will help ensure that educational stakeholders emphasize appropriate special education programming before placement (Kauffman et al., 2019). Moving forward, more systematic research may inform special education teacher preparation, collaboration, and co-teaching practice. First, the present research might be replicated and then expanded into a larger teacher education framework. The approximation of practice structure in a content methods course could be linked to the subsequent student teaching framework in which the circumstances are more closely aligned with real-world practice. Achievement, engagement, and other relevant alterable instructional variables can be assessed to determine if growth noted in supported settings can be continued in more realistic environs. Moreover, more content areas, including reading, literacy, and math methods courses could also be included in approximation of practice arrangements so that a greater curricular coherence exists in teacher preparation programs and so that a practice-based focus in teacher preparation is emphasized. For collaboration researchers, the opportunity to expand a practice-based focus exists. Inquiry can target various elements of the collaboration concept as well as applications of the collaboration practice across developmental levels of public school students, academic achievement level of public school students, and public school content. The present study, for example, collected data on the candidates as learners. Subsequent research can target the practice of candidate teachers and evaluate their abilities to appropriately model, increase and improve opportunities to respond, and provide affirmative or corrective feedback. For co-teaching

56 researchers, the focus might also be on ways to best demonstrate teaching skills associated with student engagement as well as ways to best implement the various co-teaching models. One research question might address how to implement small grouping (station, parallel, and/or alternative teaching) arrangements that are not so bothersome to teachers, since candidates, despite liking the models, expressed frustration with the noise levels of the smaller groupings. Conclusion The present study utilized an approximation of practice structure to enable teacher education candidates to learn about and experience collaborative skills in supported settings. Results of an experimental comparison of teacher candidate implementation of small- and large-group coteaching presentation models indicated that the smaller-group models evidenced greater achievement and engagement levels. Teacher candidate preferences for co-teaching models differed by certification type, with future general educators preferring teaming and special educators preferring the smaller group models. References Alberto, P. A. & Troutman, A. C. (2009). Applied behavior analysis for teachers (9th ed.). New Jersey: Pearson. Brownell, M. T., Sindelar, P. T., Kiely, M. T., & Danielson, L. C. (2010). Special education teacher quality and preparation: Exposing foundations, constructing a new model. Exceptional Children, 76(3), 357–377. Coates, H. (2007). A model of online and general campus-based student engagement. Assessment and Evaluation in Higher Education, 32(2), 121–141. Cook, B. G., McDuffie-Landrum, K. A., Oshita, L., & Cook, S. C. (2011). Co-teaching and students with disabilities: A critical analysis of the empirical literature. In Hallahan, D. P. & Kauffman, J. K. (Eds.), The Handbook of Special Education (pp. 147–159). New York: Routledge. Cook, L. & Friend, M. (1995). Co-teaching: Guidelines for creating effective practices. Focus on Exceptional Children, 28(3), 1–16. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. New Jersey: Pearson. Darling-Hammond, L. (2014). Strengthening clinical preparation: The holy grail of teacher education. Peabody Journal of Education, 89, 547–561. doi: 10.1080/0161956X.2014.939009 Elbaum, B., Vaughn, S., Hughes, M., & Moody, S. W. (1999). Grouping practices and reading outcomes for students with disabilities. Exceptional Children, 65, 399–415. Fitzmaurice, G. M.. Laird, N. M., & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley. Fredericks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59–109. Friend, M. & Cook, L. (2013). Interactions: Collaborative skills for school professionals. (7th ed). New Jersey: Pearson.

57 Friend, M., Cook, L., Hurley-Chamberlain, D., & Shamberger, C. (2010). Co-teaching: An illustration of the complexity of collaboration in special education. Journal of Educational and Psychological Consultation, 20(1), 9–27. Goe, L. (2006). The teacher preparation, teacher practices, student outcomes relationship in special education: Missing links and next steps. A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality. Grossman, P. (2011). Framework for teaching practice: A brief history of an idea. Teachers College Record, 113, 2836–2843. Grossman, P., Compton, C., Igra, D., Ronfeldt, M., Shahan, E., &Williamson, P. W. (2009). Teaching practice: A cross-professional perspective. Teachers College Record, 111, 2055–2100. Harbour, K. E., Evanovich, L. L., Sweigart, C. A., & Hughes, L. E. (2015). A brief review of effective teaching practices that maximize student engagement. Preventing School Failure, 59(1), 5–13. Hollo, A., & Hirn, R. G. (2015). Teacher and student behaviors in the context of grade level and instructional groupings. Preventing School Failure, 59(1), 30–39. Hudson, P. & Glomb, N. (1997). If it takes two to tango, then why not teach both partners to dance? Collaboration instruction for all educators. Journal of Learning Disabilities, 30, 442–448. Jackson, K. M., Willis, K., Giles, L., Lastrapes, R. E., & Mooney, P. (2017). How to meaningfully incorporate co-teaching into programs for middle school students with emotional and behavioral disorders. Beyond Behavior. 26(1), 11-18.doi: 10.1177/1074295617694408 Kauffman, J. M., Schumaker, J. B., Badar, J., & Hallenbeck, B. A. (2019). Where special education goes to die. Exceptionality, 27, 149-166. doi: 10.1080/09362835.2017.1414699 Leko, M. M., Brownell, M. T., Sindelar, P. T., & Murphy, K. (2012). Promoting special education preservice teacher expertise. Focus on Exceptional Children, 44(7), 1–16. McLeskey, J., Landers, E., Williamson, P., & Hoppey, D. (2012). Are we moving toward educating students with disabilities in less restrictive settings? The Journal of Special Education, 46, 131–140. doi: 10.1177/0022466910376670 McLeskey, J., Maheady, L., Billingsley, B., Brownell, M. T., & Lewis, T. J. (2019). High leverage practices for inclusive classrooms. New York: Routledge. Murawski, W. W. & Swanson, H. L. (2001). A meta-analysis of co-teaching research: where are the data? Remedial and Special Education, 22(5), 258–267. National Institute for Excellence in Teaching. (2020). The TAP system for teacher and student advancement. NIET. http://www.niet.org/tap-system/elements-of-success. National Research Council and Institute of Medicine. (2004). Engaging schools: Fostering high school students’ motivation to learn. Washington, DC: National Academies Press. Parsons, S. A., Nuland, L. R., & Parsons, A. W. (2014). The ABCs of student engagement. Kappan Magazine, 95(8), 23–27. Ryan, G. W. & Bernard, H. R. (2003). Techniques to identify themes. Field Methods, 15(1), 85– 109. Scantlebury, K., Gallo-Fox, J., & Wassell, B. (2006). Coteaching as a model for preservice secondary science teacher education. Teaching and Teacher Education, 24, 967–981. Scott, T., Hirn, R., & Alter, P. (2014). Teacher instruction as a predictor for student engagement and disruptive behaviors. Preventing School Failure, 58, 193–200.

58 Scruggs, T. E., Mastropieri, M. A., & McDuffie, K. A., (2007). Co-teaching in inclusive classrooms: A metasynthesis of qualitative research. Exceptional Children, 73, 203–222. Shapiro, E. S. (2004). Academic skills problems workbook (rev.). New York: The Guilford Press. Solis, M., Vaughn, S., Swanson, E., & McCulley, L. (2012). Collaborative models of instruction: The empirical foundations of inclusion and coteaching. Psychology in the Schools, 49(5), 498–510. Sweigart, C. A. & Landrum, T. J. (2015). The impact of number of adults on instruction: implications for co-teaching. Preventing School Failure, 59(1), 22–29. Wagner, M., Newman, L., Cameto, R., Garza, N., & Levine, P. (2005). After high school: A first look at the postschool experiences of youth with disabilities. A report from the National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International. Zigmond, N., Magiera, K., Simmons, R., & Volonino, V. (2013). Strategies for improving student outcomes in co-taught general education classrooms. In B. G. Cook & M. Tankersly. Research-based practices in special education (p. 120). Upper Saddle, NJ: Pearson. About the Authors Cathy L. Eschete, PhD is an Assistant Professor, Director of Clinical Practice, and CAEP Coordinator at Belmont University in the School of Education where she teaches undergraduate and graduate courses. Prior to her work at Belmont, she has worked in the field of education for the last 15 years. She was an Assistant Professor and Director of Field Experience at Louisiana College, worked in Special Education as a Clinical Supervisor in Northern California and teacher in South Louisiana. Her research interests include classroom and behavior management, multicultural education, and student engagement. Renee Lastrapes, PhD is an Assistant Professor of Educational Research and Assessment at the University of Houston-Clear Lake where she teaches graduate courses in research design and statistics, and supervises graduate student research for doctoral dissertations. Prior to her work at UHCL she was a special education teacher in public schools for 16 years in Northern California and Louisiana. Her research interests include research methodologies as well as behavioral supports for students with emotional and behavioral disorders. Paul Mooney, PhD is a Professor in the School of Education at Louisiana State University. His present research interests focus on structured formative assessment and academic interventions for students with or at risk for academic or behavioral disabilities.

59 Preparing Preservice Special Educators for Culturally and Linguistically Diverse Classrooms: A Systematic Review of Teacher Preparation Programs Krystal Lewis-Pratl, Doctoral Candidate Illinois State University Yojanna Cuenca-Carlino, PhD. Illinois State University April Mustian, PhD. Winthrop University Abstract Public classrooms across the United States have changed significantly over the last four decades. The percentage of culturally and linguistically diverse (CLD) learners continues to increase, while White females remain the stable majority of the teaching force (Sobel, Gutierrez, Zion, & Blanchett, 2011). As a result of this increased cultural gap, and with continued disparities that affect CLD learners with disabilities, teacher education programs (TEPs) of special educators are evaluating their programs and seeking input from stakeholders to gain a better understanding of the core competencies needed in order to become culturally responsive teachers. We conducted a literature review to examine how TEPs across the nation are preparing preservice special education teachers for CLD classrooms and which type(s) of professional development (PD) university faculty themselves underwent as part of the curricular redesign process to increase their own culturally responsive pedagogy (CRP). Our search yielded nine articles. The main findings were coded by three primary processes: (a) how faculty underwent professional development, (b) the program redesign process, and (c) outcome data of faculty PD or the impact of the redesign. We offer some recommendations for future research and discuss implications for practice. Keywords: teacher education programs or preparation, special education, pre-service, culturally and linguistically diverse, multicultural, culturally responsive or relevant pedagogy Preparing Preservice Special Educators for Culturally and Linguistically Diverse Classrooms: A Systematic Review of Teacher Preparation Programs The landscape of today’s classrooms reflects the myriad of cultures and languages present in our country. Students who are CLD are defined by the U.S. Department of Education (U.S. DOE) as either non-English proficient or limited-English proficient. The definition also includes students from diverse social, cultural, and economic backgrounds and/or homes where English is not the primary language (Terry & Irving, 2010). For the purposes of this paper, we define CLD in a more inclusive way that also includes students from diverse racial and ethnic backgrounds where English is the primary language. The National Center for Education Statistics (NCES) prediction trends from two decades ago suggests that, by 2024, the Hispanic and Asian/Pacific Islander student population will increase (18% to 29% and 4% to 6% respectively); while the White and Black student population will decrease (59% to 46%, and 17% to 15%, respectively, (U.S. DOE,

60 2016). By 2020, it is estimated that half of public-school students will come from non-English speaking backgrounds (Snyder & Dillow, 2015). In contrast, the demographics of teachers in our country do not reflect the growing diversity that exists within our classrooms. While we have seen increases in the CLD student population over the last two decades, changes in the diversity of the teaching workforce have been slow. In 1988, 87% of the teaching workforce was White; yet, three decades later, 82% percent of the teachers in our schools are White, with only 18% of the teachers being from CLD backgrounds (U.S. DOE, 2016). The majority of preservice and inservice teachers are White females from suburban or rural settings that may have little knowledge of learners with CLD backgrounds (Imler, 2009; Kahn, Lindstrom, & Murray, 2014; Taylor, 2010; Trent, Kea, & Oh, 2008). Additionally, educators bring their own set of values and experiences into the shared environment of the classroom. However, if the values and experiences that teachers bring into the classroom differ from their learners, and teachers have a limited understanding of languages and cultures other than their own, there is a potential for cultural dissonance (Kahn et al., 2014; Williams Shealey, 2006) that can lead to inequitable opportunities for CLD learners. The Case for Culturally Responsive Practice (CRP) Expanding core curriculum to include CRP as part of teacher preparation is needed for all teacher candidates; however, it is especially paramount for teacher candidates working with learners with disabilities who are also CLD. In 2007, the National Education Association (NEA) and the National Association of School Psychologists (NASP) both reported that there is a disproportionate representation of CLD learners in special education, and this has been a concern for nearly four decades (Cyr, McDiarmid, Halpin, Stratton, & Davis-Delano, 2012). Determining causality of this disproportionality is a complex issue. Skiba and colleagues (2008) have found that cultural dissonance does have a role in the overrepresentation of minority learners in special education, especially in those disability categories where eligibility criteria are more subjective such as cognitive impairment, emotional disability, and learning disabilities. Overall, both general and special education teachers have been ill-equipped to teach CLD learners, which has resulted in over-referrals and over-identification of CLD learners in special education (Skiba et al., 2008). Additionally, Trent and colleagues (2008) discuss conceptual confusion that can be likened to cultural dissonance as a barrier that must be addressed in the field by focusing on issues of power equity and social justice as well as de-emphasizing a deficit perspective and expanding our idea of culture to move beyond race. The data on disproportionality in special education support the need to train all preservice educators to be culturally responsive in their practices as an effort to reduce the number of referrals of CLD learners in special education programs. For instance, it is imperative that all teachers and staff understand the cultural nuances of behavior so that CLD learners do not continue to be disparately disciplined for behaviors that do not align to dominant school norms (Skiba et al., 2008; Taylor, 2010). Additionally, it is vital that special educators understand the difference between cultural and linguistic needs versus those that are characteristic(s) of a disability so that they can make more equitable and educated decisions (Hoover, 2012; Skiba et al., 2008). When preservice teachers enter the field without the familiarity of how to best support language acquisition of learners whose home language may differ from English, they may not be

61 adequately prepared to meet their needs (Klingner & Eppolito, 2014). If preservice and inservice teachers are not sufficiently trained to understand cultural differences, there may be missed opportunities, lack of appropriate instruction, and adoption of harmful teaching practices. Klingner and Eppolito (2014) explain that, even if a teacher is using instructional materials that are evidenced-based, it is likely that these interventions have not been replicated with learners from CLD backgrounds. If expected progress is not made, it can lead to a deficit view of the learner versus a reflection that the materials were even appropriate to begin with in terms of being responsive pedagogy to the learners served. Additionally, there has been tension surrounding whose responsibility it is to prepare preservice teachers to be culturally responsive practitioners. Some institutions have separate bilingual and English learners (EL) endorsements or certificates that are add-on credentials to a degree program. This often compartmentalizes programs and is the opposite of a shared model that focuses on the whole learner (Baca, Baca, & de Valenzuela, 2004). Baca and colleagues (2004) suggest merging bilingual education and special education to limit the push and pull that teachers are feeling as it relates to the best practice for working with CLD learners; however, these programs are still limited in existence. Therefore, it is imperative to ensure that special education TEPs are preparing candidates who are equipped to work with CLD learners. Culturally Responsive Practice in Special Education Current literature is rife with recommendations from the field to develop programs that prepare culturally responsive and competent special educators (Dykes, Gilliam, Neel, & Everling, 2012; Imler, 2009; Kea & Trent, 2013; Taylor, 2010; Sobel et al., 2011). In fact, Ortiz and Robertson (2018) posit that TEPs have not adequately responded to the demographic changes in student population or the number of ELs enrolled in schools today. There is little research on the programmatic redesign process that special education TEPs have undergone to better meet the growing needs to develop culturally responsive special education teacher candidates. Further research on the types of PD available to faculty to grow in their understanding of CRP is nearly nonexistent. If TEPs prioritize faculty expertise, this will impact our students and the resources they bring to their future classrooms (Ortiz & Robertson, 2018). Additionally, when faculty demonstrate culturally responsive instruction in their practice, they inadvertently encourage teacher candidates to model similar teaching styles in their classrooms (Baumgartner, Bay, Lopez-Reyna, Snowden & Maiorano, 2015). To develop culturally responsive teachers, TEPs must address pedagogical understanding about asset-based instruction, social justice, racial equity and family/community involvement for school improvement. Teachers must first establish reflective practices to develop an understanding of the assumptions and misunderstandings they hold which can lead to tensions between teachers and their learners (Dray &Basler-Wisneski, 2011). Hayes and Juarez (2012) posit that whiteness must be considered and addressed when preparing future teachers, so they are ready to actively make choices for social justice in education. Racial equity should be a value on which teacher preparers shine a light as they uncover how schools have historically undersupported and problematized CLD learners (Gulati-Partee & Potapchuk, 2014). Finally, it is critical to include community as a resource with funds of knowledge (Moll, Amanti, Neff & Gonzalez 1992) that can be built upon versus trying to excavate culture and conform to the unquestioned standards of behavior by the dominant culture (Gulati-Partee & Potapchuk, 2014).

Previous Programmatic Reviews and Rationale for this Study To add an additional layer of complexity in this work is how the construct of CRP is measured. There are definitions of CRP that are widely used and accepted in the field (Sobel et al., 2011), but researchers have yet to agree on an operational definition of cultural competence (Kahn et al., 2014). On the other hand, experts can agree on the need for a multi-dimensional approach to infusing cultural competencies, and TEPs have taken various approaches to prepare teacher candidates for the diverse needs of the ever-evolving classroom. In the late 1970s, the National Council for the Accreditation of Teacher Education and the Association of Teacher Education recognized the disparities that existed in the academic achievement of CLD learners with and without disabilities and developed diversity standards (Trent et al., 2008). One outcome related to these standards, is that TEPs began to include multicultural content into their programs in a variety of ways, including stand-alone courses and redesigned programs with cultural competencies infused into existing courses. Trent and colleagues (2008) examined how TEPs integrate multicultural education in preservice general and special education programs. The authors reviewed empirical studies from 19982004; and, of the 46 total articles reviewed, they found only seven articles related to special education TEPs. Of those seven, two categories emerged. First, four of the articles focused on attitudes and beliefs of preservice special education teacher candidates that used pre- and postmeasures such as surveys or concept maps to evidence conceptual growth. The other three related to teacher efficacy of preservice teachers and if they could implement CRP as a result of their training. The research as it relates to special education programmatic redesign of TEPs is sparse, and we intended to further the work of Trent and colleagues (2008) by examining additions to the literature since the time of their review. While there has been much discussion about how to better prepare preservice special educators for CLD classrooms, interestingly, postsecondary faculty have found themselves in a similar predicament with their public-school colleagues. They are generally comprised of a White teaching staff with minimal experience teaching culturally diverse populations (Imler, 2009; Kahn et al., 2014). The very faculty that is charged to prepare teacher candidates to be culturally responsive educators may not have the experience themselves to do so (Imler, 2009; Prater, Wilder, & Dyches, 2008; Taylor, 2010). Moreover, there is little published information available about the internal processes special education TEPs have undergone to ensure their curricula and programs are preparing teachers to be culturally responsive. Thus, in this literature review we sought to address the following research questions: 1. What processes have special education TEPs undergone to redesign their curriculum to include CRP? 2. What support and training were provided to special education faculty to carry out the redesigned program? 3. What outcome data were collected on training or redesign processes to determine impact? Method

63 We based the criteria on topics related to teacher preparation of preservice special educators to teach CLD students with disabilities. We conducted the following procedures to identify relevant articles. First, we conducted a comprehensive search of key online databases (i.e., Eric Resources Information Center (ERIC) in Ebsco, Academic Search Complete, Professional Development Collection, Primary Search, and PsychINFO). We limited the search to the years 2008-2017, only included peer-reviewed journals, and we used the Boolean OR as we searched the term teacher preparation programs OR teacher education programs and identified 37,021 sources. Next, we used the Boolean AND OR and searched the terms disability, OR disabled OR disabilities AND special education which yielded 926,027 sources. The final set of terms with the Boolean OR were cultural competence OR culturally re* (to include responsive, relevant, responsive pedagogy, relevant pedagogy, etc.) the search yielded 64,569 sources. We then combined all three sets of search terms with the Boolean term AND which resulted in 38 sources. We also excluded studies conducted outside of the United States. After we applied all exclusionary factors, the search yielded 20 articles. To determine if all articles were relevant, we read all abstracts or the article in its entirety if more information was needed and chose articles that met the following inclusionary criteria: (a) concentrated on special education teacher preparation programs (b) provided information about faculty PD as part of their redesign process and (c) considered the training that preservice special educators needed to be effective in CLD classrooms. We omitted a total of 13 articles: 5 focused on teacher beliefs about CLD learners or teacher efficacy; 4 considered components of CRP with preservice teachers but focused on a component of the program such as field experiences or the need for multicultural experiences without considering faculty PD or the redesign process; 2 were duplications; 1 article was an introduction to a special journal issue that focused on CRP; and 1 article focused on special education eligibility determination based on culture but did not connect to TEPs nor did it focus on preservice teachers or faculty PD in conjunction with curricular redesign. This resulted in six articles that met all of our inclusionary criteria. One of these six articles (Zion & Sobel, 2014) did not include faculty PD; however, it was a follow-up article (Sobel et al., 2011) where faculty PD was discussed, which resulted in seven articles. Additionally, we conducted a hand search and contacted principal investigators who had been awarded Office of Special Education Programs (OSEP) grants for culturally responsive special education teacher preparation to inquire if they had any publications in progress. We also searched the Office of English Language Acquisition (OELA) because we knew that they have funded different universities to create programs at the undergraduate level to address the needs of ELs and sought to see if they had any publications we may have missed in our search; however, this process did not yield any additional results. Finally, we were aware of the existence of a program at a particular institution; thus, we reached out to the main stakeholders to inquire if they had any publications about their program and they shared two recent articles that met our inclusionary criteria that were published after our initial search. The authors coded each article by the type of PD in which faculty engaged in regarding the redesign process and additionally if any outcome data were collected. In total, there were nine articles included in the literature review (see Table 1).

64 Table 1 Summary of Literature Review Results Article Special Education Redesign PD Monarch Training Combined elementary and special Cyr, et al., for re-design team education programs for a dual2012 licensure program

Outcome Data Not reported

Dykes et al., 2012

Journal Readings & Workshops

Complete re-design with a one-year timeline 123-credit hour program combining four programs (Early Childhood, SED, Reading & C & I) 7 new courses

Not reported

Lee, 2018

Faculty immersion with community and school collaboration, journal readings & workshops

Re-designed courses in general and special education that provide multiple exposures along with community immersion trips; summer immersion for teacher candidates; field experiences including student teaching

Not reported

Mustian et al., 2017

PD was discussed in the Lee article

10 re-designed core courses in special education with community immersion trips beginning at the Freshmen level

Pre- and post-surveys on teacher efficacy, attitudes related to diversity and likelihood of teaching in an urban environment

Pappamihiel et al., 2010

45 hours of EL instruction

Likert survey sent to Combined undergraduate and graduate program with an integrated graduates to determine efficacy EL endorsement with SED for teaching ELs

Prater et al., 2008

3 Year PD Plan Training on CREDE standards Strengths/Need Assessment Interview CLD candidates

Combined undergrad and graduate program with EL minor; Infuse CREDE standards into Course/Field Experience; Cohort Model SED Major (50 hours) EL Minor (16 hours)

Recruitment efforts increased the number of teacher candidates from diverse backgrounds

Robertson et al., 2012

Literature Review Monarch Center

Two-course sequence as a foundation for cultural and linguistic knowledge; Additional competencies were included across existing courses

Focus groups with teacher candidates; Surveys sent to cooperating teachers and principals to determine evidence

65 of cultural and linguistic diversity (amongst one of 12 domain areas) Sobel et al., 2011

Feedback loops On-going redesign activities Specialized Workshops 19 assigned Readings

New courses discussed in the Zion & Sobel article.

Not reported

Zion, & Sobel, 2014

PD was discussed in the Sobel et al., article

3 new courses were developed as part of an integrated general and special education teacher program to address gaps that were identified in the data analysis

Not reported

Results Of the nine included articles, we present findings from seven universities across the country that vary in size from small private colleges to large research institutions. All programs took different approaches towards how they conducted PD for their faculty and redesigned their core curriculum. We present the findings by describing the PD process special education faculty underwent to expand their cultural responsiveness and how TEPs then made programmatic changes to improve their practices based on increased cultural competencies. Finally, we report any outcome data faculty collected on the effectiveness of their faculty PD and/or their course redesign process. In sum, eight of the nine articles reviewed addressed faculty PD as a theme or sub-theme followed by the redesign process (i.e., Cyr et al., 2012; Dykes et al., 2012; Lee, 2018; Mustian, Lee, Nelson, Gamboa-Turner, & Roule, L., 2017; Pappamihiel, Otaiba & Hudson, 2010; Prater et al., 2008; Robertson, Garcia, McFarland, & Rieth, 2012; Sobel et al., 2011). Special Education Faculty Professional Development Of the nine articles reviewed, we noted that university faculty took a variety of approaches to increase faculty cultural awareness. Within the context of faculty PD, two themes were consistently represented in the articles reviewed: training teams and timelines for PD and curriculum redesign along with the type of faculty training. Training teams and timeline for professional development When developing PD, four of the nine articles selected a team of 3-6 faculty to facilitate the process (i.e., Cyr et al., 2012; Dykes et al., 2012; Prater et al., 2008; Sobel et al., 2011), and five did not discuss their team or timeline (i.e., Lee, 2018; Mustian et al., 2017; Pappamihiel et al., 2010; Robertson et al., 2012; Zion & Sobel, 2014). Cyr et al. (2012) designed a planning team that included the education department chair, a faculty member trained in action research, and a faculty member from the physical education (PE) department who specialized in adaptive PE. All were trained by the Monarch Center, a nationally funded technical assistance center to ensure objectives of dual licensure programs, and received both, financial and organizational support by the

66 administration. Several faculty members earned grants and stipends, one member of the planning team received a three-credit course release to research CLD, and two faculty members went on sabbatical to study CRP. Additionally, a retired faculty member acted as a consultant to help redesign syllabi with objectives and activities that met both elementary and special education licensure requirements. Dykes and colleagues (2012) had a training team that included the Director of the School of Education, two faculty members from the general and special education departments, as well as two faculty members with EL training; together they led the PD and redesign process. They established a timeline goal of one year to complete PD, gain program approval from faculty, and complete the redesign process. Prater and colleagues (2008) included the department chair, special education coordinator, and an EL mentor in their training team that developed a 3-year PD plan for all faculty to increase their knowledge of multicultural and special education issues (Prater et al., 2008). During year one, all faculty members completed training offered by the Center for Research on Education Diversity and Excellence (CREDE). The training focused on culturally responsive and EL teaching strategies and techniques. Faculty infused the cultural competencies they learned into one of their courses and one fieldwork experience during years two and three of the PD plan. In addition to training sessions, faculty members identified goals, considered areas of strength, and areas to improve as they related to the CREDE standards. In the third year of PD, the faculty learned how to conduct interviews, made home visits, and gained an understanding of effective components, as well as challenges that influenced program changes. Sobel and colleagues (2011) developed a PD team that included the Associate Dean, the Director of Teacher Education, a special education faculty member, and the Executive Director of Continuing and Professional Education. Sobel et al. (2011) noted that TEPs undergoing a redesign process must first embark on the hard work and develop content they are preparing to teach. Therefore, they partnered with 30 Professional Development School sites across their six urban districts where 350 graduate and undergraduate students enrolled in TEPs. Although Pappamihiel and colleagues (2010) did not formally mention a team, they discussed how faculty members from both the special education and multicultural/multilingual education programs that were autonomous of one another came together to develop a new program. The collaboration allowed preservice teachers to earn their EL endorsement in conjunction with their special education degree because they understood the benefits of learning from one another. Faculty training The type of training faculty took varied in formats as well. Some universities included additional EL coursework (Pappamihiel et al., 2010) and diversity training workshops (i.e., Dykes et al., 2012; Prater et al., 2008; Sobel et al., 2011) or community immersive experiences (i.e., Lee, 2018; Mustian et al., 2017) whereas, others conducted literature reviews (i.e., Robertson et al., 2012) or invited faculty to engage in selected journal readings and activities to promote cultural awareness (i.e., Dykes et al., 2012; Sobel et al., 2011). Outside of journal readings, CREDE and Monarch were most frequently cited as formal training for faculty that included diversity standards (i.e., Cyr et al., 2012; Prater et al., 2008; Robertson et al., 2012). English learner coursework Rather than creating a formal PD plan, Pappamihiel and colleagues (2010) determined it was important to understand EL theory and methodology in order to teach their preservice teachers. As

67 a result, special education faculty took 45 hours of in-service training that focused on infusing EL methodology, curriculum, and cross-cultural understanding into their existing courses (Pappamihiel et al., 2010). In addition to the EL training, faculty collaborated by sharing their own experiences from various cultural backgrounds which helped them develop a shared perspective and a sense of purpose. While this may not be typical PD, the authors discuss how this collaboration allowed faculty to learn about the others' field(s) and develop a process of integration that moved beyond the boundaries of discrete programs. Diversity training workshops Dykes and colleagues (2012) noted that journal readings and workshops were essential to faculty PD. However, they did not include the titles or timeline for PD. Sobel and colleagues (2011) explained that as part of the PD, the Achieving Special Education Equity through Diversity Project (ASEED) team established three structures for PD which included: feedback loops, on-going redesign activities, and specialized workshops (Sobel et al., 2011). Experts provided workshops on instructional strategies, multicultural education, and power and privilege. During specialized workshops, experts assisted with framework design and later provided ongoing mentorship. Finally, 40 faculty members participated in a 2-day faculty retreat where they developed outcomes, goals, and established next steps. Community immersive experiences Lee (2018) and Mustian and colleagues (2017) are each part of the same university and annually invite faculty members to submit a proposal for a course development grant to redesign one of their teacher preparation courses with an emphasis on urban preparation. If accepted, faculty must first engage in a week-long immersive trip in Chicago. Faculty members are introduced to members of community based-organizations and school administrators from partnering neighborhoods. Community partners can portray the neighborhoods through a different lens than is often portrayed in the media (Mustian et al., 2017). This process allows faculty to examine their own biases and authentically integrate CRP into their courses through the lens of community assets (Lee, 2018). Literature reviews or journal readings Robertson et al. (2012) and faculty began their PD by conducting a literature review that defined the highly-qualified competencies needed for special educators to be culturally responsive. This was the catalyst for designing a matrix to align with the Council for Exceptional Children (CEC) standards and CLD content within their program redesign. During monthly meetings, Sobel et al. (2011) explained that faculty members discussed assigned readings to build shared knowledge. They read and reviewed 19 articles in four categories (i.e., socio-cultural approaches to learning and instruction, simultaneous renewal of teacher education and P-12 schools, inquiry as a stance, and culturally responsive education). Formal training Faculty from both the special and general education departments worked with the Monarch center to develop a collaborative intervention project that preservice teachers in the general and special education cohorts would implement in inclusive settings. The collaboration would encourage them to share responsibility when teaching learners with disabilities (Robertson et al., 2012). Additionally, Cyr and colleagues (2012) were also trained by the Monarch team before embarking on their redesign process. Prater and colleagues (2008) discussed the formal training that their faculty members completed through the Center for Research on Education Diversity and Excellence (CREDE). The training

68 focused on culturally responsive and EL teaching strategies and techniques. Faculty infused the cultural competencies into one of their courses and one fieldwork experience during years two and three of the PD plan. In addition to training sessions, faculty members identified goals, considered strengths, and improvements related to the CREDE standards. Curriculum Redesign Process Just as there were varied approaches to PD, faculty also redesigned their coursework in a multitude of ways. Of the nine articles reviewed, four were specific to two TEPs (i.e., Lee, 2018; Mustian et al., 2017 and Sobel et al., 2011; Zion & Sobel, 2014); therefore, there were a total of seven university programs. Of the TEPs that redesigned their curriculums, five of the seven universities were undergraduate programs (i.e., Cyr et al., 2012; Dykes et al., 2012; Lee, 2018; Mustian et al., 2017; Robertson et al., 2012; Sobel et al., 2011, Zion & Sobel, 2014) and two combined undergraduate and graduate programs (i.e., Pappamihiel et al., 2010; Prater 2008). Three themes emerged, (a) reimagining the path to licensure through the redesign process, (b) new courses that were developed to meet newly identified needs, and (c) programs that were operating on grant funding. Path to licensure Two main subcategories emerged, which were that programs either combined their undergraduate programs (i.e., Cyr et al., 2012; Dykes et al., 2012) or combined existing undergraduate and graduate programs (i.e., Pappamihiel et al., 2010; Prater et al., 2008). One institution redesigned their program and created a dual-licensure program that combined elementary and special education programs (i.e., Cyr et al., 2012). Another university combined four undergraduate programs into one newly redesigned program (i.e., Dykes et al., 2012). Two universities integrated their EL endorsement with their special education degree program which resulted in a graduate degree upon completion (i.e., Pappamihiel et al., 2010; Prater et al., 2008). Combined programs. Cyr and colleagues (2012) recognized that some of their methods courses would need modification to include the tenets of CRP. The internal curriculum committee oversaw and approved each new objective and assessment outcome that was introduced. The first three courses to undergo this transformation were math, reading, and social studies methods courses. The new objectives included the candidates’ abilities to design and implement differentiated concepts across ability levels, design culturally relevant lessons, and consider the needs of ELs. The authors discussed the continuous nature of a redesign process and the importance of meeting to determine strengths and areas to improve. Even with release time, collaboration is an intensive process and was one of the biggest challenges that faculty members encountered. Many faculty members preferred working alone, and fostering a collaborative approach is something they are working towards. This emphasized the need for continued PD to acquire new techniques and strategies when working together. Having a timeline and distribution of tasks was essential to the success of the project. Dykes and colleagues (2012) combined four departments (Early Childhood, Reading, Special Education, and Curriculum and Instruction) into one program. The major influence of this change was survey data from recent graduates and stakeholders that confirmed the need to redesign the TEP to meet the diverse needs of learners in their state. Faculty set aside several hours each week to review data and syllabi to determine how to combine or eliminate redundant courses to keep within a 120-123 credit hour range to licensure. The team brought three options for the redesign to faculty members for consideration, and they selected a 123-credit hour path to licensure with an increased number of special education and EL courses. Their state does not require specific course content for each program, but rather teacher candidates must pass certification exams for licensure

69 in specialized areas. Seven new courses were added to the sequence including, Introduction to Special Populations, Language and Literacy Acquisition, Managing Classrooms and Behavior in School Settings, Assessment for Instruction, Instructing Diverse Learners, English Language Learners and Collaborating with Families and Community. Additionally, existing courses now included modules in EL and special education. The initial implementation began during the Fall of 2011 with a 21-credit hour load that included introduction to teaching and special populations courses. The second phase included 15-credit hours plus 30 hours of fieldwork. Preservice teachers must earn a minimum GPA or 2.5 to move on to each additional phase. Phase 3 consisted of methods courses in math, science, social studies, and language arts along with 60 hours of fieldwork. Phase 4 is student teaching. The authors noted that the implementation did have a few complications but believed the success was due to open lines of communication amongst the personnel and organizing the courses into implementation phases. Additionally, a student advisory focus group met twice during each semester to provide feedback on each implementation phase. Combined undergraduate and graduate programs. Pappamihiel and colleagues (2010) developed a combined undergraduate-graduate special education program. Teacher candidates join the program during their junior year and then graduate with a bachelor's and master's degree to teach exceptional learners. Students can choose an area of focus including high incidence disabilities, transition, special education technology, early childhood special education, visual impairments, autism, or severe disabilities. All TEPs in their state require an equivalent of 15 hours of EL training as part of their preservice program. The authors surmise that their demographics are not unique and other TEPs need an integrated program to improve school for all children. The authors believe that learning is interwoven and did not want to perpetuate the idea that EL and special education are isolated fields, so they began their work on integrating EL methodology, curriculum, and cultural understanding into the existing special education coursework. The faculty decided that experts in EL would teach specialized areas such as applied linguistics and assessment for ELs. Faculty members determined that a cohort model would best support the teacher candidates as they progressed through the program as a group. The first cohort in the integrated model had a group of 25 preservice teachers who entered as juniors and graduated with their master’s degrees in special education and an endorsement in EL. The authors likened this redesign experience to that of a blind date. Faculty from the two departments looked for areas of compatibility and mutual interests. They believed that getting to know one another professionally was instrumental in the success of their integrated program. The authors also discussed the importance of keeping the lines of communication open with graduates during the challenging first years of teaching. In addition to faculty applying knowledge from the PD and infusing cultural competencies based on the CREDE standards into their existing coursework or clinical experiences, Prater and colleagues (2008) initiated a plan to offer an undergraduate and graduate program that combined special education with an EL minor. The Bachelor of Science in Special Education was reestablished during the Fall of 2004 and included 50 credit hours of special education coursework, including assessment pedagogy, special education law, and multicultural education. Additionally, they provided 16 hours of EL courses which focused on language acquisition and five CREDE standards (i.e., Joint productivity, Language Development, Contextualization, Challenging Activities, and Instructional Conversations). Development of new courses or re-designed curriculum. Five of the articles reviewed discussed the development of new courses, or how they redesigned existing courses to reflect cultural

70 competencies (i.e., Lee, 2018; Mustian et al., 2017; Robertson et al., 2012; Sobel et al., 2011; Zion & Sobel, 2014). Lee (2018) outlines how faculty within the College of Education redesigned their courses with input from schools personnel and community partners to integrate academic content with community needs as part of the Urban Teacher Preparation (UTP) model. Mustian and colleagues (2017) discussed specifically how faculty within the Department of Special Education has partnered with UTP to systematically redesign 10 of their core courses. The courses explore social issues that encircle disability, including cultural and linguistic pedagogy, disciplinary practices, and the inclusion of families and communities in the educational environment. In addition to the redesign of courses, preservice teachers participated in community immersion trips in underserved neighborhoods which are built-in components of each redesigned course. The course instructor is paired with a program coordinator that is affiliated with the pipeline and a community liaison from one of the partner neighborhoods. Together, they design an experience that aligns with course objectives to develop change (Mustian et al., 2017). This early exposure to partner neighborhoods allows preservice teachers to change the damage-centered single story that has been shared with them. Since the redesigned sequence is systematic, the experiences build on one another are unique to each course. Ultimately, preservice teachers can choose to complete their clinicals and student teaching in one of the partner neighborhoods in Chicago. Mustian and colleagues (2017) reiterate the importance of building foundational skills rooted in culturally responsive practice that allow preservice teachers to celebrate the diversity and respecting the cultural history of local urban communities. Robertson et al. (2012) revised their existing special education program to include linguistic and cultural competencies. Selected activities and assignments were reflective, community-based, and emphasized culture. Instructors designed two new courses that served as the foundation for cultural competencies. Sociocultural Influences on [Teaching and] Learning is a requirement for all undergraduates and focuses on culturally responsive practice and the intersectionality of culture, language, and disability. Special education majors take a variation of this course that aligns with CEC standards. Students enroll in the second course, Intercultural Communication and Collaboration during their student teaching semester. Front-loaded content allows students to apply their knowledge throughout their student teaching experience. Sobel and colleagues (2011) conducted an external review of the special education curriculum before the redesign with a panel of six experts in inclusive education and culturally responsive pedagogy. They found a strong emphasis on clinical experiences, but a lack of social justice and community-based learning. In a follow-up article, two of the authors of the original article, Zion and Sobel (2014) discussed the survey data they collected to inform the redesign process. The authors wanted to go beyond enhancements of existing course syllabi and use data from the external review as well as input from stakeholders to identify gaps. They sent a qualitative survey to five different focus groups (N = 102) including: current teacher candidates, recent graduates, school personnel (clinical teachers/principals), culturally and/or linguistically diverse students with disabilities, and families of culturally and/or linguistically diverse students with disabilities. The purpose was to determine the critical skills that preservice teachers needed to instruct CLD learners. The survey results indicated that stakeholders recognized high emerging awareness in the following categories: affirmative attitudes, collaborative skills, and diversity of pedagogy. They also identified a large number of deficit-oriented comments made by teacher candidates that related to a lack of awareness of the impact that social, historical, or cultural influences have on learning.

71 Based on the findings, the department developed an action plan and created new courses. They utilized a conceptual framework that focused on social justice and equity to backward map and redesign these courses. Instructors designed the Social Foundations and Cultural Diversity in Urban Education course to examine how disability, race, gender, ethnicity, sexual identity, and power and privilege issues affect education. To address deficit views of CLD learners, the team created the Developing Culturally Responsive Classroom Communities course to examine how students have been marginalized and misunderstood, and how schools determine ability levels. Finally, to address the gap in collaborative skills, they developed the Data Informed Decision Making for Diverse Learners Across Learning Environments course. Grant-funded support. Six of the nine articles reviewed disclosed that they received either external or internal grant funding during their redesign process. Three universities were awarded OSEP grants (i.e., Prater et al., 2008; Robertson et al., 2012; Sobel et al., 2011). One was supported through internal stipends or course release time (Cyr et al., 2012) and two offered financial support through grants for faculty to redesign their courses (Lee, 2018; Mustian 2017). An OSEP grant provided funding to support the costs of tuition, fees, and textbooks to students who enrolled in the undergraduate or graduate program (Prater et al., 2008). Sobel and colleagues (2011) received an OSEP grant for their redesign project titled Achieving Special Education Equity through Diversity Project (ASEED). Robertson and colleagues (2012) earned an OSEP grant to implement Project RISE (Restructuring Instruction in Special Education), which is a 5-year federally funded project to help redesign and prepare preservice teachers to serve CLD students with disabilities (Robertson et al., 2012). Their state is responsible for educating nearly five million students, many of whom come from CLD or economically disadvantaged backgrounds. The Special Education Department was one of five departments housed in the College of Education. Each department offered a stand-alone path to licensure that adhered to a cohort model with 25 candidates. Candidates could not take courses outside their program cohort. Therefore, special education majors could not take any of the bilingual education classes or vice versa prior to their redesign. Outcome Data The researchers were interested in examining the types of data different universities collected and recognized that not all reported outcome data (i.e., Cyr et al., 2012; Dykes et al., 2012; Lee, 2018; Sobel et al., 2011; Zion & Sobel, 2014). When included, three types of outcome data were reported; recruitment efforts (i.e., Prater et al., 2008), teacher efficacy surveys (i.e., Mustian et al., 2017; Pappamihiel et al., 2010) and stakeholder surveys (i.e., Robertson et al., 2012). Recruitment efforts Prater and colleagues (2008) focused on recruiting students from diverse backgrounds into the redesigned cohort. In the initial year, faculty members recruited 19 students into the cohort, 16 of those students were White. They continued to develop their recruitment efforts, and by the third year of the cohort, admitted 33 students, 18 of whom had racial backgrounds other than White. Two cohorts completed the program: 79% of the students in the first cohort graduated, and 89% graduated in the second. Efficacy surveys After graduation, Pappamihiel and colleagues (2010) sent a Likert-scale using a 1 (highly disagree) to 5 (highly agree) survey to all 25 graduates to determine how prepared they felt to teach ELs. Fifteen graduates responded to the survey and response averages ranged from 3.78 to 4.72 indicating that they felt prepared to teach ELs. Additional responses from the 15 graduates

72 indicated that 80% chose jobs in urban settings, and 53% had ELs in their classrooms. Most graduates took positions in elementary settings (66%), and 73% were working in high-incidence disability settings. Each semester, preservice teachers who are enrolled in the redesigned courses complete pre- and post-surveys (Mustian et al., 2017) to assess (a) the likelihood of teaching in an urban environment, (b) candidates’ attitudes towards diversity, and (c) teaching efficacy. Paired t-test scores increased in all assessed categories and showed statistical significance in teaching efficacy and school-based experience across both semesters. In the Spring semester, paired t-test scores showed statistical significance in urban teaching intentions, additionally, the Fall semester showed significance in the attitudes toward diversity category. Stakeholder surveys Robertson and colleagues (2012) sent surveys to cooperating teachers and principals to provide input on strengths such as effective instructional practices, issues of cultural and linguistic diversity and progress monitoring as well as areas that led to program modifications in competencies for response to intervention and differentiated instruction. Of the 12 domain areas identified in the survey, nine areas averaged 3.5 or above on a 4-point scale (1 = Strongly disagree; 2 = Disagree; 3 = Agree; 4 = Strongly agree). The lowest score from a principal or cooperating teacher was a 3.3 which still falls in the agree category (Robertson et al., 2012). The authors discussed that their implementation efforts have been hard to sustain due to focusing on single courses rather than transforming the TEP. Although cultural competencies are "infused" in other courses, the fidelity of delivery is dependent on instructors. In addition, student focus groups were utilized to provide inputs on the revised program at the end of each semester. However, no outcome data were provided on how they responded. Discussion In this literature review, we explored the programmatic redesign processes of seven universities that acknowledged the need to better prepare preservice special education teacher candidates for a diverse student population. Our research questions guided us to explore peer-reviewed journal articles on the PD of special education faculty as it relates to preparing culturally responsive special educators, as well as the varied approaches that TEPs are taking to implement CRP into their programs. Finally, we sought to understand the effectiveness of the PD or course redesigns by examining outcome data provided within these findings. We summarize commonalities and differences found across all programs reviewed, as well as possible solutions for some of the problem areas that emerged. Specifically, we discuss the areas of (a) special education faculty professional development, (b) curricular/course redesign process, and (c) outcome data to support programmatic changes. Special Education Faculty Professional Development Overall, we noted that faculty took varied ways to increase their cultural awareness. In some instances, the impetus for special education faculty PD often stemmed from a cultural mismatch between faculty, preservice teachers, and the cultural backgrounds of learners in public schools (i.e., Cyr et al., 2012; Prater et al., 2008). Several TEPs analyzed in this literature review utilized inputs of their stakeholders to determine areas of strength as well as development opportunities to meet the needs of CLD learners in today's classrooms (i.e., Cyr et al., 2012; Dykes et al., 2012; Sobel et al., 2011; Zion & Sobel, 2014). Inviting stakeholders to the conversation empowered teachers, families, and administrators to have a voice in the process since they are the ones in the

73 trenches. Their collective voice can advise TEPs to reflect the needs in the field when designing PD. Though faculty members may embody a personal commitment to increasing cultural competencies within their content areas, it may be most effective when they organize as a team with established timelines, roles, and responsibilities to ensure systematic inclusion of culture across the program. Additionally, the majority of authors shared their timeline for delivering PD and discussed the members that comprised the PD team (i.e., Cyr et al., 2012; Dykes et al., 2012; Prater et al., 2008; Sobel et al., 2011). Redesign Process After undergoing PD, each institution moved into the redesign process. There were various approaches, many of which took a more collaborative approach versus working in silos (Baca et al., 2004). It is apparent that CLD learners with and without disabilities have a unique set of needs that cannot be addressed in disjointed or decoupled programs. Each of the universities were collaborative in their redesign approach and either (a) did work in partnership with others outside of their discipline as they combined programs (i.e., Cyr et al., 2012; Dykes et al., 2012; Pappamihiel et al., 2010; Prater et al., 2008) or (b) designed new courses to their existing programs (Lee, 2018; Mustian et al., 2017; Robertson et al., 2012; Sobel et al., 2011; Zion & Sobel, 2014). As faculty redesigned their programs and/or courses, they reviewed syllabi and course objectives to ensure that required components were not overlooked when adding in cultural competencies. We know that teacher educators are often faced with the challenging task of designing programs that meet state licensure requirements while maintaining a timeline of four years (Ortiz & Robertson 2018). Two university programs re-designed clinical experiences to include cultural competencies (i.e., Lee, 2018; Mustian et al., 2017; Prater et al., 2008) and one included 60 hours of fieldwork as part of a redesigned literacy course (Dykes et al., 2012). We also found it necessary to note that several of the universities were receiving federal or internal grant funding to support their redesign efforts (i.e., Cyr et al., 2012; Lee, 2018; Mustian et al., 2017; Prater et al., 2008; Robertson et al., 2012; Sobel et al., 2011). The work of redesigning courses and/or combining programs is involved and time-consuming. Having additional compensation or release time from teaching duties may have been a motivating factor to complete the changes. Outcome Data Little objective data exist to evidence the impact that PD had on how faculty have included CRP into their instructional practices or if course redesigns have achieved the goal of increasing cultural competencies of special education teacher candidates. Specifically, only four of the nine included articles provided data at the preservice teacher level. Two of the four articles reviewed sent Likert surveys to graduates to determine efficacy for teachers teaching learners whose native language is not English (Pappamihiel et al., 2010) and Robertson and colleagues (2012) sent a survey to cooperating teachers and administrators to determine cultural and linguistic diversity of their teacher candidates. One set of authors analyzed pre- and post- surveys to determine preservice teacher efficacy and attitudes to evaluate the impact of the redesigned course (i.e., Mustian et al., 2017). Finally, one TEP collected outcome data on their increased recruitment efforts of teacher candidates from diverse backgrounds (Prater et al., 2008). Focus groups were also used within two university programs to understand student perception of the redesigned courses at different stages of implementation (i.e., Dykes et al., 2012; & Robertson,

74 2012), however, neither included these results in their articles. Robertson and colleagues (2012) discussed the difficulty of sustaining implementation efforts when faculty members infuse cultural competencies into courses as it depends on the fidelity of their delivery. Limitations and Future Research When interpreting the findings from our review, the reader should consider several limitations. Without an operational definition of CRP, it is challenging to measure competence and selfreported outcomes by the authors who are typically integral members of the redesign team. Another limitation was an overall lack of quantitative or qualitative data to determine the effectiveness of the PD and its impact on faculty. Attendance, participation, and completion of readings and workshops fail to accurately measure the effects that PD has on an individual or how it will transform their future instruction. Additionally, the outcome data included on effectiveness of course or programmatic redesign are limited, in that they are often self-report measures and do not provide quantitative evidence of how these participants evidence CRP in their teaching or how this impacts PK-12 students. Future research should include outcome data that can be disseminated in the field to support these program revisions and guide universities in improving their TEPs in this specific area. Our first suggestion is that future research should include pre- and post-assessments to determine implicit bias and efficacy of faculty members. In reviewing the literature, a systematic approach was lacking that would allow other universities to replicate these efforts. Furthermore, it is likely that more publications are needed to better understand the outcomes of faculty PD and the impact that course and program redesigns have on preparing culturally responsive special educators. Archibald and colleagues (2011) recommend process and impact metrics as a way to evaluate the outcome of professional development activities. Published evaluations of high-quality PD is scarce, which can be frustrating when trying to determine which process, tools, or approaches will best meet the needs of a particular institution (Archibald, Coggshall, Croft & Coe 2011). Evaluating PD provides evidence of effectiveness, facilitates program improvements, and advances the field (Archibald et al., 2011). Second, our review focused primarily on the actions of faculty members, through PD and teacher education programs. We did not concentrate on the attitudes, values, and beliefs that preservice candidates personally hold about working with CLD learners. One set of authors addressed this, but it was not the focus of their article (Mustian et al., 2017). Considering the attitudes and beliefs of special education preservice teachers is a critical aspect to consider when redesigning programs; however, our focus was on TEPs and their process. Third, it is likely that there are currently several federally funded grants with a focus on culturally responsive practices in special education indicating that additional TEPs are moving forward in redesigning their special education programs to include cultural competencies for faculty and preservice teachers; however, little has translated to published research. We suggest that TEPs that embark on a redesign process of their own will also examine metrics that will provide objective data to evaluate the effectiveness of their efforts. Research must go beyond the perceptions of CRP self-efficacy and explore actualization at the preservice level where teacher candidates are applying CRP in their lesson plans and through direct observations of their classroom performance. Robertson and colleagues (2012) made mention that it was challenging to sustain CRP implementation efforts due to the fidelity of delivery amongst faculty. How can we ensure that faculty members have internalized culturally responsive practices and consider it a critical part of

75 the content they are teaching? Perhaps future studies can determine the long-term effectiveness and fidelity in which faculty members embed cultural competencies across course sequences. As faculty seek ways to collect data on the impact of their course redesigns, they may want to consider the work of McHatton and colleagues (2013) as an outcome measure of effectiveness in developing cultural competencies amongst preservice teachers. They used pre-and post-tests to gain an understanding of the impact that differentiated instruction had on becoming a culturally responsive teacher. The researchers used the Developmental Model of Intercultural Sensitivity (DMIS) to determine overall intercultural developmental and perceived sensitivity. Participants answered 50 questions using a Likert scale, and then researchers analyzed results and grouped students for instruction based on their baseline category. This allowed faculty to differentiate their instructional approaches, provide scaffolds, and choose various readings to facilitate discussion. When instructors chose materials at the developmental level that was appropriate for each group, there was growth amongst special education teacher candidates in the post-assessment. Having objective data to determine the effectiveness of instruction and impact on preservice teachers goes beyond perceived impact. Implications for Practice This review synthesizes what is currently known about PD for special education faculty to increase their cultural competence and how TEPs are taking that knowledge to redesign or reimagine traditional special education programs to meet the needs of CLD learners. Just as our goal is for preservice teachers to actualize CRP in their classrooms, faculty members would also benefit from peer observation to increase implementation fidelity of their redesigned courses. Participating in peer review sends a message to our students that learning is a critical part of our PD, and that we foster a sense of lifelong learning as we posit they should do (Richardson, 2000). Peer observation and reflection are critical to examining the real impact of faculty PD of CRP in the university classroom. Based on the literature, we have some recommendations for practice. First, increasing cultural competencies of preservice teachers begins with faculty in TEPs and their understanding of their own implicit biases and knowledge of CRP. We recommend that faculty review the literature and design PD that is systematic and can be replicated. Second, course redesign also involves a systematic approach that considers input from stakeholders when designing objectives and outcomes. Finally, including community and field experiences that deepen exposure and application of cultural competencies under the guidance and mentoring of faculty members is paramount to increased self-efficacy and skills of preservice teachers (Baumgartner et al., 2015). We suggest a recent framework that focuses explicitly on incorporating CRP into the college classroom that includes eight teaching recommendations for culturally responsive practice (Baumgartner et al., 2015). First, faculty should consider themselves as cultural beings and recognize how beliefs shape our interactions with members of various groups that have been influenced by the dominant society. Faculty are encouraged to model their process to assist teacher candidates in understanding how they can self-reflect on their biases. Next, faculty should develop relationships with their students and select curricular content carefully. It is crucial for faculty to get to know their teacher candidates, by understanding the values and experiences that their students bring to the classroom as a way to inform their teaching. Through the use of questionnaires and autobiographical essays, faculty can better meet the needs of

76 their learners. Further, faculty need to ensure that different viewpoints and voices are being represented in their coursework. Other recommendations are related to the learning environment, the importance of self-reflection and shared accountability for learning as faculty model student-centered teaching techniques through the use of collaborative groups, experiential learning, and instructional activities. Through modeling, teacher educators will demonstrate how to consider differences, communication styles, and cultural values which will likely impact how teacher candidates perform in their future classrooms in positive ways (Baumgartner et al., 2015). Further, faculty should create a supportive classroom climate where teacher candidates can engage in dialogue with their instructor and peers. Performance-based assessment can help teacher candidates identify areas of growth and encourage candidates to be active participants in their learning. Finally, Baumgartner and colleagues (2015) suggest that TEPs increase opportunities for pre-service teachers to observe culturally responsive practices in the classroom. We believe that this framework outlines and offers suggestions for faculty to implement in their courses while modeling purposeful application of CRP practices to preservice special educators. References Archibald, S., Coggshall, J. G., Croft, A., & Goe, L. (2011). High-quality professional development for all teachers: Effectively allocating resources (Research and Policy Brief). Retrieved from National Comprehensive Center for Teacher Quality website: http://www.tqsource.org/publications/HighQualityProfessionalDevelopment.pdf Baca, L.M., Baca, E., & de Valenzuela, J. S. (2004). Development of the bilingual special education interface. In Baca, L. M., & Cervantes, H. T. (Ed.), The bilingual special education interface (Fourth Edition) pp.100-123. Upper Saddle River, NJ: Pearson. Baumgartner, D., Bay, M., Lopez-Reyna, N. A., Snowden, P. A., & Maiorano. M. J. (2015). Culturally responsive practice for teacher educators: Eight Recommendations. Multiple Voices for Ethnically Diverse Exceptional Learners, 15, 44-58. doi:10.5555/2158396.15.1.44 *Cyr, E., McDiarmid, P., Halpin, B., Stratton, J., & Davis-Delano, L. (2012). Creating a dual licensure program in elementary and special education that prepares culturally responsive teachers. Interdisciplinary Journal of Teaching and Learning, 2(3), 158-168. Dray, B. J., Basler-Wisneski, D. (2011). Mindful reflection as a process for developing culturally responsive practices. TEACHING Exceptional Children, 44, 28-36. doi: 10.1177/004005991104400104 *Dykes, F. O., Gilliam, B. K., Neel, J., & Everling, K. (2012). Peeking inside Pandora's Box: One university's journey into the redesign of teacher educator preparation. Current Issues in Education, 15(2), 1-7. Gulati-Partee, G., & Potapchuk, M. (2014). Paying attention to white culture and privilege: A missing link to advancing racial equity. The Foundation Review, 6, 25-38. doi: 10.9707/1944-5660.1189 Hayes, C., & Juarez, B. (2012). There is no culturally responsive teaching spoken here: A critical race perspective. Democracy & Education, 20, 1-14. Hoover, J. J. (2012). Reducing unnecessary referrals: Guidelines for teachers of diverse learners. TEACHING Exceptional Children, 44(4), 38-47. Imler, S. J. (2009). Becoming culturally responsive: A need for preservice teacher candidates. Teacher Education and Practice, 22(3), 351-367. Kahn, L. G., Lindstrom, L., & Murray, C. (2014). Factors contributing to preservice teachers' beliefs about diversity. Teacher Education Quarterly, 41(4), 53-70.

77 Kea, C. D., & Trent, S. C. (2013). Providing culturally responsive teaching in field-based and student teaching experiences: A case study. Interdisciplinary Journal of Teaching and Learning, 3(2), 82-101. Klingner, J. K., & Eppolito, A. M. (2014). English language learners: Differentiating between language acquisition and learning disabilities. Alexandria, VA: Council for Exceptional Children. *Lee, R. E. (2018). Breaking down barriers and building bridges: Transformative practices in community and school based urban teacher preparation. Journal of Teacher Education, 69(2) 118-126. doi:10.1177/0022487117751127 McHatton, P. A., Smith, M., Brown, K. H., & Curtis, J. (2013). "First, do no harm": Purposeful preparation of culturally competent educators. Multiple Voices for Ethnically Diverse Exceptional Learners, 13(2), 19-31. Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory Into Practice, 31(2), 132141. doi:10.1080/00405849209543534 *Mustian, A. L., Lee, R. E., Nelson, C., Gamboa-Turner, V., & Roule, L. (2017). Jumping into the deep end: Developing culturally responsive urban teachers through community-immersive partnerships. The Educational Forum, 81(4), 467-481. doi:10.1080/00131725.2017.1350239 Ortiz, A. A., & Robertson, P. M. (2018). Preparing teachers to serve English learners with language and/or literacy related difficulties and disabilities. Teacher Education and Special Education, 41(3), 176-187. doi: 10.1177/0888406418757035 *Pappamihiel, N. E., Al Otaiba, S., & Hudson, R. F. (2010). Integrating English as a second language into special education teacher personnel preparation programs. Teacher Education and Practice, 23(4), 495-506. *Prater, M. A., Wilder, L. K., & Dyches, T. T. (2008). Shaping one traditional special educator preparation program toward more cultural competence. Teaching Education, 19(2), 137151. doi:10.1080/10476710802040765 Richardson, M. O. (2000). Peer observation: Learning from one another. The NEA Higher Education Journal, 16, 9-20. *Robertson, P. M., García, S. B., McFarland, L. A., & Rieth, H. J. (2012). Preparing culturally and linguistically responsive special educators: It "does" take a village. Interdisciplinary Journal of Teaching and Learning, 2(3), 115-130. Skiba, R. J., Simmons, A. B., Ritter, S., Gibb, A. C., Rausch, M. K., Cuadrado, J., & Choong-Geun Chung. (2008). Achieving equity in special education: History, status, and current challenges. Exceptional Children, 74(3), 264-288. Snyder, T. D., & Dillow, S. A. (2015). Digest of Education Statistics 2013 (NCES 2015-011). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. *Sobel, D. M., Gutierrez, C., Zion, S., & Blanchett, W. (2011). Deepening culturally responsive understandings within a teacher preparation program: It's a process. Teacher Development, 15(4), 435-452. Taylor, R. W. (2010). The role of teacher education programs in creating culturally competent teachers: A moral imperative for ensuring the academic success of diverse student populations. Multicultural Education, 17(3), 24-28. Terry, N. P., & Irving, M. A. (2010). Cultural and linguistic diversity: Issues in education. Special Education for all Teachers, 5, 109-132. Trent, S. C., Kea, C. D., & Oh, K. (2008). Preparing preservice educators for cultural diversity: How far have we come? Exceptional Children, 74(3), 328-350.

78 U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. (2016). The State of Racial Diversity in the Educator Workforce. Washington, DC: Author. Williams Shealey, M. (2006). The promise and perils of “scientifically based” research for urban schools. Urban Education, 41, 1-16. doi: 10.1177/0042085905282250 *Zion, S., & Sobel, D. M. (2014). Mapping the gaps: Redesigning a teacher education program to prepare teachers for inclusive, urban U.S. schools. Journal of the International Association of Special education, 15(2), 63-73. About the Authors Krystal Lewis-Pratl., is a doctoral candidate and Clinical Professor in Special Eduation at Illinois State University. Her research interests include culturally responsive pedagogy and teacher preparation. Yojanna Cuenca-Carlino, Ph.D., is an Assistant Vice President for Academic Administration and a Professor in the Department of Special Education at Illinois State University. Her teaching and research interests include culturally responsive practices, reading, writing and math interventions for students with mild disabilities, and professional development for faculty. April L. Mustian, Ph.D., is an Associate Professor and director of the B.S. in Special Education Program at Winthrop University. Her teaching and research interests include culturally responsive academic and behavioral interventions for students with and without disabilities, community-engaged teacher preparation, and implicit bias and its impacts on academic and discipline outcomes for students who have been historically minoritized in public schools.

79 Self-Monitoring Among Secondary and Post-Secondary Students with Intellectual and Developmental Disabilities: A Systematic Review Melvin S. Marsh, MS Stephanie M. Devine, PhD Georgia Southern University Abstract Individuals with intellectual and developmental disabilities (IDD) are going to college to earn certificates of completion in ever-increasing numbers in the pursuit of obtaining competitive employment (Cullen, Simmons-Reed, & Weaver, 2017). However, these students may need more support in order to reach their full potential and to survive and thrive independently as they transition to adulthood. Learning how to self-monitor is an important skill for success in college and employment. Unfortunately, there are significant gaps in the literature on self-monitoring for students with IDD who are attending post-secondary institutions. A preliminary search of the literature has revealed very few studies applicable to self-monitoring behaviors in college-aged individuals with IDD. This paper will reflect the results of a detailed search of the peer-reviewed literature surrounding this topic. Keywords: developmental disabilities, self-monitoring, intellectual disabilities, postsecondary, transition Self-Monitoring Among Secondary and Post-Secondary Students with Intellectual and Developmental Disabilities: A Systematic Review For those with intellectual disabilities and developmental disabilities (IDD), the future often does not consist of as many positive outcomes as peers who do not have such disabilities. This can be seen in many quality-of-life outcomes where those with disabilities are more likely to be overweight, exercise less, have more medical difficulties, and significantly lower incomes (Lobenius-Palmér, Sjöqvist, Hurtig-Wennlöf, & Lundqvist, 2018; Ptomey et al., 2017). In addition, those students with IDD are frequently unable to have the same educational opportunities as traditional students (Cullen et al., 2017). A post-secondary education, even if it does not lead to a completion of a degree, is often found to lead to a higher quality of life, including increased likelihood of employment as well as increased wages when employed (Cullen et al., 2017). To promote better postsecondary outcomes, many students with IDD are encouraged to attend postsecondary education such as inclusive college opportunities, of which, there are more than 290 programs across the United States (ThinkCollege.net). However, to complete a college program successfully, students need a high degree of self-determination. Poor outcomes for those with IDD are often linked to relatively low self-determination skills and self-management skills (Cullen et al., 2017). While self-determination skills can be challenging for anyone to learn, it is often more challenging for those with disabilities. Self-determination skills can include advocating for one’s self, setting one’s own goals, making decisions, and doing what they can to reach whichever goal they set for themselves. Cullen et al. (2017) suggest that the first step towards developing or improving self-determination skills often starts with improving and mastering independent completion of daily living skills. While other same-age peers without disabilities may be expanding their opportunities and increasing their independence as they reach adulthood, those with IDD may have fewer opportunities, if any at all, to participate in their

80 community or complete tasks independently (Bouck, Savage, Meyer, Taber-Doughty, & Hunley, 2014; Cannella-Malone, Brooks, & Tullis, 2013). Learned helplessness is a common side effect of others making decisions or completing tasks for another person, which is frequently a risk for individuals with IDD as they tend to have limited exposure to self-determination skill development (Cannella-Malone et al., 2013). One way for educators to help increase self-determination skills so individuals with IDD are able to have more opportunities to successfully transition into the real world may be in improving selfmonitoring skills. Self-monitoring is often considered a component of self-management (Clemons, Mason, Garrison-Kane, & Wills, 2016) and is defined as a method to change behavior through selfdirected observation and recording of one’s own behavior (Mace et al., 2001). Increased selfmonitoring can improve self-reliance and quality of life and decrease dependence on others (Bouck et al., 2014). Self-monitoring has also been shown to improve the academic skills of students in middle and high school in the areas of reading comprehension (Kozminsky & Kozminsky, 2001), assignment accuracy (Cancio, West, & Young, 2004), writing productivity (Trevino-Maack, Kamps, & Wills, 2015) and general on-task behavior (Faul, Stepensky, & Simonsen, 2012). Unfortunately, there is very little research relating to young adults with IDD as it pertains to improving their self-monitoring skills. Most recent research is related to the elementary age population both with and without IDD (Clemons et al., 2016; Lively, Myers, & Levin, 2019), which may be due to structural characteristics of high schools as opposed to elementary schools (Clemons et al., 2016). This gap in the literature needs to be explored because young adults with IDD are attending college at record high rates. Therefore, the purpose of this review is to consolidate quality self-monitoring studies found on secondary or postsecondary-aged students with IDD. While the focus is ideally the college-aged population, due to limited studies in this age range, this review includes the high school population for relevant studies. The purpose of this study is to review the findings in previous literature relating to self-monitoring technique use among high school students and young adults with IDD. The research questions include the following: 1. What systems (personnel, technology, software, etc.) are being used to support selfmonitoring skill development for adolescents or young adults with intellectual and/or developmental disabilities? 2. What skills are being targeted in these self-monitoring interventions? 3. How successful is the system/intervention? 4. What level of evidence is provided by the reviewed studies? Method A literature search was conducted in order to find relevant studies that might be appropriate to help young adults with IDD enrolled in inclusive post-secondary education programs (IPSE) in order to improve their self-monitoring skills. Due to limited research on this age group, this literature review included studies for participants ranging in age from 14 to 26. Eligibility Criteria

81 All studies included in this review met the following criteria: a) contained at least one participant who had a diagnosis of IDD between 14 and 26; b) completed single-case or group design study published in a peer-reviewed journal within the last 7 years; and c) included an independent variable with some form of technology-based self-monitoring technique or support. For the purposes of this review, the authors considered self-monitoring techniques to include any technologically based intervention that supported the participant observing and recording their own behavior. Due to the importance of current and relevant self-monitoring technology to inform practice, only studies published since 2013 were included. Search Strategy A literature search was conducted to identify peer-reviewed articles relating to adolescents and young adults with IDD in which a technology-based self-monitoring intervention was attempted. A multi-database search engine at the authors’ university was utilized in order to efficiently search many databases at once. This EBSCO Discovery Search engine included 97 peer-reviewed academic journal databases such as Academic Search Complete, APA PsychInfo, Child Development & Adolescent Studies, Directory of open access journals ERIC, JSTOR, Medline, ScienceDirect, Medline, the Psychology and Behavioral Sciences Collection, Sage Online, and Teacher Resource Center. Search terms were systematically created and applied using a variety of combinations to maximize the possible resources. The following keywords were used: intellectual disability, developmental disability, IDD, self-monitoring, technology, secondary, post-secondary, adult, grooming, hygiene, work tasks, employment tasks, peer or caregiver, goal setting. All results were peer-reviewed English language journal articles. The database searches resulted in a total of 35,856 articles. All records were imported into Endnote X9 for screening. Duplicates were identified by the software and the first author confirmed and deleted the duplicates. Titles and abstracts were then handscreened by the first author. If an abstract appeared to meet the inclusion criteria, the article was read to confirm relevance and included in the study. For questions in which the researcher was uncertain, a second researcher was utilized. The primary reason for exclusion was that the article did not relate to people with disabilities. The second reason was the study failed to include at least one person in the target age range. The tertiary reason was not including a self-monitoring intervention. After review, 27 studies were found to meet the inclusion criteria. Information about the articles was extracted and recorded, including the APA citation information, research problem, sample, independent and dependent variables, any instrumentation used, who provided support to the person with IDD, procedures, results, conclusions, any obvious weaknesses of the study, relevance, and major focus area. All of these were coded by the first author. Inter-rater reliability of 87% was collected when a second author extracted the same information for 10 randomly selected studies. Discrepancies were then discussed between the two authors and revised collectively. Of the 27 remaining studies, each was analyzed using the quality indicators provided by the National Technical Assistance Center on Transition (NTACT), either the single case or group experimental as appropriate (NTACT 2016a, 2016b). See Table 1 for quality indicators used. The primary author coded each study using the appropriate scale. The second author randomly selected 25% of articles to code. Inter-rater reliability for quality indicators was 75% overall agreement across all articles. Discrepancies were then discussed between the two authors until agreement was obtained and revised collectively. Table 1.

82 Criteria for Inclusion and Quality Assessment for Each Study Overall Characteristics  The dependent variable skill is related to postsecondary transition  Includes youth with disabilities between 11 and 26  Rigorous research design. For Single Case research, this would include reversal, multiple baseline, multiple probe, changing criterion, or alternating treatment Numbered Characteristics (each must be operationally described in such a way as other research teams could easily replicate) 1. Participant characteristics 2. Participant selection process 3. Setting features critical to the physical environment 4. Dependent variable (DV) 5. Quantifiable index measure procedure for each DV 6. Measurement process 7. Repeated measures of DV over time 8. Inter-observer agreement (IOA) and or reliability data were collected and met minimal standards (e.g., IOA = 80%) 9. Independent variable (IV) 10. Systematic manipulation of IV and demonstration of control of the experimenter 11. Fidelity of implementation was directly measured and reported 12. Baseline measures included repeated measurement of the DV, demonstrated a stable and predictable pattern (5 or more data points recommended) 13. Baseline procedures and conditions 14. A minimum of three demonstrations of experimental effect at different point in time 15. Common threats to internal validity were controlled in the design 16. External validity is established through the replication of the experimental effects across participants, settings, or materials. 17. Socially important DV 18. The size of the change from intervention is deemed socially important 19. The author states that the IV implementation is practical and cost effective 20. The social validity of the intervention is enhanced by extended implementation time periods, supported by typical intervention agents, and completed in typical physical and/or social contexts. High quality (Study meets all 20 indicators) Acceptable quality (Study meets indicators 1-16 and at least one of 17-20) Did not meet quality standards

2 13 13

Note. Based on the Quality Indicator Checklists for Group Experimental and Single Case Research Designs from the National Technical Assistance Center on Transition (2016). Results Of these studies, 14 met the criteria for acceptable or high-quality studies as provided by the NTACT Quality Indicators. All 14 were single-case research designs. A total of 12 were considered acceptable quality while 2 met the criteria for high quality studies. A summary of each of the 14 studies were included in Table 2.

Table 2. Characteristics from Acceptable Quality Studies Reference, Affiliation, and Quality Rating Bouck et al. (2014) Purdue University Acceptable

Dependent SM Technology/ Variable Intervention and Support

Setting / Participants

Research Design

Results

Food preparation : % of correctly completed steps in a 10-step recipe process with four ingredients

Comparison of high-tech and low-tech selfmonitoring systems: iPad with Upad App (notetaking app with photos) and paper/pencil, Researcher support, Verbal prompting

Public school, USA

Alternatin g treatment design

All students performed better with iPad than pencil and paper method selfmonitoring method

iPod Touch, InPromptu App (used to selfcheck steps in task)

Special school, USA

CannellaCleaning: Malone et al. % of (2013) correctly completed The Ohio steps to State Washing a University, table Ball State (Phase 1) University and teaching Acceptable themselves to vacuum (Phase 2)

Teacher support, Video prompting with error correction, Most-to-least prompts to teach iPod

13, female Two 15-year-old students (one male, one female) with ASD, ID, and language impairment. All Caucasian.

Multiple probe across 16, Caucasian participan male with CP and ts ID 16, African American male with DD. 15, male with DS, aphasia, patent ductus arteriosus. Race not specified. 17, female with ADHD, PDD. Race not specified.

All students improved task performance with video prompting in the stages they experienced. Of the students were maintenance was acquired, students maintained at 100%

84 CannellaMalone, Chan, and Jimenez (2017) The Ohio State University, Northern Illinois University

Vocational skills: % of steps completed correctly in collating, folding, copy in groups, copy doublesided tasks

iPod Touch, InPromptu App (used to selfcheck steps in task)

Classroom engagemen t On-task behavior

Samsung Galaxy Player 5.0, IConnect, Gymboss interval timer (used for self-check)

Teacher support, Least to most prompting, Video prompting with error correction

IPSE, USA

Adapted alternatin 28, male with g ASD, DD, treatment moderate ID, and design speech and language impairment. 26, DS, moderate ID, and a hearing impairment.

Both students improved their performance on the tasks they were assigned.

Public school, USA

ABAB singlesubject withdrawa l

All students improved their performance during intervention with maintenance ranging from 83% to 100%

multiple probe across participan ts

All students improved their performance during intervention, regardless of how many components the tasks had.

Acceptable Clemons et al. (2016) Missouri State University, University of Kansas

Researcher support, Verbal prompting to identify target behaviors, teaching technology, Error correction

Acceptable

Cullen et al. (2017) Ball State University, The Ohio State University Acceptable

General daily living:

iPad, MyPicsTalk App, Support not explicitly stated, Four types Self-directed of cleaning video prompting, tasks: Video created original task plus 3 variations

17, male with specific LD 17, male with ASD 15, female with ID

IPSE, USA 22, Caucasian male with ASD 20, Caucasian male with DS, ID, and mild visual impairment 24, Caucasian male with ID and TBI

85 English et al. Vocational Video monitoring (2017) skills (video prompting Gardening for one Monash skills participant) used University (Picking, as self-check planting, Acceptable quality Researcher control, support, bed Video creation marking, (researcher as hoeing, model) and labeling)

Non-profit, Australia

Gushanas and Thompson (2019)

IPSE, USA

Hygiene:

General online self-monitoring Reducing checklist Body Odor (SurveyMonkey)

All students successfully reduced their body odor using self-monitoring techniques.

Vocational iPhone, video skills: prompting

Public school USA

Making name tags, setting up coffee trays, wrapping boxes

21, female with moderate ID, DS, selective mutism, moderate hearing loss 21, male with moderate ID and DS

All students successfully learned and maintained at 100%, three of the four tasks with no student maintaining at 100% for the wrapping task

Researcher support

Acceptable

Trainings on the checklist

The Ohio State University Acceptable

Two participants acquired skills with video monitoring while one required video prompting. All maintained skills.

multiplebaseline 21, Caucasian acrossmale with CP and participan OI ts 19, Caucasian male with ADHD, EBD 22, Hispanic male with ASD 21, African American female with TBI 20, Caucasian male with unspecified ID

Texas A&M University

Heider, CannellaMalone, and Andzik (2019)

18, male with ASD 23, male with Asperger’s and social anxiety 23, male with ASD, anxiety, selective mutism, epilepsy

multiple probe design across skills

Researcher support, Self-directed prompting, Training on technology, Video creation

multiple probe across behaviors

86 Kellems, Rickard, Okray, Sauer-Sagiv, and Washburn (2018) Brigham Young University, Eugene 4J School District, University of Oregon, Springfield School District

General daily living, cleaning, food preparation Possible tasks included: Food – Making, Cleaning, Daily living – mailing a letter

iPad, VideoTote App to show video modelling of task Static picturebased task analysis (used as self-check), Support person not explicitly stated, Video creation, Technology training

Community living, USA 20, male with Fragile X Syndrome and ID 19, male with ASD and ID 20, female with ASD and ID

multiple probe across behaviors

All students improved their performance on tasks learned and maintained at between 80% to 100% depending on student and task.

multiple probe across participan ts

All students improved on task behavior, weeding, watering during intervention.

singlecase concurren t multiplebaseline design

All students improved on task behavior during intervention (70 to 91% average) and all maintained after intervention withdrawn (77 to 85%)

High Kim (2018)

Samsung mininetbook, video monitoring

Rehabilitation facility, Korea

Acceptable

Vocational skills: Gardening (Watering and Weeding), On-task behavior

Support Person not explicitly stated although there was a staff model

Three Korean males (ages 13 to 14) with ASD and language impairment

Lively et al. (2019)

On task behavior

Vibrating watch (to alert student for on-task behavior)

Juvenile justice, USA

Kim, Lotus Flowers Children Center

Texas Woman’s University, University of WisconsinMadison, University of Arizona High

Teacher support Training on the technology Training on intervention specific paperwork

15, male with EBD 16, male with no documented disability 16, male with a specific LD and ADHD

87 Mackey and Nelson (2015)

Vocational skills; hygiene, active University of engagemen St Joseph t, decision making, Acceptable appropriate interaction s with others, and transitions

Video Feedback (self-evaluation of video of behavior)

Wu, CannellaMalone, Wheaton, and Tullis (2016)

iPod Touch, iMovie (to provide video prompting as a self-check), Auxiliary speaker Researcher

National Kaohsiung Normal University, The Ohio State University, Ball State University Acceptable

Cleaning Table washing, Window washing

Job Coach

Transition Academy USA

withinparticipan t multipleprobe Two, 19-year-old design identical across Caucasian male behaviors, twins with ASD and ID

All desired behaviors increased during intervention 1 except for those that were near the ceiling. Very few changes occurred between intervention 1 and 2.

Public school USA

Video prompting with error correction effectively taught two daily living/workplace skills which were generalized. Video prompt fading was more effective within the intervention.

multiple probe across 14, nonverbal participan male student with ts design ASD and ID with a 17, male with reversal Prader-Willi design Syndrome and ID

88 Yakubova and TaberDoughty (2017) Duquesne University, Purdue University Acceptable

Vocational skills Decision making skills

iPad with pointof-view video modelling Teacher, Video creation, Cue sheet used as Missing a self-check, material, Technology inappropri- training ate material, insufficient material

Public school USA 20, Caucasian male with ASD and communication deficits 19, Native American with ASD 17, Caucasian male with ASD 19, Caucasian male with ASD

multiple probe across participan ts

Students improved from their baseline scores with the intervention with significant stimulus generalization. Maintenance varied with scores ranging from 57.2% to 100%, with one student decreasing trend and the other participants staying steady or having a positive trend.

Note. SM = self-monitoring; ASD = Autism Spectrum Disorder; ID = Intellectual Disability; DD = Developmental Disability; CP = Cerebral Palsy; DS = Down Syndrome; ADHD = Attention Deficit Hyperactivity Disorder; PDD = pervasive developmental disorder; EBD = Emotional Behavioral Disorder; LD = Learning Disability; TBI = Traumatic Brain Injury; OI = Orthopedic Impairment. Study Characteristics The primary aim of the studies was to teach forms of self-monitoring in order to improve a variety of skills, including general daily living skills as well as specific skills such as cleaning and food preparation. In addition, improving either work related behaviors or improving vocational skills were also often targets for intervention. Other topics included improving personal hygiene, social skills, improving classroom engagement, and improving on task behaviors in general. Technology use Technology used for the self-monitoring behavior primarily included the use of an iPad (Bouck et al., 2014; Cullen et al., 2017; Kellems et al., 2018; Yakubova & Taber-Doughty, 2017), with 12 students of 42 students (29%) using this technology. Other tablets included a Samsung Galaxy Player 5.0 tablet (Clemons et al., 2016) and a Samsung mini-netbook (Kim, 2018). Other common technology included the iPod Touch (Cannella-Malone et al., 2017; Wu et al., 2016) or an iPhone (Heider et al., 2019). Specific apps included Upad App (Bouck et al., 2014), MyPicsTalk App (Cullen et al., 2017), InPromptu App (Cannella-Malone et al., 2017), and I-Connect selfmonitoring software (Clemons et al., 2016) (see Table 3 for more information). In addition, several forms of video modeling (English et al., 2017; Kim, 2018; Yakubova & Taber-Doughty, 2017), video prompting (Heider et al., 2019), and video feedback (Mackey & Nelson, 2015) were used. Some used a more general online checklist (Gushanas & Thompson, 2019). Other studies also used a Gymboss interval timer (Clemons et al., 2016) and a vibrating watch (Lively et al., 2019) to remind participants to check their behavior at various intervals.

Table 3. Description of Applications Used Application name UPAD for iCloud My Pictures Talk InPromptu I-Connect

Description A productivity and note-taking app by PockeySoft. A video modelling tool used to develop social stories and more using video, pictures, and text. By Grembe Apps. An app with video prompts already created or that can be developed for daily living skills tasks. By Ohio State University Department of Education. A self-monitoring mobile app that can be set up for individualized goals. The app includes interval timers and reminders for specific tasks as well as a data collection tool for students and teachers or service providers. By the University of Kansas.

Intervention supports Initial instruction on and continuing use support is usually needed for individuals with IDD when developing self-monitoring skills (Heider et al., 2019). In the reviewed studies, the support was provided by the researcher (Bouck et al., 2014; Clemons et al., 2016; English et al., 2017; Gushanas & Thompson, 2019; Heider et al., 2019; Wu et al., 2016), teacher (Cannella-Malone et al., 2013; Cannella-Malone et al., 2017; Lively et al., 2019; Stauch, Plavnick, Sankar, & Gallagher, 2018; Yakubova & Taber-Doughty, 2017), a job coach (Mackey & Nelson, 2015), other observers/peers (Gushanas & Thompson, 2019), a staff model (Kim, 2018), or the specific support was either not listed or not explicitly stated (Cullen et al., 2017; Kellems et al., 2018; Kim, 2018). Specific supports provided included verbal prompting (Bouck et al., 2014; Clemons et al., 2016; English et al., 2017; Wu et al., 2016), video prompting with error correction (Cannella-Malone et al., 2013; Cannella-Malone et al., 2017; Clemons et al., 2016; Wu et al., 2016), video monitoring (English et al., 2017; Kim, 2018), most-to-least prompting (Cannella-Malone et al., 2013), least-tomost prompting (Cannella-Malone et al., 2017), technology training (Cannella-Malone et al., 2013; Clemons et al., 2016; Heider et al., 2019; Kellems et al., 2018; Lively et al., 2019; Yakubova & Taber-Doughty, 2017), self-directed prompting (Cannella-Malone et al., 2013; Cannella-Malone et al., 2017; Cullen et al., 2017; Heider et al., 2019), prior video creation (Cullen et al., 2017; English et al., 2017; Heider et al., 2019; Kellems et al., 2018; Kim, 2018; Yakubova & Taber-Doughty, 2017), survey creation (Gushanas & Thompson, 2019), trainings on intervention paperwork (Gushanas & Thompson, 2019; Lively et al., 2019), and a cue sheet (Yakubova & Taber-Doughty, 2017). Research design All included studies used some form of single-case research design. This included one multipleprobe across skills, two multiple-probe across behaviors, four multiple-probe across participants, a multiple-probe across participants with a reversal design, and a within-participant multiple-probe across behaviors. A multiple baseline across-participants design and a single-case concurrent multiple-baseline design were also performed. Other studies implemented an alternative design, an adapted alternative design, and an ABAB single-subject withdrawal design. For more detailed information, please see Table 3. Location and setting A total of 12 studies were conducted in the United States. In addition, one study each was found in Australia and Korea. Of these studies, three were performed in an IPSE setting, one in a non-

90 profit, one community living setting, one transition academy, one rehabilitation facility, and one juvenile justice facility. The remaining studies were performed in a public-school classroom setting. Participant characteristics In total, there were 42 participants across all 14 studies. The average age of participants was 18.5 (range 13 to 28) with only one person being outside the target age range. A total of 6 participants were female and 36 were male. There was not enough demographic data provided in the studies to include a breakdown of race. The most common disability diagnosis under the umbrella of IDD was ASD, with 20 students (58%) having this disorder with or without intellectual disability and 8 exclusively having this diagnosis. In addition, five participants were diagnosed with Down Syndrome. A total of two participants had an unspecified intellectual disability. One participant had no disability, but the other participants in the study had a range of disabilities, some of which fall under the umbrella of IDD, including ADHD and Prader Willi Syndrome. Due to the small sample size of most of the studies, attrition was low with most not having any participants removed from the study. Discussion and Conclusions This systematic review highlighted the state of the literature as it pertains to self-monitoring interventions for secondary and post-secondary students with intellectual and developmental disabilities. There were a wide variety of goals of the self-monitoring studies as well as a variety of different disabilities that were considered. In total, 27 studies were analyzed with 12 meeting acceptable standards and 2 meeting high quality standards for research. These are further discussed in the following section. Cooper et al. (2007) define self-monitoring as a procedure where an individual systematically observes and records when a behavior occurs or does not occur. NTACT, Rowe et al. (2020) lists self-monitoring, within which they also include the terms self-observation and self-recording, as a research-based practice for transition-age students (high school to college) with specific learning disabilities. This literature review included video modelling and video prompting as applicable self-monitoring interventions when participants were required to use the video prompting or modelling as a self-check while completing tasks. The results of this review, which includes 14 methodologically rigorous studies, 42 participants with primarily positive functional relationship, and over 20 different professional affiliations, suggest that technology-supported self-monitoring meets the level of evidence needed to be considered an evidence-based practice by the NTACT (2018). All study participants improved their baseline performance during the self-monitoring intervention. In addition, most students learned the skill at a high level that maintained over time. Further, some students were able to generalize their learned skills to other tasks. These results show promise for the use of technology-based self-monitoring techniques to support independent skill development for adolescents and young adults with IDD. However, there is still more work to be considered in this area. Currently, much of the research on self-monitoring focuses on elementary school students. There are only a few studies on the selfmonitoring of post-secondary students with disabilities. One question that may need to be asked is why are there so few secondary and post-secondary studies in this research in comparison to elementary school aged population? Focusing on post-secondary school can be valuable as the percentage of students with disabilities on college campuses have doubled or tripled in the past 25 years (Francis et al., 2018). Despite this increase, only 20% of students with disabilities graduate from a four-year college (Francis et al., 2018). Self-monitoring training may help improve skills

91 and allow students to overcome some obstacles to success as they continue in life and potentially in their education. Another consideration for future research would be to improve the power of these self-monitoring studies, which would make this strategy more generalizable to other secondary and post-secondary students with disabilities. One of the biggest challenges with these studies has been that these were single case designs with small numbers of participants. Unfortunately, due to the inability to randomly assign participants to having a disability or not, the low percentage of the population with an IDD, as well as potential challenges with acquiring enough funding to acquire a larger sample size, single case designs, with their lower sample sizes, are often the most appropriate selection. In order to overcome the challenge that small studies provide, future researchers need to replicate these studies in both similar and different settings (classroom, workplace, etc.), locations (inner city, rural, different countries), different participants, in different settings, and with different technologies, and researchers. Potentially, it may be helpful to look at as many specific disabilities, both with and without comorbidities in order to get a broader perspective on the use of selfmonitoring tools. Further, the studies performed will need to be of higher quality than many studies found in the current literature. During the quality assessment, 13 of the 27 articles selected (48%) did not meet minimum standards and thus were not counted. In order to meet acceptable standards, NTACT (2016b) requires a study to meet items 1 through 16 on their quality indicators checklist (see Table 1) and at least one quality indicator from items 17 through 20. High quality studies met all 20 criteria. Of the studies disqualified, all were disqualified due to not meeting the first 16 quality indicators. Nine did not meet by one quality indicator while the remaining missed two or more. The most common quality indicator missed, by 7 of 13 disqualified studies, was including enough significant detail in the setting in order to recruit similar participants for replication by another researcher. The second most common indicator was controlling for threats to external validity, this was missed by four studies. If additional acceptable and high-quality studies are performed, this would better lend more support for the use of self-monitoring to help students with IDD succeed. The use of video monitoring and video prompting as a self-check were the most common in the reviewed studies. Due to the time constraints of video production, it may be worthwhile to consider other self-monitoring technologies that do not rely so heavily on videos and video creation. Some self-monitoring goals, such as on-task behavior and asking whether one meets a pre-identified step in a task analysis, might be easier to perform without the time and expense of preparing videos, which should be a consideration in future research. Lastly, most studies in this review were supported by the researcher, teacher, or job coach. Peerassisted instruction is considered a research-based and promising practice for transition-aged students (NTACT, 2019). For postsecondary students with IDD, the use of peers to reinforce or implement self-monitoring skill instruction provides an opportunity for more natural supports that lead to less stigma for students with IDD, particularly in a college setting. More research on the use of classmates or peers to help with implementing self-monitoring interventions is needed. References Bouck, E. C., Savage, M., Meyer, N. K., Taber-Doughty, T., & Hunley, M. (2014). High-Tech or Low-Tech? Comparing Self-Monitoring Systems to Increase Task Independence for Students With Autism. Focus on Autism and Other Developmental Disabilities, 29(3), 156167. doi:10.1177/1088357614528797

92 Cancio, E. J., West, R. P., & Young, K. R. (2004). Improving Mathematics Homework Completion and Accuracy of Students with EBD Through Self-Management and Parent Participation. Journal of Emotional & Behavioral Disorders, 12(1), 9-22. doi:10.1177/10634266040120010201 Cannella-Malone, H. I., Brooks, D. G., & Tullis, C. A. (2013). Using Self-Directed Video Prompting to Teach Students with Intellectual Disabilities. Journal of Behavioral Education, 22(3), 169-189. doi:10.1007/s10864-013-9175-3 Cannella-Malone, H. I., Chan, J. M., & Jimenez, E. D. (2017). Comparing self-directed video prompting to least-to-most prompting in post-secondary students with moderate intellectual disabilities. International Journal of Developmental Disabilities, 63(4), 211-220. doi:10.1080/20473869.2017.1301695 Clemons, L. L., Mason, B. A., Garrison-Kane, L., & Wills, H. P. (2016). Self-Monitoring for High School Students With Disabilities: A Cross-Categorical Investigation of I-Connect. Journal of Positive Behavior Interventions, 18(3), 145-155. doi:10.1177/1098300715596134 Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River, N.J.: Pearson/Merrill-Prentice Hall. Cullen, J. M., Simmons-Reed, E. A., & Weaver, L. (2017). Using 21st century video prompting technology to facilitate the independence of individuals with intellectual and developmental disabilities. Psychology in the Schools, 54(9), 965-978. doi:10.1002/pits.22056 English, D. L., Gounden, S., Dagher, R. E., Chan, S. F., Furlonger, B. E., Anderson, A., & Moore, D. W. (2017). Effects of video modeling with video feedback on vocational skills of adults with autism spectrum disorder. Developmental Neurorehabilitation, 20(8), 511-524. doi:10.1080/17518423.2017.1282051 Faul, A., Stepensky, K., & Simonsen, B. (2012). The Effects of Prompting Appropriate Behavior on the Off-Task Behavior of Two Middle School Students. Journal of Positive Behavior Interventions, 14(1), 47-55. Retrieved from Francis, G. L., Duke, J., Brigham, F. J., & Demetro, K. (2018). Student Perceptions of CollegeReadiness, College Services and Supports, and Family Involvement in College: An Exploratory Study. Journal Of Autism And Developmental Disorders, 48(10), 3573-3585. doi:10.1007/s10803-018-3622-x Gushanas, C. M., & Thompson, J. L. (2019). Effect of Self-Monitoring on Personal Hygiene Among Individuals With Developmental Disabilities Attending Postsecondary Education. Career Development and Transition for Exceptional Individuals, 42(4), 203-213. doi:10.1177/2165143418781297 Heider, A. E., Cannella-Malone, H. I., & Andzik, N. R. (2019). Effects of Self-Directed Video Prompting on Vocational Task Acquisition. Career Development and Transition for Exceptional Individuals, 42(2), 87-98. doi:10.1177/2165143417752901 Kellems, R. O., Rickard, T. H., Okray, D. A., Sauer-Sagiv, L., & Washburn, B. (2018). iPad® Video Prompting to Teach Young Adults With Disabilities Independent Living Skills: A Maintenance Study. Career Development and Transition for Exceptional Individuals, 41(3), 175-184. doi:10.1177/2165143417719078 Kim, J. (2018). Effects of point-of-view video modeling for Korean adolescents with autism to improve their on-task behavior and independent task performance during vegetable gardening. International Journal of Developmental Disabilities, 64(4-5), 297-308. doi:10.1080/20473869.2017.1341449 Kozminsky, E., & Kozminsky, L. (2001). How do general knowledge and reading strategies ability relate to reading comprehension of high school students at different educational levels? Journal of Research in Reading, 24(2), 187. doi:10.1111/1467-9817.00141 Lively, R., Myers, D., & Levin, J. R. (2019). Using Self-Monitoring to Support Student Behavior in a Juvenile Justice Facility. Journal of Correctional Education, 70(1), 36-52.

93 Lobenius-Palmér, K., Sjöqvist, B., Hurtig-Wennlöf, A., & Lundqvist, L.-O. (2018). AccelerometerAssessed Physical Activity and Sedentary Time in Youth With Disabilities. Adapted Physical Activity Quarterly, 35(1), 1-19. Mackey, M., & Nelson, G. (2015). Twins with autism: utilising video feedback to improve jobrelated behaviours. British Journal of Special Education, 42(4), 390-410. doi:10.1111/1467-8578.12107 National Technical Assistance Center on Transition (2016a). Quality Indicator Checklist: Group Experimental. Retrieved from https://transitionta.org/system/files/effectivepractices/Quality%20Indicator%20Checklist_G roup_11-04-16%20(2).pdf?file=1&type=node&id=1131&force= National Technical Assistance Center on Transition (2016b). Quality Indicator Checklist: Single Case Retrieved from https://transitionta.org/system/files/effectivepractices/Quality%20Indicator%20Checklist_Si ngle%20Case_11-04-16%20(1).pdf?file=1&type=node&id=1132&force= National Technical Assistance Center on Transition. (2018). Introduction to NTACT Criteria for Levels of Evidence. Retrieved from https://transitionta.org/system/files/effectivepractices/NTACT%20Criteria%20for%20Level s%20of%20Evidence_Final_Spring2018.pdf?file=1&type=node&id=1473&force= National Technical Assistance Center on Transition, Rowe, D. A., Alverson, C. Y., Kwiatek, S., & Fowler, C. F. (2020). Effective Practices in SecondaryTransition: Operational Definitions. Retrieved from https://www.transitionta.org/system/files/resourcetrees/Description%20of%20EBPs_02Feb 2020.pdf?file=1&type=node&id=1985&force= Ptomey, L. T., Willis, E. A., Lee, J., Washburn, R. A., Gibson, C. A., Honas, J. J., & Donnelly, J. E. (2017). The feasibility of using pedometers for self-report of steps and accelerometers for measuring physical activity in adults with intellectual and developmental disabilities across an 18-month intervention. Journal of Intellectual Disability Research, 61(8), 792801. doi:10.1111/jir.12392 Stauch, T. A., Plavnick, J. B., Sankar, S., & Gallagher, A. C. (2018). Teaching social perception skills to adolescents with autism and intellectual disabilities using video-based group instruction. Journal of Applied Behavior Analysis, 51(3), 647-666. doi:10.1002/jaba.473 Trevino-Maack, S. I., Kamps, D., & Wills, H. (2015). A Group Contingency Plus SelfManagement Intervention Targeting At-Risk Secondary Students' Class-Work and Active Engagement. Remedial and special education : RASE, 36(6), 347-360. doi:10.1177/0741932514561865 Wu, P.-F., Cannella-Malone, H. I., Wheaton, J. E., & Tullis, C. A. (2016). Using Video Prompting With Different Fading Procedures to Teach Daily Living Skills: A Preliminary Examination. Focus on Autism and Other Developmental Disabilities, 31(2), 129-139. doi:10.1177/1088357614533594 Yakubova, G., & Taber-Doughty, T. (2017). Improving Problem-Solving Performance of Students With Autism Spectrum Disorders. Focus on Autism and Other Developmental Disabilities, 32(1), 3-17. doi:10.1177/1088357615587506 About the Authors Melvin S. Marsh, M.S. is a second-year graduate student in the Experimental Psychology program at Georgia Southern University. His research interests are varied but include drama therapy for those with intellectual disabilities, neuropsychology, as well as hypnosis and suggestibility.

94 Stephanie M. Devine, PhD is an Assistant Professor of Special Education and the Executive Director of EAGLE Academy an inclusive postsecondary education program for young adults with intellectual and developmental disabilities at Georgia Southern University. Her research focuses on the use of Universal Design for Learning, technology, and peer mentors to support successful postsecondary transition and competitive integrated employment for individuals with intellectual and developmental disability.

95 Pre-Service Teachers’ Noticing of Struggling Students’ Mathematical Thinking Emily N. Rutherford, EdD. Midwestern State University Dittika Gupta, PhD. Midwestern State University Abstract We all notice the same things differently. This subjective nature of “noticing”, especially in student work, requires explicit attention in teacher preparation programs to prepare Pre-Service Teachers (PSTs) for making sound instructional decisions for the students. This becomes even more critical for assessing the work of students who have or are suspected of having learning disabilities that cause them to struggle with mathematical skills. This research study was aimed at providing PSTs with a focused class activity aimed at developing skills to recognize, identify, and make next-step instructional decisions in their teacher preparation programs, specifically for differentiating instruction for struggling students, including students with disabilities. Pre and post surveys were analyzed to gauge the impact of a focused lesson on developing PSTs “noticing” of children's work samples. Key words: Mathematics, Struggling Learners, Pre-Service Teachers, Noticing, Pre-Service Teachers’ Noticing of Struggling Students’ Mathematical Thinking Teachers continuously collect data on every student in their classroom, however the question of how well they are able to analyze data effectively to inform the teaching process is always a struggle and questionable. Many researchers (Sherin, 2007; Van Es & Sherin, 2008) have explored strategies to enhance teachers’ learning to notice and develop this skill as it directly influences classroom instruction. Improving teachers’ ability to analyze the data’s effect on instruction and the ability to analyze the data then use it to revise instruction is imperative to having quality instruction in the classroom (Morris, 2006; Saunders & Goldenberg, 2005). The ability to adequately examine student work often takes years of practice and hence these skills are more than likely underdeveloped or not developed at all, especially in PSTs (Morris, 2006). PSTs need to be exposed to opportunities to develop this skill because, as future teachers, their beliefs and awareness will directly affect classroom teaching, In general, PSTs are prepared to plan appropriate instruction for students who perform on or above grade level, but not well prepared to instruct struggling students or students with disabilities (Polloway, Patton, & Serna, 2008). Hence, there is a greater need to engage PSTs in activities that will develop their understanding of planning instruction for struggling students. In this paper, we discuss results from a pilot intervention undertaken by a mathematics education professor and a special education professor with the motivation to support and develop PSTs learning about noticing struggling children's mathematical thinking and making next-step instructional decisions. Children’s Noticing of Mathematical thinking Star and Strickland (2007) state that teachers are the key for the learning process as it is a teacher’s acts that lays foundation of present and future learning. Jacob and Empson (2015) call for teachers to use “responsive teaching, a type of teaching in which teachers’ instructional decisions about what to pursue and how to pursue are continually adjusted in response to children’s contentspecific thinking” (p. 187). A framework that include, “(a) attending to children’s strategies, (b) interpreting children’s understanding, and (c) deciding how to respond on the basis of children’s

96 understanding” (Jacobs, Lamb, & Phillip, 2010). This framework can provide a lens through which teachers develop their skills to notice and respond effectively for all children. Considering the importance and need to develop the noticing skill, it is crucial that PSTs, who are training to be future teachers, be engaged in activities that provides opportunities for them to experience and develop an understanding of the skill especially for making instructional decisions for struggling learners. Struggling Learners Math has always been a subject of great challenge for all students, especially for students who are struggling learners. The No Child Left Behind Act (NCLB) of 2001 requires that all students mastering the general education curriculum be administered standardized assessments and should perform at a proficient or passing level of performance similar to their nondisabled peers. This requires schools to provide inclusive educational opportunities to students with disabilities. Hence, a classroom teacher should not only be make instructional decisions for students who are at grade level but also need to differentiate instruction for struggling students. In spite of the push for inclusive mathematical education for students who struggle academically, these students continue to fall further and further behind their peers on the achievement continuum (Tan & Thoriusm 2018). Students who struggle are traditionally given mathematical content that is of, “low rigor”, even though research suggests these students can participate in higher-rigor and more challenging grade-level mathematical content (Hostins & Jordão, 2015). This may be due to many general education teachers having little or no special education training in their teacher preparation programs but still have the duty of teaching struggling students. Jung, Cho, & Ambrosetti (2011) calls for a need for pre-service training in working with students who are struggling learners in the classroom in order to boost teacher confidence, improving learning outcomes. Chester and Beaudin (1996) implied that several different factors could affect interactions between teachers and students struggling students. These include, time given for collaboration, mentoring, staff evaluation procedures, students’ strengths and individual needs, and resources available to teachers in the general education classroom (Chester and Beaudin, 1996). This particular research study also indicated that preservice teachers have a crucial need for training in teaching struggling students and this training is a major component in determining the teachers’ attitudes towards working with these students. One of the first steps toward assisting PSTs in providing successful inclusive mathematical classrooms is to provide them with activities that assists them in planning instruction, understanding remediation skills, and assisting students with disabilities in a positive and productive mathematical inclusive experience. Providing these experiences can support PSTs, develop mathematical understanding needed to meet the diverse learning needs of struggling learners placed in their classrooms. Additionally, the experiences will also allow PSTs to leave their teacher preparation programs with skills necessary to meet the unique learning needs of all students in their classrooms and to establish needed support for inclusive programs in the future. Hence, the study explored the following research question: What is the impact of a focused lesson aimed at developing PSTs “noticing” skills on their interpretation of struggling students work? Methodology Participants and Setting The participants in the study were college students within the age group of 19 years to 25 years. The participant group consisted of both males and females with different demographics, majority of them being Caucasian females, majoring in education. Prior to the onset of the data collection,

97 all participants were provided consent forms and explained the purpose of the study. There was no penalty applied to students who did not wish to participate. Only students who returned their signed consent form, in compliance with Institutional Review Board requirements, were allowed to participate in the study. Participants of this study thus consisted of 16 undergraduate students currently enrolled in the college of education at a four-year university in North Texas. Research Design and Analysis The research study used descriptive statistics with a pre- and post- 1-6 Likert scale survey to examine the impact of the intervention. The intervention was a focused lesson on developing PSTs’ understanding of struggling students’ mathematical thinking and was implemented in phases. The PSTs analyzed real student work samples in whole groups through guided practice and independently. Pre and post survey was conducted before and after the focused lesson. Since the student work samples used were real students from a neighboring local school district, the PSTs were provided an authentic learning experience. Moreover, two experts purposefully chose the work samples through analysis: math education and special education faculty that would best facilitate PSTs learning and understanding. Discussions The responses received from the pre and post surveys are summarized in Figure 1 and Figure 2 below. The response frequency reveals that even though PSTs were comfortable in planning instruction for struggling students, their understanding of the nuances of preparation for instruction developed. The post survey responses, being positively skewed, indicated that the class activity had an impact on PSTs’ confidence in their ability to plan instruction for struggling students. The openended question, “What information can a student work sample give you?” further supports that engagement in the noticing lesson had a positive impact PSTs. Evidence is provided by the fact that 4 out of 16 students indicated that they did not have any ideas about what information can be gathered from student work samples. Eleven PSTs could only provide superficial ideas about what information can be collected from student work samples. Only one PST was able to provide a specific example of, “creating an IEP”. In contrast, all PSTs provided specific examples, such as “misconceptions”, “varied ways to solve problems”, “student errors”, “learning levels”, “common mistakes”, and “strengths and weaknesses” in the post-survey. Furthermore, on the open-ended question, “Do you think looking at student work is important?”, every pre-survey response was the same, “agree” that it is important; however, it was on the post-survey that some PSTs indicated the importance of using student work as an important tool for planning future instruction. Hence, the descriptive statistics and open-ended questions provide evidence that teacher preparation courses should purposefully design lessons that provide PSTs with opportunities to participate in authentic learning using real student work.

Figure 1. Comfort levels of PSTs in planning instruction for struggling students.

Figure 2. PSTs confidence in building instruction based on students’ needs.

100

Limitations and Future Work The present study extends our knowledge of PSTs learning about noticing struggling children's mathematical thinking and making next-step instructional decisions. However, the current research also had several limitations. First, due to purposeful sampling, it could be said that there was not equal representation of respondents from all geographical areas. Second, since this study was conducted in the university classroom, the data reflect only PSTs’ perspectives of instruction, which may not reflect actual practice. No doubt PSTs’ confidence in planning instruction for struggling students and students with disabilities can be increased by introductions and opportunities to practice strategies, such as noticing, further collection of data on PSTs work can provide insight into how to best develop the skill of noticing in the classroom. Preservice teacher preparation programs need to be restructured to increase the amount of information provided on the learning needs of included students and pedagogical practices for all students, including students with disabilities. Pre-service teacher training that focuses on particular mathematics topics and strategies for teaching students with learning disabilities appear to be necessary to increase comfort with teaching students with disabilities in an inclusive setting. The researchers recommend using research based practices such as noticing framework, especially in courses designed to develop instruction for struggling students. References Act, N. C. L. B. (2002). No child left behind act of 2001. Publ. L, 107-110. Chester, M. D., & Beudin, B. Q. (1996). Efficacy beliefs of newly hired teachers in urban schools. American Educational Research Journal, 33, 233-257. Forlin, C., Hostins, R. C. L., & Jordão, S. G. F. (2015). School inclusion policy and curricular practices: Teaching strategies for the conceptual preparation of the target public of special education. Education Policy Analysis Archives, 23(28), 1–19. https://doi.org/10.14507 /epaa.v23.1661 Jacobs, V. R., Lamb, L. L., & Philipp, R. A. (2010). Professional noticing of children's mathematical thinking. Journal for Research in Mathematics Education, 169-202. Jung, W., Cho, G., & Ambrosetti, D. (2011). Preservice teachers' confidence levels in working with students with special needs: Improving preservice teacher training programs. Electronic Journal for Inclusive Education, 2(7), 7. Morris, A. K. (2006). Assessing pre-service teachers’ skills for analyzing teaching. Journal of Mathematics Teacher Education, 9(5), 471-505. Polloway, E. A., Patton, J. R., & Serna, L. (2008). Strategies for teaching learners with special needs. Prentice Hall. Saunders, W., & Goldenberg, C. (2005). The contribution of settings to school improvement and school change: A case study. Culture and context in human behavior change: Theory, research, and applications, 127-150. Tan, P., & Thorius, K. K. (2018). En/countering inclusive mathematics education: A case of professional learning. Mathematics Teacher Educator, 6(2), 52-67. Van Es, E. A., & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10(4), 571-596. About the Authors

101 Emily Rutherford, Ed.D. is an assistant professor in the West College of Education at Midwestern State University. Dr. Rutherford serves as the Special Education Program Coordinator as well as the Graduate Coordinator for her college. She has spent over fifteen years working in public schools as a teacher, educational diagnostician, special education administrator, and as a university professor. Dr. Rutherford presents at regional, state, and national conferences on autism, learning disabilities, teaching special education and other related topics. Dittika Gupta, Ph.D., is currently an Associate Professor in the West College of Education at Midwestern State University where she teaches mathematics methods courses and supervises preservice teachers in the field. In addition to that, she also teachers research courses to graduate students. Dr. Gupta is a former public high school teacher and her research interests include teacher preparation, integrating reading in math, professional noticing of children’s mathematical thinking, and lesson study.

102 Intellectual Development and Specific Learning Disability: The Role of Norm-Referenced Tests Edward K. Schultz, Ph.D. Emily Rutherford, Ed.D. Dennis Cavitt, Ed.D Midwestern State University Abstract This article describes the construct of “intellectual development”, and suggests how to use normreferenced tests (NRT) more effectively when identifying specific learning disabilities (SLDs). Intellectual development is defined as both a policy construct and in practical terms. Five assumptions concerning individualized NRT are outlined with supporting data; the best application of NRT is also explained. Keywords: intellectual development, specific learning disability, norm-referenced tests, test interpretation Intellectual Development and Specific Learning Disability: The Role of NormReferenced Tests The most accurate method to identify specific learning disability (SLD) has been deliberated in the professional literature since the inception of the SLD construct and has continued following the 2004 reauthorization of IDEA. This debate has intensified in recent years due to the controversy over methods that use a pattern of strengths of weaknesses (PSW; Decker et al. 2013; Flanagan & Schneider, 2016; Fletcher & Mikiak, 2019; Kransler et al,; 2016; 2020; Mather & Gregg, 2006; McGill & Busse, 2016; Mikiak et al. 2015; Schneider & Kaufman, 2017; Schultz & Stephens-Picesso, 2018; Stuebing et al., 2012 ). While many of the components of the SLD identification process are not up for debate (e.g., exclusionary factors, lack of appropriate instruction, procedural safeguards, disorder in one or more psychological processes), sharp division remains regarding the role of cognitive tests in the identification process (Fletcher & Miciak, 2017; Hale et al., 2010, Kranzler et al., 2019; Scheider & Kaufman, 2017; Stuebing, et al., 2012). The 2004 IDEA statute references “intellectual development” when applying a PSW model and it is impossible to claim “comprehensiveness” without assessing this construct. The overall purpose of this paper is to advocate for the appropriate use of “cognitive tests” in the identification of SLD using a PSW, specifically when examining the intellectual development of students suspected of having SLD. To address the inadequacies in the literature and limitation of the most common methods of SLD identification, Schultz and Stephens-Picesso (2018) proposed the use of the Core-Selective Evaluation Process (CSEP) to identify SLD. This method adheres to current policy, incorporates

103 teaching and learning, informal data, as well as appropriate use of tests of cognition to measure, pattern of strengths and weaknesses relative to intellectual development. Schultz and StephensPicasso (2018) described the complexity of the SLD construct and the focus on the “cognitive aspects” of the disability, the reliance on standard score discrepancy methods, and little attention to the use of informal data and the influence of language. In addition, SLD identification must be considered within the policy constraints of state education agencies (SEAs) and local educational agencies (LEAs) and most importantly considered in the context of teaching and learning. This paper will further investigate the construct of “intellectual development” and suggest how to use norm-referenced tests (NRTs) more effectively when identifying SLD. Policy and Practical Construct The term “intellectual development” is mentioned in the 2006 Code of Federal Regulations (CFR) 29 times, specifically as part of “third method” approaches: § 300.309(a)(2)(i), or the child exhibits a pattern of strengths and weaknesses in performance, achievement, or both, relative to age. State-approved grade level standards or intellectual development consistent with §300.309(a)(2)(ii). Several commenters requested intellectual development (ID) be defined and clarified. Excerpts of the regulations below will highlight the concerns of the commentators and response from the Department of Education. The reader is encouraged to read the full CFR for a broader context. Comment: Several commenters requested that the regulations include a definition of ‘‘intellectual development.’’ Discussion: We do not believe it is necessary to define ‘‘intellectual development’’ in these regulations. Intellectual development is included in § 300.309(a)(2)(ii) as one of three standards of comparison, along with age and State-approved grade-level standards. The reference to ‘‘intellectual development’’ in this provision means that the child exhibits a pattern on strengths and weaknesses in performance relative to a standard of intellectual development such as commonly measured by IQ tests. Use of the term is consistent with the discretion provided in the Act in allowing the continued use of discrepancy models. Comment: Several commenters stated that intra-individual differences, particularly in cognitive functions, are essential to identifying a child with an SLD and should be included in the eligibility criteria in § 300.309. Discussion: As indicated above, an assessment of intra-individual differences in cognitive functions does not contribute to identification and intervention decisions for children suspected of having an SLD. The regulations, however, allow for the assessment of intra-individual differences in achievement as part of an identification model for SLD. The regulations also allow for the assessment of discrepancies in intellectual development and achievement. (p. 46651) Comment: Some commenters recommended using ‘‘cognitive ability’’ in place of ‘‘intellectual development’’ because ‘‘intellectual development’’ could be narrowly

104

interpreted to mean performance on an IQ test. One commenter stated that the term ‘‘cognitive ability’’ is preferable because it reflects the fundamental concepts underlying SLD and can be assessed with a variety of appropriate assessment tools. A few commenters stated that the reference to identifying a child’s pattern of strengths and weaknesses that are not related to intellectual development should be removed because a cognitive assessment is critical and should always be used to make a determination under the category of SLD. Discussion: We believe the term ‘‘intellectual development’’ is the appropriate reference in this provision. Section 300.309(a)(2)(ii) permits the assessment of patterns of strengths and weakness in performance, including performance on assessments of cognitive ability. As stated previously, ‘‘intellectual development’’ is included as one of three methods of comparison, along with age and State-approved grade-level standards. The term ‘‘cognitive’’ is not the appropriate reference to performance because cognitive variation is not a reliable marker of SLD and is not related to intervention. (p. 46654) The policy definition leaves the operational “definition” of intellectual development to the SEA and LEA., furthermore, it clearly refers to cognition and constructs measured by NRT commonly used in educational classification systems. Tests commonly used to identify SLD are tests of cognition, language, and achievement which arguably capture the most salient components of “intelligence.” According to Sattler (2018), most experts in the fields of psychology and education generally agree that the important elements of “intelligence” include abstract thinking or reasoning, problem-solving ability, capacity to store knowledge (including academic knowledge), memory, environmental adaptation, mental speed, and linguistic competence. It is important to understand that these “processes” are interdependent and overlapping (Peterson et al., 2017; Potocki, et al., 2017), related to achievement (Fletcher & Miciak 2017; Fuchs et al., 2011), and influenced with environmental conditions such as the classroom setting versus the testing setting. This understanding forms the basis of the practical definition that will guide test selection and interpretation. The elements described in the previous paragraphs can be correlated with the definition of SLD and its most salient features (psychological processing, language, achievement). Additionally, other regulatory requirements on the statute encompass other critical components of intelligence. For example, environmental adaptation can be associated with the required classroom observation for each student suspected of SLD. It can be argued that all schoolwork, for instance a classroom spelling test, measures an aspect of intelligence under different conditions. According to the KTEA-3 manual (Kaufman & Kaufman, 2014), spelling involves listening, acquired knowledge, verbal working memory, executive functions, and strategy use. These are all components of “intelligence” and can be measured in various conditions or “environments.” These cognitive-linguistic processes are used by an individual whether they are taking a spelling test in a diagnostician’s office or in Miss Johnson’s third grade language arts class. When evaluators, using NRTs, connect “intellectual development” with conditions of the classroom and compare “intellectual development” with individualized testing conditions, a deeper understanding of teaching and learning (or not learning) can be made.

105

Critical Assumptions and Evidence Norm-referenced cognitive tests have a long history in the identification of educational disabilities. They have been applied in several ways; usually as some form of discrepancy (i.e.IQ-achievement discrepancies, dual-discrepancy/consistency models, concordancediscordance) to identify the presence of SLD (Decker et al; 2013). When assessing intellectual development using a PSW framework such as Core-Selective Evaluation Process (CSEP), discrepancies and standard scores from norm-reference testing (NRT) data inform decisionmaking and professional judgement and are not determinative. In addition, a task demands analysis for each set if scores fully exploit the NRT data (Schultz & Stephens-Pisecco, 2018). The following assumptions with supporting evidence are required in order to properly use NRTs to assess intellectual development in the PSW framework. Assumption #1: Tests of Cognition (e.g., WISC-V. WJ IV-COG), Tests of Achievement) e.g., KTEA-3, WIAT-3), Tests of Language (e.g., WJ IV Oral Language, CELF-5) all measure aspects of intellectual development. This assumption is crucial to understand for several reasons. First, all of the aforementioned tests are interdependent, and every test and subtest measures an aspect of cognition, language, and achievement to some degree. Evidence for this can be found in the sections of the technical manual of an NRT relating to test structure and descriptions of what each subtest measures. For example, the Applied Problems of the Woodcock Johnson IV Tests of Achievement (WJIV-ACH) requires the “achievement” area of math calculations and problem solving, construction of mental models via language comprehension, and the cognitive process of quantitative reasoning (McGrew et al.; 2014). The Similarities subtest in the Wechsler Intelligence Scales for Children– Fifth Edition (WISC-V), in addition to cognition, requires skills related to achievement such as word knowledge and word meaning (Vocabulary) and the language skill of verbal expression (Wechsler, 2014). Likewise, the Understanding Spoken Paragraphs subtest on the Clinical Evaluation of Language Fundamentals (CELF-5), in addition to measuring receptive language, also measures aspects of cognitive processing, specifically verbal working memory, verbal reasoning and the achievement areas of listening comprehension, vocabulary, and prior knowledge (Wiig, et al., 2013). In addition to the subtest’s descriptions, each test is a measure of an individual’s ability to “listen, think, and speak”. This is evident by each administration of a test requiring input (listen), cognitive activation and application (think), and output (speak). This is an added value to testing considering that every classroom task/assignment requires these same cognitive activation and application under instructional inputs (teaching) and outputs (learning). Information relating to this, along with descriptive tables, are often included in test manuals. When performance is analyzed from this perspective, valuable insights into teaching and learning can occur. In addition to evidence contained in test manuals for this assumption, even scholars divided on the role of cognitive assessments disregard the artificial cognitive/academic distinction and have recognized the reciprocal and interdependent relationship of these variables (Fletcher & Miciak, 2017; Schneider & Kauffman, 2017). This is also reflected in test design, especially those whose theoretical anchor is the Cattel-Horn-Carroll (CHC) Theory. The examiner manual of the WJ IV Achievement Manual refers to quantitative knowledge and reading and writing as cognitive

106

abilities (Schrank et al., 2014). In writings by well-established neuro-scholars (Woodcock et al., 2017) the term cognitive is referred to as both traditional ability (i.e., intellectual), and achievement measures. When tests are considered in this fashion, discrepancy analysis shifts beyond identification to a greater understanding of the learner. Assumption # 2: Standard Scores are Not Equivalent to Functioning Concerns regarding the use of standard scores in the identification of SLD have been raised for decades (Katz & Slomka, 2000; Phillips & Clarizio, 1988) including measurement issues with discrepancy models (Gresham & Vellutino, 2010; Taylor et al., 2017; Van den Broeck, 2002). Test publishers recognize the limitations in standard score interpretation for students with disabilities. The fundamental misunderstanding is that a common interpretive error in standard scores are equivalent to functioning or performance. This leads to faulty generalizations. For example, a standard score of 90 on a memory test could be interpreted to mean that the student has “average” functioning in memory when in fact a more accurate description of this score is that it represents an individual’s relative position or “place” in line as it is ordinal data (Jaffe, 2009; Adeyemi, 2010). The WIAT-3 manual provides guidance on this limitation: Students with mathematics disabilities typically present a profile of math strengths and weaknesses, and may perform considerably below average in some areas and average or better in other areas. For a student with this profile, an overall subtest or composite score may overestimate or underestimate his or her math ability. For this reason, performing a skills analysis is particularly important for evaluating a student’s profile of strengths and weaknesses on the Math Problem Solving and Numerical Operations subtest (p.8) This is not to say there is little or no value in standard scores as they are a necessary superficial first step in test interpretation, rather to caution practitioners against misuse and misunderstanding. The trustworthiness of standard scores can be increased when considering the task demands of the test, converging other data, and comparisons with other metrics such as the WJ-IV relative proficiency index (RPI) scores or an error analysis reported on the KTEA-3. Assumption # 3. Norm Referenced Tests are Academic Behavior Samples Behavior can be defined as observable, measurable student actions (Lewis et al. 2017) and is not limited to social behavior but academic behavior as well. For example, total words written is a measurable student action as well as number of correct responses. Academic behavior is the outward expression of underlying traits such as language (Chow et al., 2020) and cognition (Malanchini, 2019). Because we cannot observe “language” or “cognition,” we must rely on observing behavior and making inferences. We can observe academic behavior in the classroom, which is influenced not only by cognition and language but also classroom by dynamics, teacherstudent relationships, subject, competing stimuli, etc. Understanding behaviors that occur in the natural environment are critical in both identification of students suspected of having SLD but perhaps even more important in planning instruction and interventions. It is important to emphasize that an “artificial” environment is being created when a diagnostician administers an individualized NRT in a controlled setting. In research terms, we can think of the classroom as an “applied” setting with less controls and testing situation as “lab

107 conditions with tighter controls. In the testing situation we must provide a stimulus allowing us to collect a behavior sample that provides insight to the role of cognition and language. When NRTs are viewed as the collection of academic behavior samples instead of a vehicle to make statistical comparisons, a deeper insight into student thinking and language will be gained. This will be further explained in the discussion of assumption #4. Assumption #4. Conditional and Task Analysis is as Important as Scores, if not more. Considering the limitation of standard scores discussed earlier, it is vital that evaluators support impressions with additional analysis. Conditional analysis and task demand analysis are accepted educational practices and have been used to understand students with disabilities for decades. Systematic direct observations are often used by school psychologists to quantify student behavior to identify predictable patterns of behavior (Doggett, 2001; Dufrene, 2017; Eckert et al., 2005). This observational technique is usually utilized in applied settings (i.e., classroom) to determine what conditions occur prior to the behavior that can be easily utilized in testing situations. Individualized NRTs, by definition, are systematic and require direct observation. When conditional analysis is applied to both the testing session and teaching setting, predictable patterns of academic behavior can be identified and remediated; for example, when instructions are given verbally, the student may not understand. Low or marginal NRT scores (e.g., 85-92) can point to constructs that require further exploration and explanations. High NRT scores (e.g., 95 and above) usually indicate the student met the cognitive, linguistic, and academic demands of a set of items. An associated practice, task demand analysis, has been used extensively in special education to understand learning. Tasks can be analyzed as cognitive, linguistic, input (e.g., visual, auditory), and output (e.g., written response, spoken). Task demands have been used to understand dyslexia and math disabilities. Alt et al., (2019) compared students with typical development vs atypical development. This research manipulated the reading task demands of second graders, some of whom had language disorders and dyslexia, some had dyslexia, and some had typical development. By manipulating the tasks, they were able to differentiate these three groups. Cross et al. (2019) reviewed similarly designed research using task demands to examine mathematical cognition in children with math disabilities. This review indicated that the instruction and assessments were able to be differentiated when considering the variability of the tasks (e.g., verbal, nonverbal, etc.).Other studies have examined the complex working memory-span tasks by increasing the processing load and storage demands varying processing to understand the relationship with language (Archibald & Griebeling, 2015). The classroom setting and the testing session provide numerous opportunities to observe performance under several task demands. When conditional analysis and task demand analysis are utilized in both the testing session and teaching setting, a more detailed profile of student’s strengths and weaknesses can emerge. This information can also inform specially designed instruction (SDI). Consider the usefulness of this statement when these two practices are interwoven: “Victoria is able to accurately recall verbal information when it is presented in a test session when distractions are reduced, and she is looking directly at the examiner.” This test behavior can be compared to the task and conditions of the classroom to gain further insight. Low standard scores can assist the evaluator in identifying these areas; moreover, additional task demand similarities and differences can be made within the test batteries.

108

Assumption #5. Norm-referenced tests of achievement are not valid measures of curriculum or grade level standards. According to: 34 CFR 300.309 determining the existence of a specific learning disability. (a) The group described in § 300.306 may determine that a child has a specific learning disability, as defined in § 300.8(c)(10), if— (1) The child does not achieve adequately for the child's age or to meet State-approved gradelevel standards in one or more of the following areas, when provided with learning experiences and instruction appropriate for the child's age or State-approved grade-level standards: As discussed earlier, achievement measures (e.g., WIAT-III, WRAT-5. KTEA-2; WJ4 Ach) are measures of intellectual development. They do not measure curriculum or state standards and are not validated as such. In addition to the evidence provided earlier, further evidence can be found in the technical manual of achievement tests regarding the validity of each battery. Validity evidence for NRTs are supported via correlations with other instruments and usually expert review, examiner feedback, some curriculum, and special group studies. It is impossible to capture the depth and breadth of a state’s curriculum using an NRT due to the limited number of items required to differentiate performance between ages and grade. The best way to measure achievement adequately for age or meeting grade level standards is using curriculum-based assessments based on standards typical for the student’s age and grade. NRTs of achievement, however, are useful in identifying and understanding students with SLD due to the cognitive complexity of these measures and their relationship to academic domains. The most accurate description of an achievement test is a “cognitive process paired with an academic skill.” This is what test manuals explicitly state in their validity descriptions. Some statements from common test manuals supporting this assertion include:

a) WIAT-III: For the WIAT-III, validity evidence on response process should provide support that the student engages in the expected cognitive process when responding to subtest items (academic skills). p. 40 b) KTEA-3: The responses should provide support that the examinee engages in the expected cognitive processes when responding to test items (academic skills). p. 50 c) Woodcock Johnson IV Tests of Achievement: The WJ IV ACH contains test that tap two other identified cognitive abilities: quantitative knowledge (Gq) and readingwriting ability (Grw). The WJ IV ACH also includes additional measures of comprehension-knowledge (Gc), long-term retrieval (Glr), and auditory processing (Ga). Because most achievement tests require the integration of multiple cognitive abilities, information about processing can be obtained by a skilled examiner. p. 6. d) Wide Range Achievement Test, Fifth Edition (WRAT5): The responses should provide support that the examinee engages in the expected cognitive processes when responding to test items. p. 46. When “achievement” tests are properly used as the have been validated (i.e., academic skill paired with a cognitive process), discrepancy between cognition and achievement is of limited value. A much deeper analysis and a greater understanding of the learner can be obtained when considering the cognitive complexity of the test or task. For example, an evaluator can examine

109

processing speed as a discrete cognitive ability; however, paired with an academic skill such as Word Reading Fluency could help one gain a deeper insight. Consider this interpretive statement: Sophia’s processing speed can be described as average when completing simple tasks such as discriminating between letters patterns. She struggles with tasks that require processing speed and reading (specifically oral reading), but she has average ability to when using processing speed to quickly solve math problem. When reading this statement, we can see that processing speed was measured under three conditions, with two conditions being average and one requiring further explanation. A skilled examiner would be able to compare this finding to other data sources to identify strengths and weaknesses as well as areas to remediate or accommodate. A logical instructional implication would be that Sophia continue to get phonics instruction and would require extra time to complete tasks that require this skill. When standard scores are used in this manner, as markers or pointers, it naturally forces a deeper level of interpretation. Role of Norm-Referenced Tests Individualized NRTs are used as one tool in a variety of assessment tools and strategies to complete a full and individual evaluation (FIE). When using a PSW strategy to identify SLD, intellectual development must be assessed to a) rule out other disabilities, and b) to identify a PSW in intellectual development. The most practical, valid, and reliable tools available for schools to measure intellectual development are individualized NRTs. The primary role for using these tests when considering the five assumptions is to explain the meaning behind the scores including variance, low scores, and how the constructs that are measured impact learning. They are vital for proper identification and understanding learning. An evaluator who suspects SLD using a PSW strategy has the charge of explaining “underachievement.” The three possible explanations for this can be broadly classified into a) instructional, b) exclusionary factors (Whittaker & Ortiz, 2019), or c) due to strengths and weakness in achievement, performance relative to age, grade level standards or intellectual development. “Achievement,” or adequate progress toward state standards, is best assessed using data sources (e.g., state test, work samples, progress reports, curriculum based assessments, etc.). While some inferences concerning intellectual development can certainly be made using these informal data sources, they are not as precise, nor do they have the necessary psychometric controls. Regarding identification using a PSW model, Schultz and Stephens-Pisecco (2018) recommend identifying patterns of intellectual development using statistical variations (i.e.~1 SD) supported with other data and professional judgment. A student’s profile of intellectual development as measured by NRTs of cognition, language, and achievement should show evidence of significant variance (i.e., PSW) to support an SLD identification. This profile is considered with all other data sources. Using individualized NRTs requires a skill set beyond test administration, scoring, and test interpretation, specifically the relationship between the learning process and executive functioning (Barnes et al. 2020; Keenan et al., 2019). Evaluators also need a complete understanding of the educational environments (conditions) and their impact on student learning

110

(Simonsen et al., 2019). This understanding is key to conduct both conditional and task demand analysis. A car transmission can be used as an analogy of intellectual development. If a car does not run, it may be the transmission (intellectual development). In order to understand how and where the breakdown is, one must use mechanical tools to take the transmission apart to explain what is going on. If we substitute “norm-referenced tests” for mechanical tools and “intellectual development” for transmission, we are essentially taking apart teaching and learning, to find out specifically what is causing or contributing to the problem and how to best address it. When this information is put in context with all other data (e.g., teaching and learning data, exclusionary factors, historical data) an evaluation has done much more than simply identify a disability. Instead a powerful, precise student learning profile will emerge that is complete and informative. According to the CFR §300.304 Each public agency must ensure that— (1) Assessments and other evaluation materials used to assess a child under this part… (iii) Are used for the purposes for which the assessments or measures are valid and reliable; (iv) Are administered by trained and knowledgeable personnel; and (v) Are administered in accordance with any instructions provided by the producer of the assessments. Critics of cognitive tests often cite cost as a reason to use alternate methods of SLD identification, which t do not require much, if any, individualized norm referenced testing (Stuebing et al., 2012; Taylor et al., 2017; Williams & Miciak, 2018). If testing consists of giving an extensive number of tests to primarily obtain discrepancy scores, then this criticism is partially correct. To use tests in a more efficient, cost-effective manner and fully exploit (not abuse) the data, examiners may implement the following practices: 1. Administer tests in accordance with any instructions provided by the producer of the assessments, use publishers’ statistical calculations to inform decision making. 2. Classify, sort, and examine the task demands of all low scores. Consider the following low scores (Range=SS 65-73) obtained from a WJ IV. Table 1 Task Demands of selected WJ-IV Tests Task Demands of selected WJ-IV Tests Test

Primary Broad CHC Ability

Stimuli

Task Requirements

Cognitive Processes

Response

5: Phonological Processing A: Word Access B: Word Fluency C: Substitution

Auditory Processing (Ga) Phonetic coding (PC) Word fluency (GlrFW) Speed of lexical access (Glr-LA)

Auditory (words)

Semantic activation, access; speed of lexical access

Oral (words)

3: Segmentation

Auditory Processing (Ga) Phonetic coding(PC)

Auditory (words)

Providing a word with a specific phonic element; naming as many words as possible that begin with a specified sound; substituting part of a word to make a new word Listening to a word and breaking it into syllables or phonemes

Analysis of acoustic, phonological elements in

Oral (word parts, phonemes)

111

3: Spelling

Reading & Writing Ability (Grw ) Spelling ability (SG)

Auditory (words)

Spelling orally presented words

immediate awareness Access to and application of knowledge of orthography of word forms by mapping whole word phonology onto whole-word orthography, by translating phonological segments into graphemic units, or by activating spellings of words from the semantic lexicon

Motoric (writing)

All three of these tests have phonetic coding in common and are presented orally. In addition, these three tasks involve storage and retrieval in some capacity. A logical next step would be to ask the teacher if this student has difficulty in the classroom with tasks presented auditorily which require phonics and obtain supporting work samples. Comparing and contrasting this information to other data will aid in understanding specific areas to target instruction and provide additional interpretative information. 3. Use conditional analysis to aid in understanding the learning process for an individual by comparing tasks in testing conditions versus tasks in teaching and learning conditions. Consider a construct such as reading comprehension and this interpretive statement: According to Olivia’s teachers, she struggles with reading comprehension as measured informally with unit tests, homework, and benchmark assessments (teaching and learning conditions), however her KTEA-3 Reading Comprehension score of 96 indicates average ability. By comparing and contrasting testing and teaching conditions, a testable hypothesis can be explored. An explanation of this variance may be that the KTEA-3 Reading Comprehension requires immediate recall without competing stimuli, while the actual classroom reading comprehension requires delayed recall and increased memory and retrieval demands. This example also illustrates the importance of making appropriate generalizations to the classroom from norm-referenced tests. 4. Standard scores obtained from individual norm-referenced tests should inform professional judgment and other than providing the examiner with a student’s position in line and have no meaning beyond that until it is put in context with all other data. Scores are not “strengths” or “weaknesses;” rather, scores help the examiner decide if the construct (e.g., working memory, reading comprehension, oral expression) as a whole is a “strength” or “weakness.”

112

Conclusion The SLD identification debate will no doubt continue as long as it remains a construct. Individualized norm-referenced tests assist in the identification of SLD specifically to measure intellectual development and to aid examiners in gaining a deeper understanding of the learner. Testing technology continues to improve and, when used in the manner described in this article, identification and instructional decisions can be made in a more comprehensive manner. References Alt, M., Gray, S., Hogan, T. P., Schlesinger, N., & Cowan, N. (2019). Spoken word learning differences among children with dyslexia, concomitant dyslexia and developmental language disorder, and typical development. Language, speech, and hearing services in schools, 50(4), 540-561. Adeyemi, T. O. (2011). The effective use of standard scores for research in educational management. Research Journal of Mathematics and Statistics, 3(3), 91-96. Archibald, L. M., & Harder Griebeling, K. (2016). Rethinking the connection between working memory and language impairment. International Journal of Language & Communication Disorders, 51(3), 252-264. Barnes, M.A.,Clemens, N.H., Fall, A.M., Roberts, G., Klein, A., Starkey, P., McCandliss, B., Zucker, T., & Flynn, K. (2020). Cognitive predictors of difficulties in math and reading in pre-kindergarten children at high risk for learning disabilities, Journal of Educational Psychology, 112 (4). Breaux, K,C. (2010). Wechsler Individual Achievement Test–Third Edition: Technical Manual. NCS Pearson. Inc., Bloomington. Chow, J. C., Walters, S., & Hollo, A. (2020). Supporting students with co-occurring language and behavioral deficits in the classroom. TEACHING Exceptional Children, 52(4), 222- 230. Cross, A. M., Joanisse, M. F., & Archibald, L. M. (2019). Mathematical abilities in children with developmental language disorder. Language, Speech, and Hearing Services in Schools, 50(1), 150-163. Decker, S. L., Hale, J. B., & Flanagan, D. P. (2013). Professional practice issues in the assessment of cognitive functioning for educational applications. Psychology in the Schools, 50(3), 300-313. Doggett, R. A., Edwards, R. P., Moore, J. W., Tingstrom, D. H., & Wilczynski, S. M. (2001). An approach to functional assessment in general education classroom settings. School Psychology Review, 30(3), 313-328. Dufrene, B. A., Kazmerski, J. S., & Labrot, Z. (2017). The current status of indirect functional assessment instruments. Psychology in the Schools, 54(4), 331-350. Eckert, T. L., Martens, B. K., & DiGennaro, F. D. (2005). Describing antecedent-behaviorconsequence relations using conditional probabilities and the general operant contingency space: A preliminary investigation. School Psychology Review, 34(4), 520-528. Flanagan, D.P., & Schneider, J. L. (2016). Cross-Battery Assessment? XBA PSW? A case of mistaken identity: A commentary on Kranzler and colleagues' “Classification agreement analysis of Cross-Battery Assessment in the identification of specific learning disorders in children and youth”, International Journal of School & Educational Psychology, 4(3), 137145. Fletcher, J. M., & Miciak, J. (2017). Comprehensive cognitive assessments are not necessary for the identification and treatment of learning disabilities. Archives of Clinical Neuropsychology, 32(1), 2-7.

113 Fletcher, J. M., & Miciak, J. (2019). The identification of specific learning disabilities: A summary of research on best practices. Austin, TX: Texas Center for Learning Disabilities. Fuchs, D., Hale, J. B., & Kearns, D. M. (2011). On the Importance of a Cognitive Processing Perspective: An Introduction. Journal of Learning Disabilities, 44(2), 99-104. Gresham, F. M., & Vellutino, F. R. (2010). What is the role of intelligence in the identification of specific learning disabilities? Issues and clarifications. Learning Disabilities Research & Practice, 25(4), 194-206. Hale, J., Alfonso, V., Berninger, V., Bracken, B., Christo, C., Clark, E., ...Schultz, E.K. (2010). Critical Issues in response-to-intervention, comprehensive evaluation, and specific learning disabilities identification and intervention: An expert white paper consensus. Learning Disabilities Quarterly, 33, 223-236. Individuals with Disabilities Improvement Act, 34 C.F.R. § 300 (2006) Jaffe, L. E. (2009). Development, interpretation, and application of the W score and the relative proficiency index (Woodcock-Johnson III Assessment Service Bulletin No. 11). Rolling Meadows, IL: Riverside Publishing. Katz, L. J., & Slomka, G. T. (2000). Achievement testing. Handbook of psychological assessment, 149-182. Kaufman, A.S., & Kaufman, N.L. (with Breaux, K.C.). (2014). Administration manual. Kaufman test of educational achievement, third edition. Bloomington, MN: NCS Pearson Keenan, L., Conroy, S., O’Sullivan, A., & Downes, M. (2019). Executive functioning in the classroom: primary school teachers’ experiences of neuropsychological issues and reports. Teaching & Teacher Education, 86,N.PAG. Kranzler, J. H., Gilbert, K., Robert, C. R., Floyd, R. G., & Benson, N. F. (2019). Further Examination of a Critical Assumption Underlying the Dual-Discrepancy/Consistency Approach to Specific Learning Disability Identification. School Psychology Review, 48(3), 207-221. Lee, K., Swee, NG, & Bull, R. (2018). Learning and solving algebra word problems: The roles of relational skills, arithmetic, and executive functioning. Developmental Psychology, 54, (9), 1758-1772. Lewis, T. J., Hatton, H. L., Jorgenson, C., & Maynard, D. (2017). What beginning special educators need to know about conducting functional behavioral assessments. Teaching Exceptional Children, 49(4), 231-238. Malanchini, M., Engelhardt, L. E., Grotzinger, A. D., Harden, K. P., & Tucker-Drob, E. M. (2019). “Same but different”: Associations between multiple aspects of self-regulation, cognition, and academic abilities. Journal of personality and social psychology, 117(6), 1164. Mather, N., & Gregg, N. (2006). Specific learning disabilities: Clarifying, not eliminating, a construct. Professional Psychology: Research & Practice, 37(1), 99-106. McGill, R. J. & Busse, R. T. (2016). When theory trumps science: A critique of the PSW model for SLD identification. Contemporary School Psychology, 21(1), 10-18. McGrew, K. S., LaForte, E. M., & Schrank, F. A. (2014). Technical Manual. Woodcock- Johnson IV. Rolling Meadows, IL: Riverside. Miciak, J., Taylor, W. P., Denton, C. A., & Fletcher, J. M. (2015). The effects of achievement test selection on identification of learning disabilities within a pattern of strengths and weaknesses framework. School Psychology Quarterly, 30(3). 321-334. Peterson, R. L., Boada, R., McGrath, L. M., Willcutt, E. G., Olson, R. K., & Pennington, B. F. (2017). Cognitive prediction of reading, math, and attention: Shared and unique influences. Journal of learning disabilities, 50(4), 408-421. Phillips, S. E., & Clarizio, H. F. (1988). Limitations of standard scores in individual achievement testing. Educational Measurement: Issues and Practice, 7(1), 8-15.

114 Potocki, A., Sanchez, M., Ecalle, J., & Magnan, A. (2017). Linguistic and cognitive profiles of 8-to 15-year-old children with specific reading comprehension difficulties: The role of executive functions. Journal of Learning Disabilities, 50(2), 128-142. Schneider, W. J., & Kaufman, A. S. (2017). Let's not do away with comprehensive cognitive assessments just yet. Archives of Clinical Neuropsychology, 32(1), 8-20. Schrank, F. A., Mather, N., & McGrew, K. S. (2014). Technical Manual: Woodcock-Johnson IV. Itasca, IL: Riverside Publishing. Schultz, E.K., & Stephens-Pisecco, T.L. (2018). Using the core-selective evaluation process to identify a PSW: Integrating research, practice, and policy. Special Education Research, Policy & Practice, Fall 2018 Simonsen, B., Freeman, J., Swain-Bradway, J., George, H.P., Putnam, R., Lane, K.L. Sprague, J., & Hershfeldt, P. (2019). Using data to support educators’ implementation of positive classroom behavior support (PCBS) practices. Education & Treatment of Children, 42 (2), 265-289. Stuebing, K. K., Fletcher, J. M., Branum-Martin, L., Francis, D. J., & VanDerHeyden, A. (2012). Evaluation of the technical adequacy of three methods for identifying specific learning disabilities based on cognitive discrepancies. School Psychology Review, 41(1), 3-22. Taylor, W. P., Miciak, J., Fletcher, J. M., & Francis, D. J. (2017). Cognitive discrepancy models for specific learning disabilities identification: Simulations of psychometric limitations. Psychological assessment, 29(4), 446. Van den Broeck, W. (2002). The misconception of the regression-based discrepancy operationalization in the definition and research of learning disabilities. Journal of Learning Disabilities, 35(3), 194-204. Wechsler, D. (2014). WISC-V: Technical and Interpretive Manual: NCS Pearson. Inc., Bloomington. Whittaker, M, & Ortiz, S.O. (2020). What a specific learning disability is not: examining exclusionary factors [White paper]. New York, NY: National Center for Learning Disabilities. Woodcock, R.W., Miller, D.C., Maricle, D., & McGill, R.J. (2017). Evidence-Based Selective Assessments for Academic Disorders. School Neuropsych Press: Middletown, MD. Wiig, E.H., Semel, E., & Secord, W.A. (2013). Clinical Evaluation of Language Fundamentals: About the Authors Edward K. Schultz, PhD, NCED; is a Full Professor and a distinguished West Scholar at the West College of Education at Midwestern State University (MSU). He is the co-author of the CoreSelective Evaluation Process (C-SEP) and has written numerous peer-reviewed articles, presented at the national and international level, and has provided trainings across the country to schools and state departments of education. Interests include SLD identification including dyslexia, MTSS, and students with EBD. Emily Rutherford, Ed.D. is an assistant professor in the West College of Education at Midwestern State University. She has spent over fifteen years working in public schools as a teacher, educational diagnostician, special education administrator, and as a university professor. Dr. Rutherford presents at regional, state, and national conferences on autism, learning disabilities, teaching special education and other related topics. Dennis Cavitt Ed.D. was graduated from Texas Tech University in Special Education with an emphasis in Autism and High Incidence Disabilities. Cavitt also has a Master’s in Education

115 Leadership from Tarleton State University and A Master’s in Clinical Psychology from Abilene Christian University. Cavitt is an Assistant Professor in Special Education in the West College of Education at Midwestern State University (MSU). He teaches Undergraduate and Graduate courses in Special Education.

116 Training Preservice Special Educators for Professional Collaboration: What Teacher Educators Think and Do Laurie A. Sharp, Ed.D. Tarleton State University Michelle Simmons, Ed.D. West Texas A&M University Abstract Special educators must possess well-developed knowledge and skills for professional collaboration. However, much literature has revealed inadequacies with professional collaboration among preservice special educators enrolled in special education teacher preparation programs, as well as low self-efficacy beliefs among practicing special educators in the field. The present study sought to extend these findings by: (1) evaluating teacher educators’ views of preservice special educators’ preparedness for professional collaboration and (2) identifying preparation practices that teacher educators use to cultivate preservice special educators’ knowledge and skills with professional collaboration. To achieve these goals, the present study disseminated a cross-sectional survey among teacher educators who were affiliated with university-based special education teacher education programs and train preservice special educators. Quantitative and qualitative data were collected among 46 respondents affiliated with special education teacher preparation programs in the South Central United States. Quantitative data analyses demonstrated high views concerning preservice special educators’ preparedness for professional collaboration, and qualitative data analyses generated three themes regarding current preparation practices. Based upon these findings, several implications for teacher educators, study limitations, and areas for future research were described. Keywords: preservice special educators, professional collaboration, special education, teacher educators, teacher training Introduction Professional collaboration is a high-leverage practice that PreK-12 special educators use to address the individual needs of students with disabilities (McLeskey et al., 2017). Professional collaboration involves working with a wide range of education stakeholders, such as other teaching professionals, school administrators and staff members, family members, and caregivers, to provide a foundation upon which effective teaching and learning occurs (Friend & Cook, 2017). When special educators collaborate with other education stakeholders, all who are involved experience benefits. Students with disabilities have equal opportunities to engage with the core PreK-12 general education curriculum (Fisher & Frey, 2001), gain access to expanded services and educational enrichments (Thomas, 2004), experience successful transitions to life after high school (Roberts, 2010), and receive appropriate supports that promote learning (Silverman & Millspaugh, 2006). Additionally, PreK-12 education professionals enhance their individual and collective efficacy (Morgan, 2016), and students’ families become more

117 knowledgeable about special education services and how to support their child at home (Aceves, 2014). In today’s schools, special educators must be “masters of collaboration” (Fisher, Frey, & Thousand, 2003, p. 46). However, research has shown concerns with how teacher educators equip future special educators to engage in professional collaborations during teacher training (Hamilton-Jones & Vail, 2013; Weiss, Pellegrino, & Brigham, 2017). Since Leko, Brownell, Sindelar, and Kiely (2015) posited that professional collaboration was one of the cornerstones for effective delivery of individualized services to students with disabilities, special educators must be effectual professionals who possess refined professional collaboration skills. With this in mind, the present study sought to (1) evaluate teacher educators’ views of preservice special educators’ preparedness for professional collaboration and (2) identify preparation practices that teacher educators use to cultivate preservice special educators’ knowledge and skills for professional collaboration. Findings from the present study offered a snapshot of special education teacher training for professional collaboration and suggested avenues with which teacher educators may improve current preparation practices. Review of Relevant Literature The term professional collaboration has recently received increased attention and a rise in prevalence within the field of special education (Hernandez, 2013). Unfortunately, current understandings of professional collaboration may be distorted, narrow, and oversimplified. According to Friend and Cook (2017), professional collaboration is a “style of interaction” where two or more people are “voluntarily engaged in shared decision making as they work toward a common goal” (p. 5-6). Successful professional collaborations involve a wide range of education stakeholders—teaching professionals, educational leaders, evaluation personnel, parents, families, caregivers, and service providers—who pool their collective expertise to develop comprehensive understandings about the strengths and needs of students with disabilities. To achieve this shared goal and mutual investment, collaborative partners must work together to design, plan, and implement instruction and support services that promote success among each individual student (McLeskey et al., 2017). Teacher educators must incorporate carefully coordinated, coherent, and well-designed learning activities throughout teacher training to develop preservice special educators’ knowledge and skills for professional collaboration (Leko, Brownell, Sindelar, & Murphy, 2012). Specifically, preservice special educators must acquire understandings about fundamental principles of professional collaboration, discover ways to function as a collaborative partner among others, and become competent in how to foster success among students with disabilities in diverse contexts (Council for Exceptional Children [CEC], 2015). Since knowledge and skills for professional collaboration do not come naturally to many individuals, teacher educators need to ensure that coursework and field experiences promote preservice teachers’ understandings in a way that is meaningful (Da Fonte & Barton-Arwood, 2017). Even though professional collaboration has been recognized as a vital aspect of special education teacher training (CEC, 2015), there has not yet been a collective effort among teacher educators to provide preservice special educators with explicit training on its vital components (Weiss et

118 al., 2017). As such, education researchers have highlighted inadequacies with current teacher training efforts. For example, Hamilton-Jones and Vail (2013) reported that preservice special educators held inflated self-efficacy beliefs for professional collaboration. They noted that preservice special educators verbalized high levels of confidence and preparedness to practice professional collaboration in the workplace, yet they seldom demonstrated any knowledge of specific abilities and skills associated with professional collaboration. The lack of appropriate teacher training for professional collaboration is further supported by studies conducted among practicing special educators. Findings from these studies have revealed that practicing special educators do not feel adequately prepared to engage in successful professional collaborations with general education teachers (Keefe & Moore, 2004; OtisWilborn, Winn, Griffin, & Kilgore, 2005; Zagona, Kurth, & MacFarland, 2017); students’ families (Latunde & Louque, 2012); and other professionals, such as career and technology education staff members (Schmalzried & Harvey, 2014), mental health professionals (Babyak & Koorland, 2001), and school nurses (Pufpaff, McIntosh, Thomas, Elam, & Irwin, 2015). Based upon these research findings, it is evident that ongoing training for professional collaboration must occur among practicing special educators in the field. Due to the specialized nature of their role, special educators need frequent opportunities to engage in learning activities that continuously develop and refine their ability to engage in successful professional collaborations with a wide range of individuals in complex school settings (Ketterlin-Geller, Baumer, & Lichon, 2015). Methods Context The present study is part of a state-wide analysis that examined the viewpoints of teacher educators concerning special education teacher training. Due to guidelines associated with teacher licensure among state education agencies, the researchers conducted this state-wide analysis in one state located in the South Central United States. The researchers employed a cross-sectional survey research design to ascertain descriptive information from teacher educators who train preservice special educators (Ruel, Wagner, & Gillespie, 2016). Selection of this research design was an appropriate and efficient way to recognize how teacher educators view the preparedness of preservice special educators and to identify preparation practices they used to develop competence with the required behaviors, knowledge, and skills of novice special educators. Research Sample A full list of teacher educators in the state was unavailable at the time of the present study. Therefore, the researchers used purposive sampling techniques to create a participant pool of teacher educators who were affiliated with university-based teacher education programs and trained preservice special educators. First, the researchers consulted the state education agency’s website and identified 55 state-approved, university-based teacher preparation programs for special educators. Next, the researchers accessed each university’s website and searched through publicly available information (i.e., class schedules, course syllabi, faculty listings in academic departments) to locate teacher educators who train preservice special educators. The researchers stored the names and email addresses of teacher educators who fit this criterion in a virtual

119 spreadsheet stored within a password-protected Google Drive. These purposive sampling techniques resulted in the identification of 283 participant pool members. Instrumentation The researchers created a web-based survey instrument using Google Forms to collect data. The survey instrument included closed-ended items that collected demographic information (i.e., age range, gender, years of teaching experiences) and ratings for respondents’ views of preservice special educators’ preparedness with each of the CEC’s (2015) initial preparation standards: 1. 2. 3. 4. 5. 6. 7.

Learner Development and Individual Learning Differences Learning Environments Curricular Content Knowledge Assessment Instructional Planning and Strategies Professional Learning and Ethical Practice Collaboration

Within these seven standards, the CEC delineated a total of 28 key elements that define the needed behaviors, knowledge, and skills of novice special educators. For each key element, respondents used a 5-point Likert scale (i.e., Not At All Prepared, Slightly Prepared, Somewhat Prepared, Very Prepared, or Extremely Prepared) to indicate their viewpoints. The survey instrument also contained one open-ended item for each standard that asked respondents to provide information about preparation practices they use to develop preservice special educators’ competence for associated key elements. To establish reliability and validity with the survey instrument, the researchers performed a pilot test among 20 special education experts (Ruel et al., 2016). These experts were teacher educators who were affiliated with university-based teacher preparation programs located in different states beyond the context of the present study. Each pilot test participant completed the survey instrument and provided feedback for survey administration, organization, and content. Based upon this feedback, the researchers made minor revisions with wording on the survey instrument. Data Collection and Analysis The researchers collected data during a four-month period. At the beginning of the survey period, the researchers sent an initial email to all participant pool members that provided information about the study and included a hyperlink to the survey instrument. The researchers tracked survey completion in the virtual spreadsheet that contained information for participant pool members. To address nonresponse bias, the researchers sent three monthly follow-up emails that encouraged participation. To achieve the purpose of the present study, the researchers retrieved and analyzed quantitative and qualitative data from the three survey items related to professional collaboration (see Figure 1). First, the researchers retrieved quantitative data from the three closed-ended survey items. The researchers summarized these data descriptively with frequency counts and percentages (Johnson & Christensen, 2014). For each quantitative survey item, the researchers examined

120 findings to uncover patterns concerning respondents’ viewpoints for preservice special educators’ preparedness for professional collaboration. Next, the researchers retrieved qualitative data from the open-ended survey item. The researchers analyzed these data independently with two levels of coding (Saldaña, 2016). In the first level, the researchers used open coding to label initial concepts in the data. In the second level, the researchers used axial coding to confirm the accuracy of codes and group similar codes into themes. While coding independently, the researchers made analytic memos to document their questions, reflections, and thoughts. Once both researchers completed their independent analyses, they held a virtual meeting to debrief about their coding processes, share their interpretations, and discuss differences until they reached complete intercoder agreement.

Figure 1. Survey items for professional collaboration. Results When the survey period closed, the researchers completed surveys from 46 respondents, which yielded a survey return rate of 16.25%. As shown in Table 1, respondents were predominantly females who were between 40-69 years of age. Moreover, most of the respondents reported that they had five or more years of experiences with training preservice special educators. Table 1 Demographics of Survey Respondents Characteristic N Gender Female 36

121 Male Age Range 20-29 years 30-39 years 40-49 years 50-59 years 60-69 years 70-79 years Years of Teaching Experiences Less than 1 year 2-4 years 5-7 years 8-10 years More than 10 years

10 1 7 13 9 14 2 2 3 12 9 20

Quantitative Results As shown in Figure 1, the CEC’s (2015) initial preparation standard related to professional collaboration consists of three key elements. Respondents indicated their views of preservice special educators’ preparedness for: (a) uses fundamentals of effective collaboration, (b) works as a collaborative resource with others, and (c) uses collaborative methods to support well-being among students with exceptionalities in diverse settings. As shown in Table 2, the majority of respondents expressed views that preservice special educators were either Somewhat Prepared or Very Prepared with each of these three key elements. Table 2 Respondents’ Views of Preservice Special Educators’ Preparedness for Collaboration Key Elements Not At Slightly Somewhat Very Extremely All Prepared Prepared Prepared Prepared Prepared Uses fundamentals of effective -1 15 19 11 collaboration. (2.2%) (32.6%) (41.3%) (23.9%) Works as a collaborative -1 14 21 10 resource with others. (2.2%) (30.4%) (45.6%) (21.7%) Uses collaborative methods to --12 24 10 support well-being among (26.1%) (52.2%) (21.7%) students with exceptionalities in diverse settings. Qualitative Results Of the 46 survey respondents, 33 respondents provided a response to the open-ended question related to current preparation practices for professional collaboration. In total, these responses consisted of 562 words. Qualitative data analyses generated the following three themes: Preparation Practices during Coursework, Preparation Practices during Field Experiences, and

122 Concerns about Training for Professional Collaboration. A summary of each theme is provided below. Preparation practices during coursework. Within this theme, respondents specified preparation practices they use to develop preservice special educators’ knowledge and skills for professional collaboration during coursework. These preparation practices included “authentic assessments” and a wide variety of specific learning activities, such as “case studies,” “class discussions,” “group projects,” “parent panels,” “role playing,” and “scenario simulations.” Respondents also provided preservice special educators with explicit instruction through coursebased lectures to “teach [preservice special educators] to learn cooperatively.” Additionally, respondents modeled common processes involving professional collaborations for preservice special educators, such as “developing IEPs (i.e., individualized education program)” and “participating in ARD (i.e., admission, review, and dismissal) committee meetings.” During coursework, respondents noted that they relied on technology, such as “email” and digital tools within their learning management system, to promote professional collaboration among preservice special educators. Respondents also noted that they used external experts at times to deliver “third party interpersonal training.” Moreover, respondents encouraged all preservice special educators to broaden their network for professional collaboration by establishing membership in the CEC professional organization. Preparation practices during field experiences. Within this theme, respondents specified preparation practices they use to develop preservice special educators’ knowledge and skills for professional collaboration during field experiences. According to respondents, field experiences provided preservice special educators with valuable opportunities to collaborate with a wide variety of professionals in authentic contexts, such as their “mentor teacher,” “site coordinator,” and university instructors. Respondents emphasized that field experiences empowered preservice special educators to continually practice “professionalism,” which helps facilitate a smooth “transition from students to teachers.” During field experiences, respondents noted that they required preservice special educators to “attend ARD meetings” and “parent meetings.” Respondents also noted that they required preservice special educators to actively participate in “PLCs (i.e., professional learning communities),” the development of “IEPs,” and “co-teaching” experiences with general education teachers. Furthermore, respondents indicated that preservice special educators practiced how to foster positive relationships with families. For example, one respondent explained that preservice special educators “write letters to parents in English and Spanish with clear language to reach all families.” Concerns about training for professional collaboration. Within this theme, respondents shared concerns they have regarding preservice special educators’ preparedness for professional collaboration. One respondent stated, “This is an area of concern in [my teacher preparation program’s] curriculum.” Similarly, another respondent explained: I feel that this area is weak in [my teacher preparation] program. We talk a lot about parental involvement and parental value, but I really think that more needs to be done to

123 really develop effective teaming practices and teaming skills in [preservice special educators]. In addition to concerns about current preparation efforts, respondents also commented that some of the key elements associated with professional collaboration “will be further developed as [preservice special educators] work in the field.” Discussion Due to the specialized nature of their role, practicing special educators must be collaborative teaching professionals (Woolf, 2015). However, education researchers have revealed shortcomings in how teacher educators train preservice special educators to practice professional collaboration (Hamilton-Jones & Vail, 2013; Weiss et al., 2017). Similarly, education researchers have also reported that practicing special educators feel unprepared to collaborate with other professionals effectively (Babyak & Koorland, 2001; Keefe & Moore, 2004; Latunde & Louque, 2012; Otis-Wilborn et al., 2005; Pufpaff et al., 2015; Schmalzried & Harvey, 2014; Zagona et al., 2017). These findings are concerning because special educators must leave their teacher preparation programs and enter PreK-12 education settings with “well-developed [professional] collaboration skills to communicate and work with various service providers” (Leko et al., 2015). To deepen understandings about special education teacher training, the researchers conducted the present study to ascertain (1) teacher educators’ views of preservice special educators’ preparedness for professional collaboration and (2) current preparation practices used to cultivate preservice special educators’ knowledge and skills for professional collaboration. Findings in the present study demonstrated that many teacher educators queried held high views of preservice special educators’ preparedness with each of the three key elements associated with professional collaboration. This finding mirrored the inflated self-efficacy beliefs for professional collaboration that preservice special educators held of themselves (Hamilton-Jones & Vail, 2013). At first glance, these findings have suggested that teacher training is sufficiently developing preservice special educators’ knowledge and skills with professional collaboration. However, it becomes quickly apparent that practicing special educators hold contradictory views and feel inadequately prepared to engage in successful collaborations with others (Babyak & Koorland, 2001; Keefe & Moore, 2004; Latunde & Louque, 2012; Otis-Wilborn et al., 2005; Pufpaff et al., 2015; Schmalzried & Harvey, 2014; Zagona et al., 2017). Findings in the present study also revealed current preparation practices that teacher educators use to prepare preservice special educators for professional collaboration. Overwhelmingly, most of the reported preparation practices occurred during coursework. Among these reported preparation practices, teacher educators relied upon teacher-centered learning activities (e.g., modeling common collaborative processes) and peer-centered learning activities (e.g., group projects). Although many of the reported preparation practices occurred during field experiences, it was not clear how teacher educators developed preservice special educators’ knowledge and skills with professional collaboration beyond the provision of working with others. Of particular interest were the expressions of concern that teacher educators shared about

124 professional collaboration. These concerns recognized shortcomings for this facet of teacher training, as well as the need for continuous on-the-job training. Findings from the present study have several important implications for teacher educators who are responsible for training preservice special educators. First, teacher educators must ensure that they themselves have clear and comprehensive understandings of professional collaboration. Professional collaboration has become a popularized buzzword in the field of special education (Hernandez, 2013), and it is imperative that the professionals involved with training preservice special educators understand distinctions of “the interpersonal style” and “critical components of [professional] collaborative behavior” (Friend, 2000, p. 132). Of equal importance, teacher educators must also possess extensive understandings of factors that impede professional collaborations, such as conflict (Bradley & Monda-Amaya, 2005). Second, teacher educators must make concerted efforts to “facilitate broader and richer crossarticulated experiences among [academic] departments and programs” to train preservice special educators and other preservice teaching professionals, such as PreK-12 generalists and content area specialists, for professional collaboration (Harvey, Yssel, Bauserman, & Merbler, 2010, p. 31). By facilitating cross-departmental training experiences, teacher educators have greater opportunities to facilitate more frequent training opportunities that reinforce knowledge and skills for professional collaboration among a wide range of prospective teaching professionals. Lastly, teacher educators must increase efforts to keep the focus of teacher training on “learning to teach” in authentic PreK-12 school settings (McLeskey & Brownell, 2015, p. 13). By doing so, preparation practices are more likely to promote the generalization of behaviors, knowledge, and skills learned during teacher training into the future work of practicing special educators (Markelz, Riden, & Scheeler, 2017). Maintaining such a focus also requires teacher educators to cultivate strong partnerships with other professionals involved with teacher training, such as cooperating teachers and university supervisors (Roberts, Benedict, & Thomas, 2013). Limitations and Future Research The present study was exploratory in nature and intended to provide a snapshot of special education teacher training for professional collaboration. Although reported findings have contributed to existing understandings in the literature, there were a couple of methodological limitations that warrant caution with interpretations. First, the sample size in the present study was small, and the survey response rate was low. Accordingly, reported results cannot be easily generalized to a larger population. However, data from the present study have provided practical knowledge to help teacher educators assess the impact of teacher training for professional collaboration in their respective special educator preparation programs. Second, data analyses in the present study did not involve testing for statistical significance. To address this limitation, future researchers should examine professional collaboration during teacher training in a more scientifically rigorous manner. For example, future researchers may use a case study research design to conduct in-depth explorations of programmatic special education teacher training for professional collaboration. Additionally, future researchers may perform longitudinal analyses that investigate the impact of preparation practices used during

125 teacher training on future professional collaboration practices used by practicing special educators in the field. Conclusion The work of special educators is quite complex and necessitates specialized expertise. Special educators have a great responsibility to attend to the wide-ranging needs of students with disabilities. As special educators strive to best serve each individual student, it is critical for them to engage in successful professional collaborations with others. Consequently, teacher preparation programs must offer preservice special educators high-impact training for professional collaboration that generalizes into their future teaching practices. Moreover, practicing special educators require continuous professional learning opportunities to further develop and refine their knowledge and skills for professional collaboration. Giving increased attention to professional collaboration among preservice and practicing special educators safeguards the interests and rights of students with disabilities. References Aceves, T. C. (2014). Supporting Latino families in special education through community agency-school partnerships. Multicultural Education, 21(3/4), 45-50. Retrieved from http://www.caddogap.com/periodicals.shtml Babyak, A. E., & Koorland, M. A. (2001). Working together: Mental health and special education collaboration. Children and Youth Services Review, 23(8), 633-649. doi:10.1016/S0190-7409(01)00151-7 Bradley, J. F., & Monda-Amaya, L. E. (2005). Conflict resolution: Preparing preservice special educators to work in collaborative settings. Teacher Education & Special Education, 28(3/4), 171–184. doi:10.1177/088840640502800404 Council for Exceptional Children. (2015). What every special educator must know: Ethics, standards, and guidelines (7th ed.). Arlington, VA: Council for Exceptional Children. Da Fonte, M. A., & Barton-Arwood, S. M. (2017). Collaboration of general and special education teachers: Perspectives and strategies. Intervention in School and Clinic, 53(2), 99-106. doi:10.1177/1053451217693370 Fisher, D., & Frey, N. (2001). Access to the core curriculum: Critical ingredients for student success. Remedial and Special Education, 22(3), 148-157. doi:10.1177/074193250102200303 Fisher, D., Frey, N., & Thousand, J. (2003). What do special educators need to know and be prepared to do for inclusive schooling at work? Teacher Education and Special Education, 26(1), 42-50. doi:10.1177/088840640302600105 Friend, M. (2000). Myths and misunderstandings about professional collaboration. Remedial and Special Education, 21(3), 130-132. doi:10.1177/074193250002100301 Friend, M., & Cook, L. (2017). Interactions: Collaboration skills for school professionals (8th ed.). Boston, MA: Pearson Education, Inc. Hamilton-Jones, B., & Vail, C. O. (2013). Preparing special educators for collaboration in the classroom: Pre-service teachers’ beliefs and perspectives. International Journal of Special Education, 28(1), 56-68. Retrieved from www.internationalsped.com/

126 Harvey, M. W., Yssel, N., Bauserman, A. D., & Merbler, J. B. (2010). Preservice teacher preparation for inclusion: An exploration of higher education teacher-training institutions. Remedial and Special Education, 31(1), 24-33. doi:10.1177/0741932508324397 Hernandez, S. J. (2013). Collaboration in special education: Its history, evolution, and critical factors necessary for successful implementation. US-China Education Review B, 3(6), 480-498. Retrieved from http://www.davidpublisher.org/Home/Journal/UCER-B Johnson, R. B., & Christensen, L. (2014). Educational research: Quantitative, qualitative, and mixed approaches (5th ed.). Thousand Oaks, CA: SAGE Publications, Inc. Keefe, E. B., & Moore, V. (2004). The challenge of co-teaching in inclusive classrooms at the high school level: What the teachers told us. American Secondary Education, 32(3), 77– 88. Ketterlin-Geller, L. R., Baumer, P., & Lichon, K. (2015). Administrators as advocates for teacher collaboration. Intervention in School and Clinic, 51(1), 51-57. doi:10.1177/1053451214542044 Latunde, Y., & Louque, A. (2012). Investing in collaboration: Preservice special educators and their readiness for home school collaboration. Journal of Urban Learning, Teaching, and Research, 8, 73-82. Retrieved from https://aera-ultr.wixsite.com/ultr/journal-of-ultr Leko, M. M., Brownell, M. T., Sindelar, P. T., & Kiely, M. T. (2015). Envisioning the future of special education personnel preparation in a standards-based era. Exceptional Children, 82(1), 25-43. doi:10.1177/0014402915598782 Leko, M. M., Brownell, M. T., Sindelar, P. T., & Murphy, K. (2012). Promoting special education preservice teacher expertise. Focus on Exceptional Children, 44(7), 1-16. doi:10.17161/fec.v44i7.6684 Markelz, A., Riden, B., & Scheeler, M. C. (2017). Generalization training in special education teacher preparation: Does it exist?. Teacher Education and Special Education, 40(3), 179-193. doi:10.1177/0888406417703752 McLeskey, J., Barringer, M. D., Billingsley, B., Brownell, M., Jackson, D., Kennedy, M., . . . Ziegler, D. (2017). High-leverage practices in special education. Arlington, VA: Council for Exceptional Children & CEEDAR Center. McLeskey, J., & Brownell, M. (2015). High-leverage practices and teacher preparation in special education (Document No. PR-1). Retrieved from University of Florida, Collaboration for Effective Educator, Development, Accountability, and Reform Center website: http://ceedar.education.ufl.edu/tools/best-practice-review/ Morgan, J. L. (2016). Reshaping the role of a special educator into a collaborative learning specialist. International Journal of Whole Schooling, 12(1), 40-60. Retrieved from http://www.wholeschooling.net/Journal_of_Whole_Schooling/IJWSIndex.html Otis-Wilborn, A., Winn, J., Griffin, C., & Kilgore, K. (2005). Beginning special educators’ forays into general education. Teacher Education & Special Education, 28(3/4), 143– 152. doi:10.1177/088840640502800401 Pufpaff, L. A., McIntosh, C. E., Thomas, C., Elam, M., & Irwin, M. K. (2015). Meeting the health care needs of students with severe disabilities in the school setting: Collaboration between school nurses and special education teachers. Psychology in the Schools 52(7), 683-701. doi:10.1002/pits.21849 Roberts, C. A., Benedict, A. E., & Thomas, R. A. (2013). Cooperating teachers’ role in preparing preservice special education teachers: Moving beyond sink or swim. Intervention in School and Clinic, 49(3), 174-180. doi:10.1177/1053451213496162

127 Roberts, K. D. (2010). Topic areas to consider when planning transition from high school to postsecondary education for students with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 25(3), 158-162. doi:10.1177/1088357610371476 Ruel, E. E., Wagner, W. E., III., & Gillespie, B. J. (2016). The practice of survey research: Theory and applications. Thousand Oaks, CA: SAGE Publications, Inc. Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Thousand Oaks, CA: SAGE Publications Inc. Schmalzried, J. E., & Harvey, M. W. (2014). Perceptions of special education and career and technical education collaboration and communication. Career Development and Transition for Exceptional Individuals, 37(2), 84-96. doi:10.1177/2165143412467666 Silverman, F. L., & Millspaugh, R. (2006). Physical proximity of occupation therapy and learning support instruction: How room sharing can promote collaboration for professionals and success for students. TEACHING Exceptional Children Plus, 2(4), 110. Retrieved from https://www.cec.sped.org/Publications/CEC-Journals/TEACHINGExceptional-Children/TEC-Plus Thomas, D. D. (2004). Use transportation as a related service. Intervention in School and Clinic, 39(4), 240-245. doi:10.1177/10534512040390040701 Weiss, M. P., Pellegrino, A., & Brigham, F. J. Practicing collaboration in teacher preparation: Effects of learning by doing together. Teacher Education and Special Education, 40(1), 65-76. doi:10.1177/0888406416655457 Woolf, S. B. (2015). Special education professional standards: How important are they in the context of teacher performance evaluation? Teacher Education and Special Education, 38(4), 276-290. doi:10.1177/0888406414557284 Zagona, A. L., Kurth, J. A., & MacFarland, S. C. Z. (2017). Teachers’ views of their preparation for inclusive education and collaboration. Teacher Education and Special Education, 40(3), 163-178. doi:10.1177/08884064176929 About the Authors Laurie A. Sharp, Ed.D., is an Associate Professor and the Dean of University College at Tarleton State University in Stephenville, Texas. Dr. Sharp promotes student success among adult learners, actively participates in professional service, and maintains an extensive research and publication record. Michelle Simmons, Ed.D., is an Assistant Professor of Special Education at West Texas A&M University in Canyon, Texas. Dr. Simmons is program coordinator in the educational diagnostician graduate program. Dr. Simmons’ research focuses on best practice in assessment and evaluation, multicultural evaluation and instruction for students with disabilities.

128

3D Printing as a Teaching Tool for People who are Blind and Visually Impaired Julia VanderMolen, Ph.D., CHES Jennifer Fortuna, Ph.D., OTR/L Grand Valley State University Abstract Three-dimensional (3D) printed materials have been found to promote spatial awareness in people who are blind and visually impaired. Research has shown 3D printed materials can make abstract concepts, such as severe weather, easier to understand. Much of the existing research related to weather and disability is focused on people with physical disabilities. Severe weather poses a safety threat. Therefore, it is important to address spatial awareness and understanding of severe weather patterns in this population. Developing 3D printed models of severe weather patterns (i.e., clouds and hail) is one way to accomplish this. Occupational therapy practitioners have the skills to develop 3D printed models for use as a teaching tool and assistive technology intervention. This study explores how 3D printed models influence spatial awareness and understanding of severe weather patterns (i.e., clouds and hail) in people who are blind and visually impaired. This study employed a nonexperimental descriptive design using an electronic survey to collect data from a nonprobability sample. Descriptive statistics were used to analyze the data. The information gained from this study can be used to inform future research and guide clinical practice. Keywords: occupational therapy, visual impairment, 3D printing, assistive technology, educational technology 3D Printing as a Teaching Tool for People who are Blind and Visually Impaired Globally, it is estimated that at least 2.2 billion people live with visual impairment or blindness (World Health Organization, 2019). The International Statistical Classification of Diseases 11 th Revision (ICD-11) defines visual impairment as deficits in the ability of the person to perform vision-related activities of daily living (ICD-11, 2019). Since vision is the primary sensory system used to gather information from the environment. People who are blind or visually impaired often rely on input from multiple sensory systems to orient themselves in the world. Touch is one of the most important senses used to compensate for the absence of vision in spatial tasks. Tactile training can enhance acquisition of spatial knowledge (Leo et al., 2017). Interactions with tangible materials such as braille labels, interactive maps, and threedimensional (3D) materials reinforce learning in people who are blind and visually impaired (Sandberg, 2016). Incorporating auditory input into educational activities provides an additional strategy for developing a conceptual understanding of the world. According to Hull (1990), the sound of rain can promote an understanding of differences in place, speed, and intensity. While seasonal weather changes may invigorate the senses in a person with sight, severe weather has the potential to disorient, threaten, and isolate people with visual impairment (Bell et al., 2018). These experiences can exacerbate feelings of marginalization. In addition, visual

129 impairment can make it difficult to avoid obstacles, problem-solve, and assess the environment for danger (Bell et al., 2018; Hull, 1990). Therefore, it is important to identify teaching tools that can be used to promote spatial awareness and understanding of abstract concepts, such as those common in geography, science, and mathematics (Giraud et al., 2017). Attending to the weather is essential if we are to fully understand people’s experiences of well-being, impairment, and disability (Bell et al., 2018). Much of the existing research on weather and disability is focused on challenges faced by people with physical disabilities. Few studies have explored how people who are blind and visually impaired experience the weather. 3D printed materials have been used to promote spatial awareness and understanding of abstract concepts in people who are blind and visually impaired. Research suggests the sense of touch compensates for the absence of sight by allowing for the coding of spatial patterns to emanate from tactile stimuli instead of visual input (Cohen et al., 2011; Ruggiero & Iachini (2010). Research has explored the translation of images of astronomical and biological objects into 3D models so students with visual impairment may explore the concepts through touch (Grice et al., 2015; Reynaga-Peña, 2015). A literature review by Ford and Minshall (2019) found 3D printing is often used to support science, technology, engineering and mathematics (STEM) curriculums in K-12 settings. In addition, 3D printed materials have been found to improve spatial awareness and understanding of spatial relationships between objects in special education settings (McLoughlin et al., 2016). For example, use of 3D printed tactile artifacts such as picture books and maps have been created to clarify abstract concepts for children who are blind or visually impaired (Jo et al., 2016). Occupational therapy practitioners have the training and education to develop educational interventions that will enhance spatial awareness and promote safety in people who are blind and visually impaired. Occupational therapists also have a unique understanding of the sensory systems which is necessary for promoting spatial awareness. The Occupational Therapy Practice Framework (4th Edition), categorizes safety, and emergency maintenance as an instrumental activity of daily living (IADL) (American Occupational Therapy Association [AOTA], 2020). Matters of safety often include activities considered to be more complex than everyday activities of daily living. Safety and emergency maintenance is defined as “Evaluating situations in advance for potential safety risks; recognizing sudden, unexpected hazardous situations and initiating emergency action; reducing potential threats to health and safety, including ensuring safety when entering and exiting the home” (AOTA, 2020, p. 31). In occupational therapy intervention, use of assistive technology may include education and training in the use of lowtech devices such as 3D printed materials. Creating 3D printed models to enhance understanding of severe weather patterns (i.e., clouds and hail) is one example. Few studies have examined how people who are blind and visually impaired perceive severe weather. In addition, the field of occupational therapy lacks research providing guidance on the use of 3D printed materials in practice. The purpose of this study is to explore how 3D printed models of severe weather patterns may influence spatial awareness and understanding of severe weather patterns in people who are blind and visually impaired. This study aims to answer the following research question: How do 3D printed models influence understanding of severe weather patterns in people who are blind and visually impaired? The information gained from this study can be used to inform future research and guide occupational therapy practice.

130

Method This study was approved by the Institutional Review Board at Grand Valley State University. This study employed a nonexperimental descriptive design using an electronic survey to collect data from a nonprobability sample. Participants attended a severe weather training program. Data was collected using a cross-sectional electronic survey. Descriptive statistics were used to analyze data. Participants and Sampling Researchers recruited a sample of adults who are blind and visually impaired. Participants were recruited in collaboration with a local community organization providing services to people who are blind or visually impaired. To qualify for this study, participants had to meet the following inclusion criteria: (a) over the age of 18; (b) either blind or visually impaired; (c) English speaking; and, (d) must have attended the SkyWarn Severe Weather Training. Of the 40 potential participants recruited, 35 people met the full inclusion criteria. Procedures SkyWarn Severe Weather Training Program Participants attended a SkyWarn Severe Weather Training Program led by a meteorologist from the National Weather Service. The four hour training took place at a local church located in Grand Rapids, Michigan. The SkyWarn Program is a nationwide network of over 350,000 trained volunteers who report severe weather to the National Weather Service (National Weather Service, n.d.). Storm spotters attend training to learn how to identify potential severe weather features such as clouds, wind, and hail. For this study, the researchers organized a SkyWarn training specifically for people who are blind and visually impaired. This training was different from traditional SkyWarn programs because the researchers incorporated several 3D printed models of cloud formations and hail into the course materials. 3D Printed Models 3D printed models representing severe weather patterns (i.e., different types of clouds formations and hail sizes) were provided to abstract concepts (i.e., severe weather patterns) easier to understand. The researchers collaborated with coordinators from the SkyWarn Storm Spotter Program to determine which cloud formations and hail models were required to fit the training needs of study participants. The following cloud types and hail sizes were developed for this study using a 3D printer: funnel clouds, wall clouds, dime size hail, quarter size hail, and golf ball size hail. The measurements of each model are referenced in Table 1. In addition to these models, the researchers incorporated another 3D model, referred to as “Mr. Fuzzy,” to assist with illustrating the conceptual components of a thunderstorm. The Mr. Fuzzy model was shaped like a wall cloud and made from cotton. Table 1. Measurements of 3D Models for Severe Weather Training Type of Model

Height (in mm)

Length (in mm)

Width (in mm)

131 Funnel Cloud

100.76

101.02

Wall Cloud

115

Hail (Golf Ball)

42.70

Hail (Quarter)

24.26

Hail (Dime)

17.91

Once the model dimensions were determined, the Ultimaker 2+ Extended Model 3D printer was used to create the models. The Ultimaker 2+ printer was readily available at the researchers’ institution. This printer was selected for its print quality, ease of use, and reliability. The cost was also a factor. The price range for this printer was approximately $2,500. Educators and researchers can explore similar 3D printers using 3D Hubs, an online index that compares printers by price and customer review (3D Hubs, 2019). Another benefit of the Ultimaker 2+ printer is that the company makes Polylactic Acid (PLA) filament that is affordable and easy to print. PLA is a dense plastic material known for its durability and highly detailed 3D printing capabilities (Wagner & Autodesk Inc., 2017). For this project, the cloud formations and hail models were designed to feature specific details. Thus, PLA was identified as the optimal choice. The cloud formation files were created using Tinkercad, a computer-aided design (CAD) software with the ability to create designs, as well as search for existing 3D models in a vast repository. Manipulating various common and identifiable shapes allowed the cloud formation and hail model designs to be created by referencing a photo of the correct cloud type. The 3D formulating process varies depending on the skill level of the developer. Tinkercad has a simple interface that can be used to create designs based on a scientific concept. On average, the design process for each cloud model took approximately 30-minutes (see Figures 1 and 2). The figure files are available on Thingiverse, a website repository dedicated to sharing 3D digital design files (Makerbot Industries, 2019).

132 Figure 1. Digital representation of the wall cloud model created using Tinkercad.

Figure 2. Digital representation of the funnel cloud model created using Tinkercad. In total, it took the researchers two to three hours to accurately define six different cloud formation models. The process to render each file took an additional 30 minutes. The time required to print one cloud model on a commercial-sized 3D printer was 30-minutes to one hour depending on the size of the extrusion nozzle. Examples of 3D printed cloud types and hail sizes are provided in Figures 3-5.

133 Figure 3. Photograph of a 3D printed wall cloud.

Figure 4. Photograph of a 3D printed funnel cloud.

Figure 5. Photograph of various 3D printed hail sizes with braille labels. Survey Instrument

134 After the training, participants were invited to provide anonymous feedback on the SkyWarn training. An electronic survey was selected due to convenience, cost, and rapid turnaround in data collection. The researchers created an 11-item electronic survey instrument using Qualtrics software (Qualtrics, 2019). This software was selected because it is accessible to participants who rely on optical devices such as screen readers. The survey gathered feedback from participants on the benefits of using 3D models to learn about severe weather. The purpose of the first question was to provide instructions and obtain informed consent. Of the remaining questions, five requested open-ended feedback on the 3D models and sound clips; and, five questions were related to content delivery and could be answered with a ‘yes’ or ‘no’ response. To reduce threats to content validity, experts from the field of blindness and low vision. Additionally, the survey was pilot tested and later refined based on feedback provided by two individuals who are blind. Pilot testing did not result in changes to the survey questions; however, formatting changes were necessary to ensure all participants using a screen reader could gain access. Variables This study incorporated 3D models into the SkyWarn training to promote understanding of spatial concepts including cloud formations and hail sizes. Audio clips were also embedded into the training to reinforce learning. By including the audio clips, participants were able to both hear and feel the differences between hail sizes. Although audio clips are not a variable of interest in this study; the researchers acknowledge the potential influence this may have on participant understanding of severe weather patterns. For this reason, audio clips are a confounding variable in this study. Table 2 relates key variables to survey items. Table 2. Variables and Related Survey Items Variable Name Independent 3D Models

Survey Item Questions 2, 3, 7, 8, 9

Dependent

Understanding

Questions 2, 3, 5, 7, 8, 9, 10

Confounding

Audio Clips

Questions 2, 3, 10

Data Analysis Data analysis was completed using IBM SPSS Statistics, Version 25 (IBM Corporation, Armonk, NY). A descriptive numerical summary and qualitative thematic analysis were used to summarize the results. The researchers analyzed open-ended survey responses with a basic coding process to identify themes. Thematic analysis was used to identify themes around core concepts. Results In total, 20 participants completed the full survey, resulting in a 57.14% overall response rate. All 20 participants (100%) reported the 3D models promoted understanding of cloud formations and hail sizes. Furthermore, all 20 participants (100%) reported the sound clips were helpful. Several participants stated the sound clips for wind and hail were especially helpful. Study participants suggested the researchers create additional 3D models that can be incorporated into

135 future training. The majority of participants (90%) found Mr. Fuzzy helpful for understanding what a severe thunderstorm looks like. A summary of survey questions, responses, and common themes are summarized in the order they were presented in Table 3. Table 3. Survey Questions and Responses Question 1.

Informed consent and instructions.

In general, what did you like, or find useful from the session?

Was there anything you did not like about the session?

Response

“Having audio clips to go along with 3D models.” “The 3D models were especially useful.” “The sound clips and 3D representations. It was good to hear what different hail sizes sound like as they fall.” “I enjoyed learning how the blind can also help with tracking weather.” “As someone interested in the weather, I enjoyed hearing about different types of severe weather and using the models of clouds from a thunderstorm.” “No” (n = 18) (90%) “The sound examples of hail stones went on too long; 15 seconds should be enough for people to get the idea.” “As a weather enthusiast, I would argue the session was too short.”

Question

Response (n = 20), n (%)

Did the presenter from the National Weather Service keep your interest throughout the session? Were the presentation slide descriptions helpful, and in an accessible format?

“Yes” (n = 19) (95%) “No” (n = 1) (5%)

Did you have adequate assistance before, during and after the training session?

“Yes” (n = 20) (100%) “No” (n = 0)

Do you think the 3D models helped you learn and understand hail sizes and cloud formations?

“Yes” (n = 20) (100%) “No” (n = 0)

136 8.

Were some images more helpful than others? If so, please explain.

“Yes, the 3D prints were better.” “Yes, I would have liked the 3D models larger.” “Having the models labelled in braille helped with all of the images.” “The wall and funnel cloud images were especially helpful.” “I liked the models and would love to see more.”

Was Mr. Fuzzy useful in terms of helping you understand what a severe thunderstorm looks like?

“Yes” (n = 18) (90%) “No” (n = 2) (10%)

10.

Did you find the sound clips to be helpful? Please explain.

“Yes” (n = 20) (100%) “Great calibration for wind speed.” “It helps to understand the difference in the sound at different wind speeds and hail sizes.” “They were helpful. It’s good to be able to hear the contrasting sounds of various wind speeds and hail sizes.” “I didn’t realize that with a tornado there are a variety of sounds you can hear. The different wind speed sounds were very helpful in distinguishing approximate wind speeds.”

11. What suggestions do you have for future sessions?

“Keep adding audio clips. Maybe one of rushing flood water. Talk about the difference in sound and how it affects mobility when snow is on the ground.” “More 3D cloud formations. The funnel cloud was great and the hail stones too.” “3D images of additional cloud types and thunderstorm components would be helpful.”

Discussion This study explored how 3D printed models influence understanding of severe weather patterns in people who are blind and visually impaired. Additionally, the researchers aimed to provide guidance on the use of 3D printing in occupational therapy practice. The results of this study found incorporating 3D models into the SkyWarn Training had a positive influence on understanding of severe weather patterns. All participants reported the 3D models enhanced their understanding. Several participants requested that the researchers create additional 3D cloud formations that can be added to future trainings. A study by Jo et al., (2016) also found that 3D

137 printed materials increased comprehension for blind and visually impaired students who found oral explanations difficult to grasp. One major advantage of 3D printing technology is the ability to create individualized materials for participants and students with varying degrees of visual impairment. 3D printing software allows the creator to design customized models and to scale down large objects to highlight core features and increase comprehension through touch (Jo et al., 2016). This study identified several advantages to incorporating 3D printing into occupational therapy practice and educational settings. For example, 3D printing provides an accessible, affordable, and adaptable means of assistive technology. There is a wide range of applications and opportunities for creativity and problem solving (Schwartz, 2018; Ganesan, Al-Jumaily & Luximon, 2016). Integrating Braille as a modality for treatment is one example. 3D printers are available at affordable prices ranging between $500 for basic printers, up to $5,000 for the more advanced models. Finally, occupational therapists who incorporate assistive technology are often limited to costly, pre-fabricated devices that may not meet their client’s individual needs (Schwartz, 2018). 3D printing also makes it easy to individualize treatment in a cost-effective manner. Limitations No study is without limitations. The results of this study were limited to a small sample of participants with visual impairment. The individual vision-related diagnoses and severity level (e.g., visual acuity) of visual impairment was not reported in this study. In addition, audio clips served as a confounding variable that was not controlled. These limitations may impact the generalizability of results to the greater population of people who are blind or visually impaired. During the process of creating the 3D cloud formations and hail sizes, the following limitations were recognized. There was a learning curve when learning to use the Tinkercad software. Also, the researchers discovered they needed additional cloud models to accommodate a large number of participants at the training. The inclusion of additional cloud types and more components would allow for more efficient use of time for the study participants. The restriction of the size with the 3D printing hardware itself was also a limitation. The cloud and hail model sizes were limited to 7.5 inches in both width and length. Finally, having larger 3D models to accompany the audio explanations may have reinforced learning. Lessons Learned The researchers faced a few challenges as the project unfolded. First, using 3D modeling software programs involves a learning curve. However, the researchers found having basic familiarity with CAD helped to mitigate the issue. Additionally, accessibility varies greatly and is determined by the needs of the individual with visual impairment. For example, with Braille, the legibility varies when using uppercase versus lowercase letters. Thankfully, Thingiverse (Makerbot Industries, 2019) offers a braille collection with a multitude of predesigned files. Because both designers and users must understand and know how to implement changes to such variables, stakeholders must track many elements throughout the project. Designing the cloud formations and hail replicas helped the researchers acquire a better understanding of the 3D printing process. Before starting the process, there was a limited

138 understanding of CAD software and Tinkercad. Throughout the design process, comprehension of the simplistic interface began to improve. This provided an opportunity for the researchers to use the software to its fullest capabilities. By creating the 3D designs to be as fluid as possible, the software allowed for a deeper understanding of creating interlocking parts and how to best represent 2D images on a 3D platform. By creating the 3D formations, the program provided a newfound respect for the printing process and how to best suit a 3D design file to print with expected quality. Conclusion This study examined how 3D printing can influence understanding of severe weather patterns in people who are blind and visually impaired. In addition, this study aimed to identify how occupational therapists may benefit from using 3D printing in clinical practice. The information gained from this project may be valuable to health care professionals, educators, and individuals who are blind and visually impaired. Teachers and other health care professionals may use this information to develop 3D printed materials to reinforce learning of abstract or science-related concepts. This information can also be used to assist occupational therapists interested in using 3D printed models to increase understanding of spatial concepts in people with visual impairment. Health care professionals can also use this information to develop individualized assistive technology interventions. 3D printed models as assistive technology may reinforce learning, safety, and engagement in occupations for people with visual impairment. Implications for the Field Occupational therapy practitioners prescribe assistive technology; however, they are often limited to costly, pre-fabricated devices that may not meet their client’s individual needs (Schwartz, 2018). This study demonstrates how occupational therapy practitioners can use 3D printing to design and manufacture their own assistive technology and educational tools that can be incorporated into intervention. Moreover, 3D printing can increase access to assistive devices for clinics serving the underserved, including those in low-income and rural facilities. Future research with a larger and more representative sample, blinding, and a control group is suggested. Furthermore, more experimentation and collaboration are needed to establish the best practice of 3D printed assistive technology and to meet the individual needs of people with visual impairment. References 3D Hubs. (2019). 3D printer guide. https://www.3dhubs.com/best-3d-printer-guide/ American Occupational Therapy Association [AOTA]. (2020). Occupational therapy practice framework: Domain & process (4th ed.). Bethesda, MD: American Occupational Therapy Association. https://doi.org/10.5014/ajot.2014.682006 Bell, S. L., Leyshon, C., & Phoenix, C. (2018). Negotiating nature’s weather worlds in the context of life with sight impairment. Transactions of the Institute of British Geographers, 44(2), 270-283. https://doi.org/10.1111/tran.12285

139 Cohen, H., Scherzer, P., Viau, R., Voss, P., & Lepore, F. (2011). Working memory for Braille is shaped by experience. Communicative & Integrative Biology, 4(2), 227-229. https://doi.org/10.4161/cib.4.2.14546 Ford, S., & Minshall, T. (2019). Invited review article: Where and how 3D printing is used in teaching and education. Additive Manufacturing, 25, 131-150. https://doi.org/10.1016/j.addma.2018.10.028 Ganesan, B., Al-Jumaily, A., & Luximon, A. (2016). 3D printing technology applications in occupational therapy. Physical Medicine and Rehabilitation International, 3(3), 1-4. Giraud, S., Brock, A. M., Macé, M. J., & Jouffrais, C. (2017). Map learning with a 3D printed interactive small-scale model: Improvement of space and text memorization in visually impaired students. Frontiers in Psychology, 8(930), 930. https://doi.org/10.3389/fpsyg.2017.00930 Grice, N., Christian, C., Nota, A., & Greenfield, P. (2015). 3D printing technology: A unique way of making Hubble space telescope images accessible to non-visual learners. Journal of Blindness Innovation and Research, 5(1). https://nfb.org/images/nfb/publications/jbir/jbir15/jbir050101.html Hasper, E., Windhorst, R. A., Hedgpeth, T., Tuyl, L. V., Gonzales, A., Martinez, B., . . . Baluch, D. P. (2015). Methods for creating and evaluating 3D tactile images to teach STEM courses to the visually impaired. Journal of College Science Teaching, 44(6), 92-99. http://dx.doi.org/10.2505/4/jcst15_044_06_92 Hull, J. M. (1990). Touching the Rock: An experience of blindness. London, UK: SPCK Publishing. I, J. H., Harianto, R. A., Chen, E., Lim, Y.S., Jo, W., Moon, M-W., & Lee, H. J. (2016). 3D Literacy aids introduced in classroom for blind and visually impaired students. Journal of Blindness Innovation and Research. 6(2). https://nfb.org/images/nfb/publications/jbir/jbir16/jbir060203.html IBM Corporation. IBM SPSS Statistics Version 25. Armonk, NY. International Statistical Classification of Diseases 11 th Revision [ICD-11]. (2019). Vision impairment including blindness. https://icd.who.int/browse11/lm/en#/http%3a%2f%2fid.who.int%2ficd%2fentity%2f1103667651 Janson, J. R. (2015). Using 3D printing. OT Practice, 33. Jo, W., I., J. H., Harianto, R. A., So, J. H., Lee, H., Lee, H. J., & Moon, M. W. (2016). Introduction of 3D printing technology in the classroom for visually impaired students. Journal of Visual Impairment & Blindness, 110(2), 115-121. https://doi.org/10.1016/S0890-6955 https://doi.org/10.1016/S0890-6955(97)00137-5 Kolitsky, M. (2014). 3D printed tactile learning objects: Proof of concept. Journal of Blindness Innovation and Research. 4(1). https://nfb.org/images/nfb/publications/jbir/jbir14/jbir040102.html Leo, F., Cocchi, E., & Brayda, L. (2017). The effect of programmable tactile displays on spatial learning skills in children and adolescents of different visual disability. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(7), 861-872. https://doi.org/10.1109/tnsre.2016.2619742 McLoughlin, L., Fryazinov, O., Moseley, M., Sanchez, M., Adzhiev, V., Comninos, P., & Pasko, A. (2016). Virtual sculpting and 3D printing for young people with disabilities. IEEE Computer Graphics Application, 36(1), 22-28. https://doi.org/10.1109/MCG.2016.1

140 National Weather Service. (n.d.). NWS SkyWarn Storm Spotter Program. https://www.weather.gov/skywarn/ Qualtrics. (2019). https://www.qualtrics.com/ Ramachandran, V. (2014). Beyond braille: 3-D printed books for the blind. NPR.org. http://www.npr.org/sections/alltechconsidered/2014/08/13/340110203/weeklyinnovation-you-don-t-have-to-be-able-to-see-to-enjoy-picture-book Reynaga-Peña, C. G. (2015). A microscopic world at the touch: Learning biology with novel 2.5D and 3D tactile models. Journal of Blindness Innovation and Research, 5(1). 10.5241/5-54 Ruggiero, G., & Iachini, T. (2010). The role of vision in the Corsi block-tapping task: evidence from blind and sighted people. Neuropsychology, 24(5), 674-679. https://doi.org/10.1037/a0019594 Sandberg, R. (2016, January 7). 3D printing for blind people: The future potential of a cutting edge technology. http://www.incobs.de/articles/items/3d.html Salinas, S. (2018). Therapists take 3D printing to a new dimension. Rehab Management: The Interdisciplinary Journal of Rehabilitation, 30–32. http://www.rehabpub.com/2018/09/therapists-take-3d-printing-new-dimension/ Schwartz, J. (2018). A 3D-printed assistive technology intervention: A phase I trial. The American Journal of Occupational Therapy, 72(4), Suppl. 1. 10.5014/ajot.2018.72S1-RP302B Thingiverse. (2019). Makerbot Industries, LLC. https://www.thingiverse.com/ Wagner, C., & Autodesk Inc. (2017, March 1). Pick the perfect material for your 3D printing job. https://knowledge.autodesk.com/support/fusion-360/gettingstarted/caas/simplecontent/content/pick-the-perfect-material-for-your-3d-printingjob.html Wagner, J. B., Scheinfeld, L., Leeman, B., Pardini, K., Saragossi, J., & Flood, K. (2018). Three professions come together for an interdisciplinary approach to 3D printing: Occupational therapy, biomedical engineering, and medical librarianship. Journal of the Medical Library Association, 106(3), 370-376. 10.5195/jmla.2018.321 World Health Organization. (2019). Blindness and vision impairment. https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment About the Authors Julia VanderMolen, Ph.D., CHES, is an Associate Professor of Public Health with Grand Valley State University in Michigan. The contribution of her research is to examine the benefits of assistive technology, UD and UDL. Additionally, Dr. VanderMolen’s recent work has included the benefits of 3D printing for the visually impaired, the concept of universal design and learning and the use of mobile technology to assist individuals with disabilities. Jennifer Fortuna, Ph.D., OTR/L, is an occupational therapist and Assistant Professor at Grand Valley State University in Grand Rapids, Michigan. Dr. Fortuna teaches graduate courses in the Department of Occupational Science and Therapy. Her research interests include health literacy, visual impairment, accessibility, and universal design.

141 A Comparison of Adolescents’ Goals for One and Five Years After High School Graduation Kendra Williams-Diehm, PhD, BCBA University of Oklahoma Penny Cantwell, PhD. Oklahoma State University Kylie Lyons, PhD University of Oklahoma Malarie Deardorff, PhD University of Oklahoma Patricia Lynch, PhD Texas A&M University Abstract Goal setting is an integral part of the self-determination process and is considered an essential element of self-determined behavior. In this study, 189 high school seniors with and without disabilities were asked to write postsecondary goals for both one-year and five-years following high school. Goals were classified into four categories and then examined through a series of chisquare analyses to determine differences between groups of students. Differences were found between students in general education and students receiving special education services based upon goals for postsecondary education, employment, and independent living at both levels. The most notable difference was in the category of postsecondary education. Students with disabilities were less likely to articulate continuity among goals, demonstrating a lack of logical progression across one-year and five-year goals. Implications are discussed and suggestions made for teachers to promote goal-setting skills and continuity across short-term and long-term goals with students. A Comparison of Adolescents’ Goals for One and Five Years After High School Graduation The Individuals with Disabilities Education Act (2004) requires students with disabilities be invited to their Individual Education Program (IEP) meetings. These meetings provide an ideal opportunity for students with disabilities to be active participants, allowing them to articulate their personal desires and aspirations, to advocate for themselves, and to set goals for their futures. Not only is encouraging students with disabilities to perform these self-determined actions best practice (Test et al., 2009), fostering students’ self-determination directly connects students to transition planning and is therefore ultimately linked to students’ positive academic and post-school outcomes (Konrad et al., 2007; Lee et al., 2010). An important aspect of transition planning includes students’ making plans for their lives and developing goals related to employment, education, independent living and participation in recreation and leisure activities. It is critical that students learn and practice goal setting, progress monitoring, and goal attainment to help ensure that content of student-developed goals is complete, realistic, and

142 follows a logical progression of steps. Students must also understand their own strengths and needs, choose their own goals, self-advocate, and self-assess (Carter et al., 2006) for them to achieve their desired post-school outcomes. Research indicates students can learn such selfdetermined behaviors as goal setting when directly instructed (Algozzine et al., 2001), provided feedback, and given opportunities for practice. Instruction in self-determination skills enriches the general and special education curricula for students with disabilities by creating opportunities for academic and transition-related goal setting, identification of processes related to those goals, and opportunities for self-monitoring and self-evaluation (Shogren et al., 2012). Goal setting requires the cognitive ability to organize abstract ideas and differentiate problem-solving strategies in order to be effective (Lenz et al., 1991). Consequently, students with disabilities may need additional instruction and support to acquire goal-setting skills (Locke & Latham, 2002). Through explicit instruction, students with disabilities can learn to appropriately set goals (Shogren et al., 2012). Student participation in this goal development process critical, as instruction in these skills prepares students to participate in and lead IEP meetings where self-determination and transition planning are discussed and goals and objectives are written (Martin & Williams-Diehm, 2013; Trainor, 2005; Ward & Kohler, 1996; Wood et al., 2004). Self-Determination and the Relation of Goal Setting and Goal Acquisition Since the early 1990s, educators have been studying self-determination as an educational outcome for individuals with disabilities. Self-determined persons “make or cause things to happen in their own lives” (Wehmeyer & Shogren, 2008, p. 95). According to Wehmeyer and Shogren (2013), individuals who are self-determined know their own interests, strengths, and needs, make decisions, set goals, and carry out the necessary steps to ensure their goals are met. Students with higher self-determination skills more effectively accomplish goal setting; however, students with disabilities have been found to be limited in self-determination skills (Wehmeyer, 1994; Carter et al., 2006). Although best practice supports active participation in students’ setting and implementing goals for their transitions (Martin & Williams-Diehm, 2013), research indicates students with disabilities do not know how to set and implement goals. Goal setting is considered an essential component of self-determination (Wehmeyer, 1994), but the current literature base presents evidence that students do not always know how to set goals, much less attain them (Martin et al., 2014). Goal attainment is a vital component of studentdirected or self-regulated learning. Zimmerman, Bandura, and Martinez-Pons (1992) proposed “self-regulated learners are not only distinguished by their proactive orientation and performance, but also by their self-motivated capabilities” (p. 664). Because adolescents with disabilities tend to lack self-motivation strategies, goal setting and attainment are difficult tasks for them to master (Ellis & Larkin, 1998). Students with disabilities may have more difficulty developing these skills than students without disabilities. Some scholars suggest students with disabilities begin learning how to set goals and work toward goal attainment in elementary years so that students have at least a rudimentary, goal-setting skill base evident when they enter high school (Martin et al., 1993). However, more research is needed in this area to determine the level of students’ understanding regarding what goals are (Martin et al., 2014) and the need for continuity between long-term and short-term goals.

143 In a study examining the effects of a learning program for students with mild to moderate disabilities, Martin et al. (2014) found students were more likely to set and achieve goals after learning a strategy (plan, act, evaluate, adjust) for accomplishing the goals they set. Apparently, without a strategy for achieving the goals they establish, students perceive goal-setting activities as mere academic exercises with no real-life purpose or concrete meaning. Although students may be included in the goal-setting process, because most high school students know little about how to accomplish goals, teaching students goal setting distinct from goal-achieving strategies remains inadequate. Students need both guidance and an additional skill set to determine how to meet their goals. Goal acquisition involves the ability to make an action plan and “act, evaluate, and adjust” the plan as necessary (Martin et al., 2014, pg. 73). One important factor in goal acquisition is the ability to adjust action plans that are not working. Martin et al. (2014) found students were likely to continue with their established plan even after students determined their plan was unlikely to help them reach their goals. One can only deduce Martin et al.’s finding to reveal students lacked the support they needed to adjust their action plans in ways to support their achieving the goals they had identified for themselves. Nature of Goals Experience, developmental level, self-knowledge, and support influence the goals students set for themselves and their ability to attain those goals. The National Longitudinal Transition Study-2 (NLTS-2) indicates most 15- to 19-year-olds with disabilities expect to graduate from high school, get a job, and live independently, while fewer expect to continue with postsecondary education (Wagner et al., 2007). These types of postsecondary goals are typical, normal, and healthy for all students, regardless of disability; however, for all students to have realistic goals, they need to understand the connection between goal setting and postsecondary outcomes. In a study of self-determination of diverse students with learning disabilities Trainor (2005) found five themes related to transition and postsecondary goals: (a) a lack of connection between transition plans and postsecondary plans, (b) lack of involvement of participants in transition planning, (c) no discussion of transition planning between students and their teachers, (d) demonstration of some self-determination skill components, and (e) identification of postsecondary education as a goal. In a study of adolescent girls with learning disabilities, Trainor (2007) noted the participants had goals but lacked self-knowledge and did not understand the relation between strengths, needs, and goals. Although scholars have published extensively on goal setting and attainment processes for students with disabilities, few have provided details on the types of goals their participants set. Williams-Diehm, Palmer, Lee, and Schroer (2010) defined goals as either product or process. Product goals have an identified end; for example, getting an A in a certain course or graduating from high school are product goals. Process goals are those that assist an individual on the path to product goals. For example, spending 20 minutes each evening on homework assignments or finishing homework assignments to meet the deadline are process goals or helpful steps toward reaching product goals (Williams-Diehm et al., 2010). This study examined 1-year and 2-year goals set by students with intellectual disabilities. Findings showed differences in the goal type (process vs. product) across participants and suggested students across ability levels can effectively articulate and make progress on goals (Williams-Diehm & Palmer, 2018). WilliamsDiehm et al. (2010) also found differences between middle school students and high school students; middle school students were more likely than high school students to establish product-

144 oriented goals. As students get older, they became more aware of the future and the need to set goals related to postsecondary objectives (Williams-Diehm et al., 2010). As students progressed, they became better equipped to think about long-term and abstract goals and implement process goals to assist them in attaining their personal goals. An additional area of interest is students’ understanding of continuity between short- and longterm goals and the relation between goals and postsecondary aspirations. Research indicates that goals of individuals are often related to the context for the individual at that time (Dunlop et al., 2017). As student’s progress through adolescence, their goals will likely change (WilliamsDiehm et al., 2010). When an individual sets multiple goals, the interrelatedness and sequential nature of those goals must be considered (Austin & Vancouver, 1996), and analysis of the product and processes of the goals should be studied. Students with disabilities struggle with continuity even before they have to examine the relation between their goals and postsecondary aspirations. Williams-Diehm et al. (2010) are the only scholars known to have attempted a student goal content analysis. After a comprehensive search through educational databases, nothing was found in terms of goals and differences among gender, ethnicity, and socioeconomic level. Purpose of the Study The purpose of this study was to examine content of postsecondary goals indicated by high school seniors to determine whether or not the goals included aspects of complete, realistic transition planning and demonstrated continuity (i.e., was the one-year goal related to the fiveyear goal in a sequential manner?), and examine differences between groups based on disability status, gender, ethnicity, and socioeconomic status. I think you need to explicitly state your research questions here. Method Participants Data were collected in a mid-sized school district (over 14,000 students) located in a large southwestern state. The population of interest included all graduating students from both the traditional high school and three alternative high school campuses. The sample was selected to include both general and special education students to maintain a strong comparison group. General education students were defined as students educated completely in general education classrooms; special education students were defined as students with a current Individual Education Program (IEP). The special education sample (n = 76) was selected using purposeful sampling to include students with disabilities who were completing their graduating year of high school and were able to accurately respond to the survey questions either independently or with assistance. Seventy-eight potential special education participants were identified, but after consulting with special education teachers, it was determined that two students with significant disabilities did not meet the study criteria. The general education sample was selected using a stratified random sample based upon the gender and ethnicity of the special education population, then doubled (n = 152). The larger sample of general education students helped provide additional power for statistical analyses and

145 helped correct any sampling error (Hinkle et al., 2003). The total sample consisted of 228 students; the total response rate was 82.9% (n = 189). The response rate for students receiving special education services was 78.9% (n = 60), while the response rate for students in general education was 84.9% (n = 129). Surveys were administered between two and three weeks prior to graduation on the students’ high school campuses. Students who did not participate in the study were either not available (absent) or declined to participate. The following information was obtained from the researcher’s notes. Fourteen students declined taking the survey (special education, n = 8; general education, n = 6). Researchers were unable to reach 25 students because they (a) were absent on all three days of survey administration at their respective campuses, (b) did not complete the survey left with their teacher, (c) or did not respond to both phone and postal mail contact attempts. Therefore, a total number of responses was 189; 60 received special education services and 129 participants were in general education and did not receive special education services. Researchers drew all student participants’ demographic information—gender, ethnicity, disability category (if applicable), and free-or-reduced lunch status—from official school-district records. This information appears in Table 1. The sample resulted in an equal ethnic breakdown of roughly one-third each of Black, Hispanic, and White students, which reflects the district and greater community’s population. In terms of student participants’ disability categories, the largest categories were learning disability (76.7%), intellectual disability (8.3%), and auditory impairment (6.7%). Other disability categories had one or fewer participants for each category. These disability categories are reflective of more mild disabilities. Individuals who were unable accurately to answer questions (i.e., were nonverbal) were excluded from the original sample as described earlier. As a result, the special education sample included more students with mild to moderate disabilities than students with severe disabilities. Table 2 shows the breakdown of disability category. Table 1 Student Demographic Data Total Sample Gender Male Female Ethnicity Black White Hispanic Social Economic Status Free/Reduced Lunch No Free Reduced Lunch N = 189 Table 2 Disability Categories Other Health Impairment

Special Education 60 (31.7%)

General Education 129 (68.3%)

30 (15.9%) 30 (15.9%)

61 (32.3%) 68 (36.0%)

19 (10.0%) 20 (10.6%) 21 (11.1%)

45 (23.8%) 41 (21.7%) 43 (22.8%)

37 (19.7%) 23 (12.2%)

64 (34.0%) 64 (34.0%)

1 (1.7%)

146 Auditory Impairment Intellectual Disability Emotional Disturbance Learning Disability Traumatic Brain Injury N = 59

4 (6.7%) 5 (8.3%) 1 (1.7%) 46 (76.7%) 2 (3.3%)

Procedure Participants were asked to complete a comprehensive high school exit survey required by the state that included questions regarding high school activities and preparation followed by questions regarding postsecondary employment, education, independent living, and communityintegration plans. The final portion of the survey included two, open-ended questions: (1) What do you see yourself doing ONE year after leaving high school? and (2) What do you see yourself doing FIVE years after leaving high school? Students were asked to respond to the two openended questions and told that one-year goals should lead to five-year goals. Students in special education were administered the survey in a small group (fewer than 10 students) or individually by the special education teacher. Students who were unable to read the survey were read to individually. All general education students were administered the survey in a large group (10 or more students). All participants were part of a larger postsecondary exit and one-year follow-up study. Students’ goals were categorized according to four areas: (a) postsecondary education, (b) employment, (c) independent living, and (d) community integration/recreation and leisure. These categories were used for both one-year and five-year goals; however, the category of community integration/recreation and leisure was eventually excluded from all analysis because few participants included goals in this area. Goals were then coded according to type of school, job type, and amount of living support. Codes for the goal areas are included in Tables 3 and 4 and are based on categories included in the NLTS-2 (Wagner et al., 2007) and found previously in the larger NLTS-2. Additionally, a separate coding system was created to account for evidence of planning and continuity between one-year and five-year goals. Continuity was coded as “yes” or “no” and evidence of planning coded as no planning, minimal planning, or clear planning. Interrater Reliability Interrater reliability using Cohen’s Kappa (Landis & Koch, 1977) was applied to all coding. Two raters independently coded 25% of participants to obtain interrater reliability. As coding progressed, definitions of content areas were refined to ensure an acceptable level of reliability between coders; this included creating the category of “unrealistic” for each of the content areas. The “unrealistic” category was used to capture goals that were inappropriate for the one- and five-year timeframe following high school. While researchers passed no judgment upon those setting unrealistic goals, they noted participants’ making such goals neither knew the steps needed to attain the goals nor understood the sequential nature of those steps. For example, an unrealistic one-year, postsecondary-education goal would include being in medical school. Although attending medical school is a notable goal, the timeline is unrealistic because one must attend college before becoming eligible for medical school. Similarly, an unrealistic, one-year employment goal would be playing in the National Football League (NFL), for NFL regulations would prevent such employment the first year after high school graduation. Researchers also

147 adjusted by collapsing Professional and Technical School responses since few students responded in either category. Next, all coding was reviewed to be certain that the refined definitions were included throughout the data set. All interrater reliability results are based upon the complete coding system. Interrater reliabilities for one-year goals were found to be K = .948 (education), K = .955 (employment), and K = .939 (independent living). Five-year goal interrater reliabilities were K = .947 (education), K = .964 (employment), and K = .868 (independent living). Because all numbers are above 0.81 as determined by Cohen’s Kappa (Landis & Koch, 1977), the interrater reliability for all categories is considered excellent, reflecting a high level of agreement between raters. The one-year and five-year goals were then examined further to check for evidence of planning and goal continuity. Evidence of planning reflects participant’s thoughtfulness and knowledge of how to realize the goals they have set. Both one-year and five-year goals were coded on a threepoint scale: (1) no planning, (2) minimal planning, or (3) clear planning. No planning indicates that the participants wrote a goal that was unfeasible given the timeline of interest (one-year or five-year) or set no goal. Examples of a goal demonstrating “no planning” include “Having a lot of money and a supermodel girlfriend to ride on the back of my motorcycle” and “I don’t know, only time will tell.” Participants evidencing “minimal planning” reveal having given some thought to the steps required to achieve a given goal. Examples of a goal demonstrating minimal planning include “I plan to go to community college and find a job somewhere” and “Getting ready to go to college but I need to get a job and save money first.” “Clear planning” indicates the participant has not only given thought to the goal but also has already put steps into place to accomplish the goal. Examples of a goal demonstrating clear planning include “completing my first year at [specific 4-year university] and living in the dorms,” or, “I have been signed as the punter for [specific college] and coming home to visit family on the weekends.” Interrater reliability for all categories was found using Cohen’s Kappa for both one-year (K = .893) and five-year (K = .932). The concept of continuity is defined as the one-year goal serving as a sub-goal that leads to the five-year goal. Coding was a simple (a) yes or (b) no. Again, Cohen’s Kappa was used to determine interrater reliability (K = .905). An example of goal continuity includes a one-year goal of “studying psychology at [specific 4-year university]” followed by a five-year goal of “potentially starting graduate school.” An example of a goal lacking continuity includes “after high school I will go to [specific community college] to become a firefighter” followed by a fiveyear goal of “I will be a teacher with a girlfriend.” The goals of becoming a firefighter and teacher do not follow the same trajectory. Again, all interrater reliability regarding evidence of planning and continuity of goals was found to be excellent, reflecting a high level of rater agreement since all are above 0.81 as determined by Cohen’s Kappa (Landis & Koch, 1977). Analysis Descriptive statistics were analyzed to determine differences between one-year and five-year goals in terms of presence of disability, gender, ethnicity, and socioeconomic status. A Pearson Chi-square test was completed to determine where statistically significant differences occurred

148 across groups. This statistical analysis was determined to be appropriate due to the categorical nature of all variables (Franke et al., 2012). Results One-Year Differences Postsecondary Education Statistically significant results were found in terms of the presence of a disability and planning to enroll in postsecondary education goals as a one-year goal (2 (5, N = 183) = 27.13, p < 0.001). Significant differences were not found for one-year postsecondary education goals in terms of gender, ethnicity, and socioeconomic status. In terms of gender, more females than males expressed a goal of attending postsecondary education, specifically, at the community/junior college level (2 (5, N = 183) = 8.13, p = 0.15). In terms of ethnicity, Hispanic students identified postsecondary education as a goal less frequently than did students identifying as nonHispanic, with the latter two reporting the goal at a very similar rate (2 (10, N = 183) = 11.23, p = 0.34). Regardless of the level of socioeconomic status, the goal of attending postsecondary education was comparable2 (5, N = 182) = 2.12, p = 0.83). Employment No statistical significance was found in terms of one-year anticipated employment goals based upon presence of a disability (2 (3, N = 183) = 0.06, p = 0.996), gender (2 (3, N = 183) = 4.96, p = 0.18), ethnicity (2 (6, N = 183) = 7.65, p = 0.27), or socioeconomic status (2 (3, N = 181) = 3.81, p = 0.28). Independent Living Similar to the findings of one-year employment, no statistical significance was found between presence of a disability (2 (3, N = 182) = 0.60, p = 0.90), gender (2 (3, N = 182) = 4.97, p = 0.17), or socioeconomic status (2 (3, N = 181) = 2.58, p = 0.46), and planned living goals. Community Integration/Recreation and Leisure The researchers originally intended to report on community integration/social activities, but fewer than half of all respondents addressed this component (one-year, 44.3%; five-year, 44.0%). The response rate was similar between students both in general education and in special education. Thus, the category was withdrawn from the analysis. One-Year Goal Summary After examining evidence of planning, researchers found statistically significant differences in only the presence of a disability (2 (2, N = 183) = 27.86, p = 0.00). Description information for one-year goals appears in Table 3.

149 Table 3 Descriptive Information for Year-1 Goals by School Setting, Gender, Ethnicity, and SES Education Level Gender Ethnicity General Special Male Female Black White Hispanic Ed Ed.

SES No Free Lunch

Free Lunch

Education 2-Yr School 4-Yr School Professional/ Technical School Generic “School” Unrealistic Does not address Employment Specific Job Generic Job Unrealistic

53 41.7% 23 18.1% 8 6.3%

14 25.0% 1 1.8% 81 4.3%

25 29.1% 10 11.6% 9 10.5%

42 43.3% 14 14.4% 7 7.2%

27 43.5% 9 14.5% 6 9.7%

21 35.6% 8 13.6% 4 6.8%

19 30.6% 7 11.3% 6 9.7%

30 35.3% 11 12.9% 7 8.2%

37 38.1% 13 13.4% 9 9.3%

31 24.4% 1 0.8% 11 8.7%

15 26.8% 4 7.1% 14 15.0%

22 25.6% 3 3.5% 17 19.8%

24 24.7% 2 2.1% 8 8.2%

14 22.6% 3 4.8% 3 4.8%

15 25.4% 2 3.4% 9 15.3%

17 27.4% 0 0% 13 21.0%

25 29.4% 2 2.4% 10 11.8%

20 20.6% 3 3.1% 15 15.5%

36 28.3% 46 36.2% 2 1.6% 43 33.9%

15 26.8% 21 37.5% 1 1.8% 19 33.9%

26 30.2% 27 31.4% 3 3.5% 30 34.9%

25 25.8% 40 41.2% 0 0% 32 33.0%

15 24.2% 26 41.9% 1 1.6% 20 32.3%

23 39.0% 18 30.5% 0 0% 18 30.5%

13 21.0% 23 37.1% 2 3.2% 24 38.7%

29 34.1% 29 34.1% 2 2.4% 25 29.4%

22 22.7% 37 38.1% 1 1.0% 37 38.1%

12 21.4%

15 17.4%

21 21.9%

8 13.1%

8 13.6%

20 32.3%

14 16.5%

21 21.9%

Does not address Independent Living With Family 24 Support 19.0%

150 Without Family Support Unrealistic Does not address

33 26.2%

15 26.8%

18 20.9%

30 31.3%

17 27.9%

18 30.5%

13 21.0%

26 30.6%

22 22.9%

1 0.8% 68 54.0%

0 0% 29 51.8%

1 1.2% 52 60.5%

0 0% 45 46.9%

0 0% 36 59.0%

0 0% 33 55.9%

1 1.6% 28 45.2%

0 0% 45 52.9%

1 1.0% 52 54.2%

151 Five-Year Differences Postsecondary Education For postsecondary education five years following graduation, statistically significant results were found in terms of the presence of a disability (2 (4, N = 175) = 14.09, p = 0.007) (Table 4). Students who had disabilities planned less cumulative activity in terms of postsecondary education when compared to peers without disabilities. Students with disabilities planned lower rates of graduating from postsecondary education (9.8% vs. 24.2%) and lower levels of attending graduate school (3.9% vs. 16.1%) compared to their counterparts without disabilities. For gender, resulting values were approaching significance (2 (4, N = 175) = 8.03, p = 0.09). Females planned higher cumulative rates of postsecondary-education engagement. Females planned higher levels of graduating from postsecondary education (24.5% vs. 14.8%) and higher rates of attending graduate school (17.0% vs. 7.4%) compared to males. No differences were found in terms of ethnicity (2 (8, N = 175) = 4.83, p = 0.78) or socioeconomic status (2 (4, N = 174) = 4.77, p = 0.31) within five-year goals. Employment No statistical significance was found in terms of five-year employment goals based upon presence of a disability (2 (3, N = 175) = 2.80, p = 0.42), gender (2 (3, N = 175) = 5.32, p = 0.15), ethnicity (2 (6, N = 175) = 5.99, p = 0.43), or socioeconomic status (2 (3, N = 174) = 6.87, p = 0.08). In terms of the differences in socioeconomic status, the results were approaching significance. The specific differences included more students receiving free-and-reduced lunch reporting a specific place of employment and more students from no free-and-reduced lunch not addressing employment in the five-year goal. Independent Living Similar to the findings of one-year independent living goals, no statistical significance was found among groups, and no differences were approaching significance. The results from the analyses were calculated to be presence of a disability (2 (3, N = 175) = 1.24, p = 0.74), gender (2 (3, N = 175) = 1.67, p = 0.64), ethnicity (2 (6, N = 175) = 7.92, p = 0.24), and socioeconomic status (2 (3, N = 174) = 1.81, p = 0.61). Five-Year Goal Summary In examining the evidence of planning, statistical significance was found in terms of presence of a disability (2 (2, N = 174) = 25.24, p = 0.00) and gender (2 (2, N = 174) = 14.65, p = 0.001). For students with disabilities, there was an overall lower level of long-term planning. No differences were found for either ethnicity (2 (4, N = 174) = 3.09, p = 0.54) or socioeconomic status (2 (2, N = 173) = 1.14, p = 0.57). Descriptive information for five-year goals appears in Table 4. Goal Continuity Goal continuity was defined as the linear relation between one-year and five-year goals. In simple terms, the question, “Does the one-year goal serve as a progression into the five-year goal?” is answered. Chi-square analyses show statistically significant results for students both with and without disabilities (2 (1, N = 174) = 30.190, p = 0.00). Students with disabilities did not demonstrate the same level of goal continuity when compared to their general education

152 peers. Statically significant results were not found in terms of gender (2 (1, N = 174) = .839, p = 0.36), ethnicity (2 (1, N = 174) = 2.880, p = 0.237), or socioeconomic status (2 (1, N = 173) = .371, p = 0.542).

153 Table 4 Descriptive Information for Year-5 Goals by School Setting, Gender, Ethnicity, and SES Education Level Gender Ethnicity General Special Male Female Black White Hispanic Ed Ed. Education Still in College Graduated Graduate School Unrealistic Does not address Employment Specific Job Generic Job Unrealistic

SES No Free Lunch

Free Lunch

20 16.1% 30 24.2% 20 16.1% 1 0.8% 53 42.7%

7 13.7% 5 9.8% 2 3.9% 1 2.0% 36 70.6%

13 16.0% 12 14.8% 6 7.4% 1 1.2% 49 60.5%

14 14.9% 23 24.5% 16 17.0% 1 1.1% 40 42.6%

7 11.9% 12 20.3% 8 13.6% 1 1.7% 31 52.5%

8 14.3% 9 16.1% 9 16.1% 0 0% 30 53.6%

12 20.0% 14 23.3% 5 8.3% 1 1.7% 28 46.7%

12 14.5% 14 16.9% 14 16.9% 0 0% 43 52.8%

15 16.5% 20 22.0% 8 8.8% 2 2.2% 46 50.5%

50 40.3% 39 31.5% 8 6.5% 27 21.8%

17 33.3% 13 25.5% 5 9.8% 16 31.4%

29 35.8% 23 28.4% 10 12.3% 19 23.5%

38 40.4% 29 30.9% 3 3.2% 24 25.5%

25 42.4% 19 32.2% 5 8.5% 10 16.9%

18 32.1% 18 32.1% 2 3.6% 18 32.1%

24 40.0% 15 25.0% 6 10.0% 15 25.0%

25 30.1% 25 30,1% 6 7.2% 27 32.5%

42 46.2% 26 28.6% 7 7/7% 16 17.6%

0 0% 15 29.4%

0 0% 26 32.1%

1 1.1% 34 36.2%

1 1.7% 25 42.4%

0 0% 15 26.8%

0 0% 20 33.3%

1 1.2% 26 31.3%

0 0% 33 36/3%

Does not address Independent Living With Family 1 Support 0.8% Without 45 Family 36.3% Support Unrealistic 2

154

Does not address

1.6% 76 61.3%

1.0% 35 68.6%

2.5% 53 65.4%

1.1% 58 61.7%

3.4% 31 52.5%

Table 5 Descriptive Information for Evidence and Continuity of Planning Education Level Gender General Special Male Female Black Ed Ed. Year-1, Evidence of Planning 9 9 No planning 7.1% 16.1% Minimal 55 42 planning 43.3% 75.0% Clear 63 5 8.9% Planning 49.6% Year-5, Evidence of Planning 12 16 No planning 9.8% 31.4% Minimal 71 34 planning 57.5% 66.7% Clear 40 1 1.0% Planning 32.5% Continuity of Planning 36 38 No 29.3% 74.5% 87 13 Yes 70.7% 15.5%

0% 41 73.2%

1.7% 39 65.0%

Ethnicity White Hispanic

1.2% 55 66.3%

2.2% 56 61.5%

SES No Free Lunch

Free Lunch

11 12.8% 46 53.5% 29 33.7%

7 7.2% 51 52.6% 39 40.2%

4 6.5% 38 61.3% 20 32.3%

6 10.2% 28 47.5% 25 42.4%

8 12.9% 31 50.0% 23 37.1%

7 8.2% 40 47.1% 38 44.7%

11 11.3% 56 57.7% 30 30.9%

22 27.2% 45 55.6% 14 17.3%

6 6.4% 60 64.5% 27 29.0%

8 13.6% 38 64.4% 13 22.0%

11 20.0% 28 50.9% 16 29.1%

9 15.0% 39 65.0% 12 20.0%

14 17.1% 46 56.1% 22 26.8%

14 15.4% 58 63.7% 19 20.9%

37 46.3% 43 53.8%

37 39.4% 57 60.6%

30 50.8% 29 49.2%

23 41.1% 33 58.9%

21 35.6% 38 64.4%

37 44.6% 46 55.4%

36 40.0% 54 60.0%

155 Discussion This study examined postsecondary goals indicated by high school seniors to assess aspects of complete, realistic transition planning and continuity. High school seniors, educated in both general and special education, expressed goals for one year and five years following high school in the areas of postsecondary education, employment, and independent living. These goals were evaluated for evidence of planning and goal continuity between students’ one-year and five-year goals. The content analysis of the goals resulted in significant differences between general education students and students with disabilities and provided additional insight, particularly regarding logical relations between short-term and long-term goal planning, not previously found in the literature. Postsecondary Education The most dramatic differences occurred between general education students and special education students. For one-year goals, significant differences were found for postsecondary education. Almost all students in general education expressed a goal for postsecondary education (90.6%), whereas only two-thirds (67.9%) of students in special education expressed this goal. Notable differences were found in planned enrollment in both 2-year and 4-year schools between the two groups, with only one student receiving special education services reporting planned enrollment at a 4-year school. Differences were also found in other categories. More students who received special education services reported unrealistic goals (7.1%) than did students without disabilities (0.8%). Students with disabilities also demonstrated less continuity across goals than did students without disabilities, underscoring the need for instruction and development related to logical progression across long- and short-term goals. Perhaps educators, faced with pressure for students with disabilities to pass standardized tests and graduate, are focused on reaching graduation, not considering life beyond high school. Upon detailed analysis, only 1.8% (n = 1) of students in special education expressed a goal of attending a 4-year school, compared to 18.1% (n = 23) of students in general education. Finally, 7.1% (n = 4) of students in special education expressed an unrealistic goal based on the timeline needed for completion for education compared to 0.8% (n = 1) for students in general education. Students with disabilities in this study were lacking in goal-setting and planning skills; this also could result from the emphasis on passing standardized tests for graduation, as goal-setting skills are not assessed on standardized tests. Although significant differences were not identified for gender, ethnicity, or socioeconomic status, some results occurred that warrant further research. In terms of gender, more females than males expressed a goal of attending postsecondary education, specifically at the community/junior college level. The fact that more females indicated a goal of attending postsecondary school corresponds with current trend rates that indicate more females than males are, in fact, attending postsecondary education (Archibald et al., 2015). Further, there is evidence of growing gender gaps within ethnicity groups as females participate and persist in education at higher rates than their male counterparts (Aud et al., 2010; Aud et al., 2011). In terms of ethnicity, non-Hispanic identifying students identified postsecondary education as a goal more frequently than did Hispanic students. This difference may reflect a larger cultural trend of Hispanic students’ focusing on employment to help support their families (Tienda &

156 Mitchell, 2006). Trends in postsecondary attendance rates indicate that White students and students from higher socioeconomic status attend postsecondary education at higher rates than minority students and/or students in poverty (Ross et.al., 2012). However, based upon studentindicated goals in this study, Black and White students indicate postsecondary goals at similar rates. Regardless of the level of socioeconomic status, the goal of attending postsecondary education was comparable across Black and White students. Similar results were found for five-year goals. Slightly over half (56.5%) the general education students surveyed expressed a five-year, postsecondary-education goal compared with only 27.5% of students in special education. A large discrepancy existed for students indicating an advanced degree or graduate school, where 16.1% (n = 20) of general education students expressed this goal compared to 3.9% (n = 2) of special education students. This difference could indicate a lack of discussion about graduate school and advanced degrees with students receiving special education services, furthering concern over decreased expectations for individuals with disabilities compared to expectations for students without disabilities (Wehman et al., 2015). Again, females (17.0%, n = 16) expressed a goal of attending postsecondary education, especially for advanced degrees or attending graduate school, more often than males (7.4%, n = 6). Employment No differences were found for employment in terms of disability, gender, ethnicity, or socioeconomic status for either one-year or five-year goals. When examined closely, small trends were found between specific job and general jobs, with general education students indicating a higher rate of a specific job at five-years (40.3%) than students in special education (33.3%). These differences are not seen with the one-year goal. One hypothesis for this is that students in general education see postsecondary education as a means to obtain a certain job or career. Students in special education are not internalizing the difference between a job and a longer-term career. This is likely related to the cognitive difficulties students with disabilities experience in realizing a long-term goal and being able to identify the steps needed to achieve that goal. Independent Living Similar to employment, no differences were found to be significant in terms of disability, gender, ethnicity, or socioeconomic status for either one-year or five-year goals. Again, this indicates that student desires, regardless of differences, are comparable. However, during one-year goals, Hispanic students reported a higher rate of living with family than other groups (13.1% Black, 32.3% Hispanic, and 13.6% White). This difference is likely attributed to strong, extendedfamily ties which contribute to Hispanic children living at home longer or until married (Tienda & Mitchell, 2006). These cultural aspects must be considered during transition planning, regardless of general or special education status. Finally, it is important to mention that most of the students failed to indicate an independent living goal (53.3%), but the differences in the goals indicated were significant enough to be reported. Community Integration/Recreation and Leisure As mentioned in the results section, we originally intended to report on community integration/recreation and leisure, but fewer than half of all respondents addressed this component, with similar rates between both general and special education students. It is hypothesized the similar rates of addressing this component reflect that both students in general

157 education and special education may not have understood the question. Community integration/recreation and leisure are important aspects of adulthood and should be addressed during transition planning. Further examination is needed to understand why students failed to respond to this component and to help develop ways for educators to support students’ understanding of appropriate community integration/recreation and leisure goals. Goal Summary The presence of a disability resulted in a statically significant difference in both one-year and five-year goals. Far more students without disabilities were able to demonstrate strong evidence of a plan in one-year goals (49.6%) than students with disabilities (8.9%). The majority of students in special education did articulate some level of planning (75%), possibly related to the requirements for transition planning mandated through IDEA. Similar results were found in fiveyear planning with 32.5% of students in general education showing a clear plan, compared to 2% of students in special education. These results demonstrate a lack of long-range planning for students with disabilities, which supports similar findings in research pertaining to goal setting and students with disabilities (Wehmeyer, 1994; Carter et al., 2006; Carter et al., 2009). Results found for five-year goals in terms of gender were varied. Gender resulted in a statically significant difference in five-year goals with a higher percentage of males (12.3%) reporting unrealistic employment compared to females (3.2%). Females were able to articulate clear planning (29%) at a rate higher than males (17.3%). Disability status was not a factor on this result; equal participants of males and females had disabilities in this sample. These findings warrant further investigation to examine differences in employment goals between male and female students and five-year goal differences. Goal Continuity Seventy-five percent of participants who received special education services were unable to articulate continuity between one-year and five-year goals. Only 25% of students with disabilities planned a logical connection between goals, while 71% of general education participants articulated continuity between goals. This difference is concerning and indicates that students with disabilities lack an understanding about cause-effect relations and ability to logically sequence steps necessary for goal attainment. In theory, students with disabilities should be given additional in-school time to address transition planning and they should have school staff helping to plan postsecondary outcomes. Results from this study may indicate students with disabilities are not participating in IEP development and transition activities such as goal setting, despite IDEA mandates requiring transition services and student participation in transition planning. However, the larger implication is that students do not understand the notion of goal setting, including short-term and long-term goals, but more importantly the notion of goals and subgoals. All students, as this study clearly demonstrates, have difficulty setting goals and this difficulty is not limited to only students with a disability. In order to help students understand the process to goal attainment, goal setting must be explicitly taught, including how to set long-term goals, break them down into sub-goals, and how step-by-step sub-goals lead to the ultimate goal attainment. Limitations

158 The sample size of this study is small and includes students from only one school district, (multiple campuses) therefore, generalizations should not be made to other populations. However, the sample does reflect a balance in numbers across individuals from the three largest demographic backgrounds found in public schools. This balance in the sample serves as a strength in terms of making comparisons among the minority students in schools and is part of the reason this school district was chosen for this research. The second major limitation was the survey instrument and data collection. Students were given a prompt to indicate goals for both one year and five years following high school. This question was vague and provided few guidelines. Having a more direct prompt may have ensured more complete goals addressing more aspects of the transition process, with more elaboration on continuity of goals. The ambiguity of questions and lack of guidelines may have contributed to another limitation—lack of responses related to community integration/recreation and leisure. It is also unknown whether students participating in this research received any formal education on goal setting. Finally, we recognize the analyses are preliminary in nature because no direct comparisons have been published regarding differences between students in general and special education in terms of ability for futures planning. We attempted to capture “the big picture” and develop an inclusive coding system to help determine a starting point for future studies examining short- and longterm goal setting and attainment between students with disabilities and students without disabilities. Implications for Practice Overall, the results of this study indicate many students with and without disabilities experience difficulties articulating goals and understanding the need for continuity between their short- and long-term goals. Significant differences were found in goals articulated by students with disabilities and goals articulated by students without disabilities. Students with disabilities demonstrated less continuity between goals compared to the goals of their general education peers. Students with disabilities expressed lower expectations in terms of attending postsecondary education and specifically in attending four-year institutions. The lack of continuity between short- and long-term goals indicates a need for increased instruction in long-range planning for students with disabilities. Students need multiple opportunities to realize their goals, identify steps to achieve short-term goals, and then describe connections between their short-term and long-term goals. This process should include feedback from educators and other members of the IEP team to help students learn to adjust goals as needed and monitor progress toward goals. Classroom teachers have a unique ability to help students increase their understanding of goal setting and goal acquisition, thus increasing students’ overall self-determination throughout all levels of education (Williams-Diehm et al., 2010). Teachers need to provide direct instruction on goal setting and the process by which goals or objectives progress into a larger goal, and the timeline to address futures planning and self-determination must start early (Papay et al., 2015). Learning this process during elementary, middle, and high school, and having opportunities to practice setting goals will allow for greater generalization of goal setting into postsecondary life settings.

159 Students would also benefit from receiving feedback on their goal setting and plans to achieve those goals so that they can learn to develop goals that are both feasible and show continuity. Goal setting is something common to adult life. One sets goals to complete certain tasks at work, creates to-do lists of daily tasks, and monitors and adjusts plans as necessary to attain the goals one has set. Acquisition of goal-setting skills develops the skills necessary to identify personal strengths and weaknesses, create an action plan, and adjust actions to ensure success (Martin et al., 2014). Goal setting is also considered a vital component of self-determination and may increase student engagement in IEP planning (Martin & Williams-Diehm, 2013). Goal setting requires planning and high-level thinking and students need support to learn skills necessary to successfully set and attain short- and long-term goals. Although short-term goals are good for the process of teaching goals (Shogren et al., 2012), students in this study had difficulty articulating long-term goals and the steps needed to accomplish these goals. More than half the sample (74.5% of special education and 29.3% of general education) did not demonstrate any goal continuity. This lack of continuity demonstrates a lack of ability for long-range planning for all students, but especially for students with disabilities. By explicitly teaching goal setting, teachers can help students gain the skills necessary for better long-term planning. Students also need to be taught when to adjust steps and when to change goals. These involve a series of higher-order thinking skills for which students with disabilities need direct instruction. Suggestions for Future Research More research is needed on goal setting, goal attainment, and goal continuity. These are critical components of self-determination and should be addressed for all students, not just students with disabilities. Because goal setting and attainment are critical to student success in postsecondary education as well as in other postsecondary environments, research related to effective methods of teaching goal setting to students with disabilities is also warranted. This research should focus on long-range goal setting and the steps necessary to acquire these goals, including problem solving and higher-order thinking strategies. Starting with short-term goals (Shogren et al., 2012), students can learn a process that can lead to development of long-range goals. The current study is only a content analysis of goals independently set by graduating high school students. Research is still needed in the area of goal completion and acquisition. It was determined from this study that some students are successful in stating future goals, but no follow-up data were collected to determine if those goals were accomplished or whether students knew how to accomplish their articulated goals. Follow-up studies related to goal setting and attainment would greatly enhance understanding of this area. Finally, research related to goal setting and attainment should examine teacher expectations regarding postsecondary expectations for students with disabilities and students without disabilities. References Algozzine, B., Browder, D., Karvonen, M., Test, D. W., & Wood, W. M. (2001). Effects of interventions to promote self-determination for individuals with disabilities. Review of Educational Research, 71(2), 219–277. https://doi.org/10.3102/00346543071002219

160 Archibald, R. B., Feldman, D. H., & McHenry, P. (2015). A quality-preserving increase in fouryear college attendance. Journal of Human Capital, 9, 265-297. https://doi.org/10.1086/682355 Aud, S., Fox, M. A., & KewalRamani, A. (2010). Status and trends in the education of racial and ethnic groups. National Center for Educational Statistics (2010-015). http://nces.ed.gov/pubs2010/2010015.pdf. Aud, S., Hussar, W., Kena, G., Bianco, K., Frohlich, L., Kemp, J., Tahan, K., Mallory, K., Nachazel, T., & Hannes, G. (2011). The condition of education 2011. National Center for Educational Statistics (2011-033). http://nces.ed.gov/pubs2011/2011033.pdf. Austin, J. T., & Vancouver, J. B. (1996). Goal constructs in psychology: Structure, process, and content. Psychological Bulletin, 120(3), 338-375. https://doi.org/10.1037/00332909.120.3.338 Carter, E. W., Lane, K. L., Pierson, M. R., & Glaeser, B. (2006). Self-determination skills and opportunities of transition-age youth with emotional disturbance and learning disabilities. Exceptional Children, 72(3), 333-346. https://doi.org/10.1177/001440290607200305 Carter, E. W., Owens, L., Trainor, A. A., Sun, Y., & Sweeden, B. (2009). Self-determination skills and opportunities of adolescents with severe intellectual and developmental disabilities. American Journal on Intellectual and Developmental Disabilities, 114(3), 179-192. https://doi.org/10.1352/1944-7558-114.3.179 Dunlop, W. L., Bannon, B. L., & McAdams, D. P. (2017). Studying the motivated agent through time: Personal goal development during the adult life span. Journal of Personality, 85(2), 207-219. https://doi.org/10.1111/jopy.12234 Ellis, E. S., & Larkin, M. J. (1998). Strategic instruction for adolescents with learning disabilities. In B. Y. L. Wong (Ed.), Learning about learning disabilities (2nd ed., pp. 585-656). Academic Press. Franke, T. M., Ho, T., & Christie, C. A. (2012). The chi-square test: Often used and more often misinterpreted. American Journal of Evaluation, 33(3), 448-458. https://doi.org/10.1177/1098214011426594 Hinkle, D. E., Wiersma, W., & Jurs. S. G. (2003). Applied statistics for the behavioral sciences (5th ed.). Houghton Mifflin Company. Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004). Konrad, M., Fowler, C. H., Walker, A. R., Test, D. W., & Wood, W. M. (2007). Effects of selfdetermination interventions on the academic skills of students with learning disabilities. Learning Disability Quarterly, 30(2), 89-113. https://doi.org/10.2307/30035545 Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://www.jstor.org/stable/2529310 Lee, S. H., Wehmeyer, M. L, Soukup, J. H., & Palmer, S. B. (2010). Impact of curriculum modifications on access to the general education curriculum for students with disabilities. Exceptional Children, 76(2), 213-233. https://doi.org/10.1177/001440291007600205 Lenz, B. K., Ehren, B. J., & Smiley, L. R. (1991). A goal attainment approach to improve completion of project-type assignments by adolescents with learning disabilities. Learning Disabilities Research & Practice, 6(3), 166-176. Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation. American Psychologist, 57(9), 705-717. https://doi.org/10.1037/0003066X.57.9.705

161 Martin, J. D., Martin, J. E., & Osmani, K. (2014). Teaching students to attain annual transition goals using the Take Action goal attainment lessons. Career Development for Exceptional Individuals, 37(2), 72-83. https://doi.org/10.1177/2165143413476544 Martin, J. E., Marshall, L. H., & Maxson, L. L. (1993). Transition policy: Infusing selfdetermination and self-advocacy into transition programs. Career Development for Exceptional Individuals, 16(1), 54-61. https://doi.org/10.1177/088572889301600105 Martin, J. E., & Williams-Diehm, K. (2013). Student engagement and leadership of the transition planning process. Career Development and Transition for Exceptional Individuals, 36(1), 43-50. https://doi.org/10.1177/2165143413476545 Papay, C., Unger, D., Williams-Diehm, K., & Mitchell, V. (2015). Begin with the end in mind: Infusing transition planning and instruction into elementary classrooms. Teaching Exceptional Children, 47(6), 310-318. https://doi.org/10.1177/0040059915587901 Ross, T., Kena, G., Rathbun, A., KewalRamani, A., Zhang, J., Kristapovisch, P. & Manning, E. (2012). Higher education: Gaps in access and persistence study. National Center for Education Statistics (2012-046). http://nces.ed.gov/pubs2012/2012046.pdf. Shogren, K., Palmer, S., Wehmeyer, M. L., Williams-Diehm, K., & Little, T. (2012). Effect of intervention with the Self-Determined Learning Model of Instruction on access and goal attainment. Remedial and Special Education, 33(5), 320-330. https://doi.org/10.1177/0741932511410072 Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., Kohler, P., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128. https://doi.org/10.1177/0885728809336859 Tienda, M., & Mitchell, F. (2006). Hispanics and the future of America. National Academies Press (US). Trainor, A. (2005). Self-determination perceptions and behaviors of diverse students with LD during the transition planning process. Journal of Learning Disabilities, 38(3), 233-249. https://doi.org/10.1177/00222194050380030501 Trainor, A. (2007). Perceptions of adolescent girls with LD regarding self-determination and postsecondary transition planning. Learning Disability Quarterly, 30(1), 31-45. https://doi.org/10.2307/30035514 Wagner, M., Newman, L., Cameto, R., Levine, P., & Marde, C. (2007, August). Perceptions and expectations of youth with disabilities. A special topic report of findings from the National Longitudinal Transition Study-2 (NLTS2). SRI International. Ward, M. J., & Kohler, P. D. (1996). Promoting self-determination for individuals with disabilities: Content and process. In L. E. Powers, G. H. S. Singer, & J. Sowers (Eds.), On the road to autonomy: Promoting self-competence in children and youth with disabilities (pp. 275–290). Paul H. Brookes. Wehman, P., Sima, A. P., Ketchum, J., West, M. D., Chan, F., & Luecking, R. (2015). Predictors of successful transition from school to employment for youth with disabilities. Journal of Occupational Rehabilitation, 25, 323-334. 10.1007/s10926-014-9541-6 Wehmeyer, M. (1994). Perceptions of self-determination and psychological empowerment of adolescents with mental retardation. Education and Training in Mental Retardation and Developmental Disabilities, 29(1), 9-21. Wehmeyer, M. L., & Shogren, K. (2008). Self-determination and learners with autism spectrum disorders. In R. Simpson & B. Myles (Eds.), Educating children and youth with autism: Strategies for effective practice (2nd ed., pp. 433-476). ProEd Publishers, Inc.

162 Wehmeyer, M. L., & Shogren, K. A. (2013). Self-determination: Getting students involved in leadership. In P. Wehman (Ed.), Life beyond the classroom: Transition strategies for young people with disabilities (pp. 41–68). Paul H. Brookes. Williams-Diehm, K., Palmer, S., Lee, Y., & Schroer, H. (2010). Goal content analysis for middle and high school students with disabilities. Career Development for Exceptional Individuals, 33(3), 132-142. https://doi.org/10.1177/0885728810380230 Wood, W. M., Karvonen, M., Test, D. W., Browder, D., & Algozzine, B. (2004). Promoting student self-determination skills in IEP planning. Teaching Exceptional Children, 36(3), 8-16. Zimmerman, B.J., Bandura, A., & Martines-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663-676. https://doi.org/10.3102/00028312029003663 About the Authors Kendra L. Williams-Diehm PhD, BCBA is a professor and director of Zarrow Center for Learning Enrichment in the Jeannine Rainbolt College of Education at the University of Oklahoma. She holds the Zarrow Endowed Family Chair and the Brian E. and Sandra L. O’Brien Presidential Professorship. Her primary research interests include comprehensive transition services with a focus on appropriate assessment and self-determination for students with intellectual and developmental disabilities. Penny Cantley, PhD is an assistant professor of special education at Oklahoma State University. Her primary research interests include transition from school-to-adult life and postsecondary education, employment, and recreation/leisure opportunities, choices, and outcomes of people with disabilities. Her work emphasizes the importance of self-awareness, self-advocacy and wellbeing. Kylie R. Lyons, Ph.D., is a school psychologist and director of Special Services at Bishop McGuinness Catholic High School in Oklahoma City, Oklahoma. As a graduate student at the University of Oklahoma, her research interests included increasing self-determination in students with mild-to-moderate disabilities. Malarie Deardorff, PhD, is the assistant director of the Zarrow Center for Learning Enrichment and assistant research professor at the University of Oklahoma. Her primary research interests are in transition assessments, incorporating transition practices into elementary and middle school classrooms, and compliance and quality transition planning. Patricia Lynch, PhD, taught in the Special Education Program at Texas A&M University for 30 years and currently holds the title of Professor Emerita. Dr. Lynch’s interests are in the transition from school to work for adolescents with disabilities and youth at-risk of failure, as well as teacher training.

163 Author Guidelines AUTHOR GUIDELINES SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE welcome manuscript submissions at any time. Authors are completely responsible for the factual accuracy of their contributions. Authors are responsible for obtaining permission to quote lengthy excerpts from previously-published articles. Authors will be notified of the receipt of their manuscripts within seven (7) business days of their receipt by the Chief Editor and can expect to receive the recommendation of the review process within 90 days. All submissions must have a cover letter indicating that the manuscript has not been published, or is not being considered for publication anywhere else, in whole or in substantial part. On the cover letter it will be noted to the authors to be sure to include their name, address, email address, and phone number. Typescript should conform to the following:             

Method of Manuscript Submission: Send Manuscripts should be submitted electronically with the words " Submission to SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE" in the subject line Language: English Document: Microsoft Word Font: Times New Roman or Arial Size of Font: 12 Point Page Limit: None Margins: 1” on all sides Title of paper: Top of page Capitals, bold, and centered Author(s) Name: Centered under title of paper Format: Manuscripts should follow the guidelines of the most recent edition of the Publication Manual of the American Psychological Association Figures and Tables: All should be integrated in the typescript Abstract: An abstract of no more than 150 words should accompany each submission References: Insert all references cited in the paper submitted on a Reference Page

164 Publishing Process PUBLISHING PROCESS The following are the steps through which a submitted article moves from initial submission to actual publication in SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE. 1. The article is submitted in Word format by the author(s) to the Editor, Dr. George Giuliani, at George.A.Giuliani@hofstra.edu 2. The Editor then sends an email to the author stating that the article has been received and a final answer will be given within 90 days of receipt. 3. The article is then placed in an online folder that holds all articles up for review. 4. The article is then removed of all author names, affiliations, etc. so that when it is sent out for review, it is a blind review and no peer reviewers have any indication who wrote the article or from what university it was submitted. 5. An email is then sent by the Editor to all members of the Editorial Board with a request to review an article. Only the title of the article is given. 6. Peer reviewers then email the Editor as to their interest in reviewing the specific article. A minimum of three (3) blind reviewers is selected. An email to each blind reviewer is then sent reviewing the peer review process. 7. The peer reviewers have 60 days from receipt to return a recommendation to the Editor. 8. Once all of the reviews have been obtained, the Editor determines whether to:    

Accept, with or without editorial revisions Invite the authors to revise their manuscript to address specific concerns before a final decision is reached Reject, but indicate to the authors that further work might justify a resubmission Reject outright, typically on grounds of specialist interest, lack of novelty, insufficient conceptual advance or major technical and/or interpretational problems

9. An email with the decision (and rationale for it) is then sent to the author(s). 10. Before publication, the article is then passed through three functions: copy editing (grammar, references), proofing (typographical errors, spelling errors), and layout (creating a Microsoft Word and PDF version of the article).

165 Copyright and Reprint Rights SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE is published by Hofstra University. SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE retains copyright of all original materials, however, the author(s) retains the right to use, after publication in the journal, all or part of the contribution in a modified form as part of any subsequent publication. If the author(s) use the materials in a subsequent publication, whether in whole or part, SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE must be acknowledged as the original publisher of the article. All other requests for use or re-publication in whole or part, should be addressed to the Editor of SPECIAL EDUCATION RESEARCH, POLICY & PRACTICE.