Standard 4 Compendium 2 - R4.2 Satisfaction of Employers

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Alignment to National Standard: CAEP Standard 4 - The EPP demonstrates the effectiveness of completers’ instruction on P12 student learning and development and completer and employer satisfaction with the relevance and effectiveness of preparation.

Component R4.2 Satisfaction of Employers: The Provider demonstrates employers are satisfied with the completers’ preparation for their assigned responsibilities in working with diverse P-12 students and their families.

How Alignment is Assured: The Assessment Coordinator in consultation with Program Leads and Department Heads, aligns the evaluation measures and assessment tasks with CAEP, InTASC, and appropriate Technology Standards. The Assessment Coordinator maintains alignments and adherence to multiple Louisiana state laws and policy regulations. All Standards have been maintained utilizing Watermark – Taskstream. This standards database is maintained by the Assessment Coordinator so that alignments can accommodate updates to standards, program competencies, courses, or assessments.

Evidence Overview

Use of Assessment as Part of the Quality Assurance System: The EPP measures the employers’ satisfaction with the preparation of completers using surveys and focus groups. The EPP has traditionally used survey data to document effectiveness of completers. This survey was internally aligned to InTASC, State of Louisiana requirements, and professional associations. In the past year, the EPP has conducted a pilot test by asking supervisors to evaluate completers using the Danielson Framework for Teaching evaluation that is used as a part of Standards 1, 2 and 5. Additionally, the EPP conducted a focus group interview of several employers representing diverse educational settings, in an effort to ascertain how well the completers were able to meet the needs of P-12 diverse students as well as create learning opportunities that were inclusive, supportive, and equitable through their knowledge, skills and professional dispositions.

Use of Assessment as Part of the Quality Assurance System: The surveys and interviews are done to ascertain how well the program prepares graduate candidates to meet the needs of their students with respect to content knowledge, applicational skills, and critical dispositions required to ensure that their classrooms are inclusive and meet the needs of diverse students. The surveys are conducted with the supervisors/employers who have been able to observe the completers in their classrooms. Those conducting the surveys are teaching in their respective fields of certification. The surveys assess the ability of the completers to use the knowledge and skills in their classrooms and/or with colleagues. This feedback allows the EPP to document areas of strength and need as well as pinpoint new areas of focus

Details of Assessment Administration:

Surveys: Surveys are sent to employers of the completers after they have secured or maintained teaching positions in their respective districts. Surveys are sent by mail, and/or electronically. The surveys measure completer knowledge and skills to teach content with the dispositions appropriate of an effective educator. The ability to meet the needs of diverse learners and create inclusive classrooms is also surveyed. To ensure a rich response rate, follow-up phone and email reminders are also employed. The surveys used a standard form which links indicators to CAEP and InTASC for more detailed analysis. In 2021, the EPP decided to do a pilot another method of capturing employer satisfaction by asking the employer to rate the completers in the classroom using the Danielson framework (proprietary). We realized that this would allow us to link more completely to Standards 1, 2 and 5 for analyses and continuous improvement. In the initial year, nine principals were emailed a letter and all nine responded

Focus Groups: To triangulate data, principals and school directors were invited to participate in a zoom focus group interview. Two administrators who have the responsibility for hiring and supervising completers were able to join the session. They spoke of candidates hired over the past two to three years.

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Evidence and Analysis

The Employer Survey was used as the first measurement tool, including alignment to appropriate standards.

Presentation

The tables below display the results for questions about program preparation as observed by all employers of GSU graduates. Overall, principals reported they were dissatisfied, satisfied, or very satisfied with teacher candidates (responded either “2” or “3” or “4” on a four-point scale).

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Grambling State University Standard Four Compendium 2

R4.2 Satisfaction of Employers

10.

The Danielson Rubric (proprietary) was used to understand the satisfaction of employers with the EPP’s graduates by conducted a pilot in which employers were asked to evaluate the completers using the Rubric.

Presentation of Revised Data:

The tables below show 2020-2021 data from nine employers who evaluated completers (six undergraduates and three alternative/MAT). The response rate for the nine employers was 100%. The Danielson rubric allowed the principals to rate the competency of the completers (undergraduate and MAT) regarding attributes based on the 10 InTASC Standards on a scale of 1 to 4, with a response of 1 indicating Ineffective, and a response of 4 indicating Highly Effective. The results are as follows: 2020-2021:

Overall Total of Nine Completers/Graduates Including Undergraduate and MAT

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Secondary

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

*One completer could not be observed due to teaching out of area of certification

Note the alignment was done from The Danielson Group. (2014). There was correlation between the Danielson Framework for Teaching and the Interstate Teacher Assessment and Support (InTASC) Standards.

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Focus Group Interview with a Principal of a Public Elementary School and Director of a K-12 Charter School both within a 20-mile radius of Grambling State University:

The interview was conducted and recorded via Zoom. The data was analyzed by an external evaluator. There were eight overall questions asked:

1. Over the past 3 years, if you have hired a teacher from our initial certification programs who completed a baccalaureate degree or Master of Arts Teaching degree, what were your experiences with the teacher?

a. If you recall, what certification did the teachers hold? What are they teaching now?

2. Did the teacher know the content or master the content quickly?

3. Can you think of some ways the teacher engaged the students?

4. How was the teacher able to teach to all students, including those who might have had IEPs or were classified as eligible for Section 504 accommodations? If so, did you notice any things that stuck in your mind as particularly effective?

5. Can you think of some ways that the teacher worked with other teachers or parents?

6. Can you think of some ways that the teacher used technology?

7. What do you think are the strengths of our Program and what can we do better for the improvement of our candidates?

8. What are some ways that we can improve the quality of the completers coming to you as teachers? What can WE do better?

QUESTION 1: If you have hired teachers who completed Grambling’s initial certification program, what are your experiences with those teachers?

OVERALL THEME: Positive Experiences with new teachers. Strengths in the areas of instruction and planning. Growth possibility in the area of dispositions.

QUESTION 2: For initial certification graduates, over the past three years, did the teacher know the content or master the content quickly?

OVERALL THEME: Teachers’ mastery of the content is very strong.

QUESTION 3: What are a couple of ways your teachers, who graduated over the last three years, were able to engage the students?

OVERALL THEME: The new teachers were effective in engaging students.

QUESTION 4: How were the teachers able to teach ALL students, including those who are diverse along multiple levels IEPs, Section 504, English Language Learners? Did you notice anything that was particularly effective?

OVERALL THEME: The new teachers are effective in dealing with diverse populations.

QUESTION 5: Can you think of ways in which the teachers work with other teachers and with parents?

OVERALL THEME: Collaboration is critical in both schools and the teachers have been successful with colleagues and parents.

QUESTION 6: What are some ways the teachers have used technology and including during the pandemic?

OVERALL THEME: Teachers engage in several ways of using technology. They are not only effective but also talented in this area.

QUESTION 7: What do you think are the strengths of our Program and what can we do better for the improvement of our candidates?

OVERALL THEME: The Grambling graduates are especially strong in readiness, planning for instruction and ability to work with diverse populations.

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

QUESTION 8: What are some ways that we can improve the quality of the completers coming to you as teachers? What can WE do better?

OVERAL THEME: Increase the marketing of Grambling’s strong Advanced Teacher Preparation Program

Analysis and Interpretation

Survey: The Grambling State University survey data of employers as well as the follow-up Danielson Rubric data revealed many positive aspects about the readiness of the EPP's completers to enter the profession. Leading examples of positive employer ratings include effective demonstration of knowledge of content and pedagogy, excellent communication and listening skills, positive student interaction, efficient use of instructional time, purposeful planning and preparation skills using tier 1 curriculum, and incredible classroom management. Areas for growth include working on increasing opportunities for student-to-student discussions, increasing student engagement, aligning standards to instruction and following timelines. In 2018-2019, only one survey was returned and that was for an MAT completer in the area of Secondary EducationMathematics and Special Education. The employer was “Satisfied” or “Very Satisfied” with the completer’s teaching abilities over all areas. Over the period from 2018 – 2022, there were only three times and three areas that an employer selected “Dissatisfied”. That rating appeared in 2019-2020 and was in the areas of “Family and community engagement,” “Development of a safe learning environment,” and “An ability to utilize technology in the instructional process.” These ratings reflected an undergraduate Music Education completer. All other employer ratings using the survey across years and certification preparation types were either “Satisfied” or “Very Satisfied”. We will have a third cycle of survey data by Fall 2023.

Evaluation using the Danielson Rubric: This evaluation tool was piloted using the completers from 2020-2021. The Danielson Rubric instrument is specific enough for the EPP to learn that employers are satisfied that the completers from the 2020-2021 programs are designing and modifying assessments to match learning objectives well and that completers are using formative and summative assessments to inform instructional practice. While there were areas of strength, the first-year teachers have more room for growth in different areas The EPP noted that none of the completers across programs were evaluated “Novice” or Level 1 in any area.

Overall initial program analysis. For the completers in the undergraduate program, their highest overall average scores were in Managing Classroom Procedures and Managing Student Behavior. They also demonstrated strong mean ratings in Domain 3: Instruction and Domain 4: Professional Responsibilities. Their lowest overall average scores were in Setting Instructional Outcomes and Designing Coherent Instruction. The MAT completers demonstrated overall ratings that were rated higher than the undergraduate completers. All three MAT completers were deemed “Highly Effective” in fourteen areas of the Danielson rubric. The weakest areas were Designing Student Assessments and Using Assessment in Instruction, yet the overall group mean was still greater than 3.0 on a 4-point scale.

Analysis by program

The completers in the Undergraduate Elementary Education (grades 1-5) program were evaluated with the highest overall average scores in Managing Student Behavior.

• In Domain 1: Planning and Preparation, they were evaluated most highly in the areas Demonstrating Knowledge of Content and Pedagogy, Demonstrating Knowledge of Students, and Designing Student Assessments. In these areas, 100% of the completers were rated as “Effective Proficient” to “Highly Effective”. The lowest evaluations were in the areas of Setting Instructional Objectives and Designing Coherent Instruction where 33% were evaluated as “Effective Emerging”

• In Domain 2: The Classroom Environment, they were evaluated highest in Creating an Environment of Respect and Rapport and Managing Student Behavior with 100% scoring at least “Effective Proficient”. The lowest evaluation was in the area of Managing Classroom Procedures with 34% scoring “Effective Emerging”

• In Domain 3: Instruction, they were evaluated most highly in the areas Communicating with Students, Using Assessments in Instruction and Demonstrating Flexibility and Responsiveness with all scoring at least at the “Effective

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Grambling State University Standard Four Compendium 2 R4.2 Satisfaction of Employers

Proficient” level. The lowest evaluations were in the areas of Using Questioning and Discussion Techniques and Engaging Students in Learning where 34% were rated as being “Effective Emerging”

• In Domain 4: Professional Responsibilities, they were evaluated highest in Showing Professionalism with all elementary completers rated as “Effective Proficient”. The lowest evaluations were in the areas of Communicating with Families, participating in a professional Community, and Growing and Developing Professionally, where two were evaluated as “Effective Proficient” and one as “Effective Emerging”.

The completer in the Undergraduate Music Education-Instrumental/Vocal K-12) program was evaluated with the highest average scores of “Effective Proficient” in fourteen areas of the Danielson rubric.

• In Domain 1: Planning and Preparation, the completer was evaluated most highly in the areas Demonstrating Knowledge of Students, and of Setting Instructional Objectives. The lowest evaluations were in the areas of Demonstrating Knowledge of Resources, Designing Coherent Instruction, and Designing Student Assessments where the completer was evaluated as “Effective Emerging”

• In Domain 2: The Classroom Environment, the completer was evaluated highest in Managing Classroom Procedures and Managing Student Behavior scoring “Effective Proficient”. The lowest evaluations were in the areas of Creating an Environment of Respect and Rapport and Establishing a Culture for Learning, again scoring “Effective Emerging”

• In Domain 3: Instruction, the completer was evaluated as being “Effective Proficient in all areas except for the area of Demonstrating Flexibility and Responsiveness where the completer was scored at the “Effective Emerging” level.

• In Domain 4: Professional Responsibilities, the completer was evaluated as “Effective Proficient” in all components.

The completers in the Undergraduate Secondary Education and Teaching program were rated as having strong teacher skills as well Both Secondary Education and Teaching completers were rated as “Highly Effective” in four areas of the Danielson rubric including Managing Classroom Procedures, Managing Student Behavior, Organizing Physical Space, and Communicating with Students.

• In Domain 1: Planning and Preparation, both completers were evaluated at either “Highly Effective” or “Effective Proficient” in all components, except one, where one completer was rated as “Effective Emerging” in Setting Instructional Objectives.

• In Domain 2: The Classroom Environment, the lowest rating was “Effective Proficient” in two components

• In Domain 3: Instruction, they were evaluated most highly in all components, receiving ratings of at least “Effective Proficient” except where one completer was rated as “Effective Emerging”

• In Domain 4: Professional Responsibilities, both completers were evaluated as “Effective Proficient” with at least one completer being rated as “Highly Effective” in all components except one where the completer was rated as “Effective Proficient”.

The MAT completers were all from the Elementary and Special Education (Mild/Mod) program. Overall, they demonstrated evaluations that were rated more highly than the undergraduate completers. All three MAT completers were rated as “Highly Effective” in fourteen areas of the Danielson rubric The weakest areas were Designing Student Assessments and Using Assessment in Instruction, yet the overall group average was still greater than 3.0 on a 4-point scale.

Focus Group Interview:

Evidence from Focus Group Interview indicates that employers have been generally pleased with the knowledge, skills, and dispositions of the completers whom they hired or supervised The principals who were interviewed had supervised at different school types (public vs. charter), yet both schools had students with diverse needs. The principals expressed satisfaction that completers from the EPP were not only knowledgeable and skilled but were generally successful at creating school environments that were supportive to diverse students.

Additional Anecdotal Data from 2021-2022 Louisiana Elementary Principal of the year Mr. Marco French, also a GSU graduate, indicated that he places completers from Grambling State University at the top of the list when given a list of applicants for employment

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Grambling State University Standard Four Compendium 2

Satisfaction of Employers

Continuous Improvement

Focus Area(s):

Focus Area 1: Ensure that all completers have standardized familiarity and training in using the Danielson rubric and Special Education addendum according to completer to ensure reliability across observers and districts.

Plan: Provide face-to-face and online training at least once a year and calculate reliability. Strive for reliability coefficient of >.80 among participants.

Focus Area 2: Secure additional data about completer strengths and weaknesses in a rapidly changing teaching environment to triangulate survey findings.

Focus Plan: Continue using focus group interviews to pinpoint specific areas of need, especially in response to ongoing covid environmental challenges.

Focus Area 3: Use data to increasingly pinpoint needs across the areas in which completers teach. Investigate to determine if completers in some areas are more successful or less successful in certain teaching environments.

Plan: Review all assessment outcomes more closely across the overall teaching environments including urban, rural, suburban, regional, economic area, size of district, traditional, online, hybrid, etc., to pinpoint specific areas of need and strengths, both generally and for specific areas.

Focus Area 4: Determine Validity of the Interview questions

Plan: Confirm validity of interview questions by using PK-16 Council/Board or another representative sample of stakeholders. Use the Lawshe Content Validity formula

Focus Area 5: Ensure Ongoing Continuous Quality Improvement practices.

Plan: Continue using focus group interviews to bolster areas in which the response rates are lower, occasions when the completers have moved out of the area, or when long term follow up data are not as complete. Securing additional data could further document that completers have the knowledge, skills and dispositions required to create educational settings that are equitable, inclusive, and welcoming for all diverse P-12 learners.