Standard Five Compendium 6 - Continuous Improvement Linked to Annual Outcome Measures

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Standards Alignment: This packet, entitled Continuous Improvement Linked to CAEP’s Annual Measures, presents evidence for: Standard Five - The provider maintains a quality assurance system that consists of valid data from multiple measures and supports continuous improvement that is sustained and evidence-based. The system is developed and maintained with input from internal and external stakeholders. The provider uses the results of inquiry and data collection to establish priorities, enhance program elements, and highlight innovations.

CAEP R5.4 Continuous Improvement - The provider regularly, systematically, and continuously assesses performance against its goals and relevant standards, tracks results over time, and documents modifications and/or innovations and their effects on EPP Outcomes.

Description of Evidence

As a result of Grambling State University’s ongoing maintenance and improvement activities, the Data Day program review process, and the self-study process, numerous opportunities for continuous improvement have been identified, planned, and implemented.

The Evidence, below, presents Table 1 that includes outcome measures, and A link to a specific website that includes the public display of the CAEP Annual Reporting Measures as well as specific consumer information. A dedicated Table to Initial Programs includes a synopsis of improvements/changes noted in evidence throughout the self-study, basis for an improvement or change, and impacted CAEP component(s).

Evidence

CAEP’s outcome and impact measures are monitored and reported as indicated in the Quality Assurance Assessment Manual. Analysis of trends, comparisons with benchmarks, evidence of corresponding resource allocations, and future directions based on data can be found in all Compendia within the Self-Study.

Table 1 Outcome Measures

Outcome Measure Synopsis

Measure 1 (Initial):

Completer effectiveness. (R4.1)

Measure 2 (Initial):

Satisfaction of employers and stakeholder involvement. (R4.2|R5.3)

Completer Effectiveness

Satisfaction of Employers and Stakeholder Involvement

CAEP Measure 3: Ability of completers to meet licensing

The EPP are continuously engaged in efforts to improve our programs in order to improve teaching and learning for all children. Our goal is to produce excellent educators who engage in ongoing inquiry and intellectual work and are committed to the highest standards of professionalism, democratic citizenship, quality and high achievement in a context that affirms human diversity. EPP candidates are required to take and pass applicable Praxis Principles of Learning and Teaching (PLT) tests and Praxis subject area test(s) to be recommended to the state of Louisiana for licensure. Our EPP uses and analyzes two Praxis data sets to obtain a full picture of how well our candidates can apply content and pedagogical knowledge and meet licensure

Initial Programs Standard Five Compendium 6 Continuous Improvement Linked to Annual Measures

Measures Outcome Measure Synopsis requirements. The EPP requires that all initial and advanced teacher candidates complete and pass all state requirements prior to graduation. Data from Title II reports show that our completers have a 100% pass rate demonstrating that all of our completers meet state certification requirements.

Table

CAEP Measure 4: Ability of completers to be hired in education positions for which they have prepared

Initial Programs

2019-2020 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 5%

BA Music Education – Instrumental K-12/Vocal K-12 – 2%

BS Kinesiology: Pedagogy (Teaching K-12) – 1%

MAT – No Graduates

2020-2021 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 50%

BS Secondary Education and Teaching (Biology/Mathematics/Chemistry) 10%

BA Music Education – Instrumental K-12/Vocal K-12 – 10%

MAT Elementary Education and Special Education (Mild/Mod) - 100%

MAT Secondary Education & Spec Ed. (Mild/Mod) 100%

2021-22 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 50%

BS Kinesiology: Pedagogy (Teaching K-12) – 50%

MAT Elementary Education and Special Education (Mild/Mod) - 100%

MAT Secondary Education & Spec Ed. (Mild/Mod) 100%

Initial Programs

2019-2020 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 100%

BA Music Education – Instrumental K-12/Vocal K-12 – 100%

BS Kinesiology: Pedagogy (Teaching K-12) – 100%

MAT – No Graduates

2020-2021 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 43%

BS Secondary Education and Teaching (Biology/Mathematics/Chemistry) 0%/100%

Grambling Measure: Graduation Rates

BA Music Education – Instrumental K-12/Vocal K-12 – 0%

MAT Elementary Education and Special Education (Mild/Mod) - 100%

MAT Secondary Education & Spec Ed. (Mild/Mod) 100%

2021-2022 (includes fall and spring terms)

BS Elementary Education (Grades 1-5) 83%

BS Secondary Education and Teaching (Biology/Mathematics/Chemistry) 0%/100%

BA Music Education – Instrumental K-12/Vocal K-12 –100%

BS Kinesiology: Pedagogy (Teaching K-12) – 100%

MAT Elementary Education and Special Education (Mild/Mod) - 100%

MAT Secondary Education & Spec Ed. (Mild/Mod) 100%

1 Outcome

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

CAEP ANNUAL REPORTING MEASURES: Below is a link to a webpage that publicly displays Grambling State University/College of Education Annual Reporting Measures as well as specific consumer information. CAEP reviewers are invited to click on this link to view the webpage in its entirety.

CAEP ANNUAL REPORTING MEASURES

Measure 1 (Initial): Completer effectiveness. (R4.1)

Measure 2 (Initial): Satisfaction of employers and stakeholder involvement. (R4.2|R5.3| RA4.1)

Measure 3 (Initial): Candidate competency at completion. (R3.3)

Measure 4 (Initial): Ability of completers to be hired (in positions for which they have prepared.)

This entire compendium was dedicated to displaying the use of data, evidence, and the input of internal and external stakeholders, to guide curriculum, assessment, and program improvements. At the time of the site visit, Site Visitors will be provided with an update on outcomes. Below are the findings and continuous improvements related to each CAEP Standard.

Initial Programs Standard Five Compendium 6

to Annual Measures

Quality Assurance Chart – Initial Programs

Various assessments are used in the undergraduate and graduate initial programs to address InTASC Standards 1, 2 and 3. Due to the low numbers, it is difficult to analyze all programs. For example, in fall 2019, only 1 candidate completed the EDPT 528 Case Study Project. The low numbers can sometimes be misleading.

Graduation rate of candidates in the undergraduate initial program are impacted by not meeting the passing score on various tests. While supports such as tutoring, The Sage group and increased communication through mandatory meetings and class meetings have been put in place, it is important to closely monitor the progress of students.

Candidates in the initial program performed at are near the target of 3.0 over the three cycles of data. The data from all

GSU will pay close attention to the low Ns during data days held each semester.

Dr. Cheyrl Ensley

A continued focus will be placed on the importance that candidate study for, sit for and pass the Praxis II Content Exam earlier during their academic career. Ways to prevent procrastination and decrease anxiety around the test will be explored.

Dr. Cheyrl Ensley

Although the numbers are small in the program, GSU will continue to monitor this program over the

Dr. Cheyrl Ensley

Standard One Compendium 1 – R1.1 Learner Growth and Development

Standard One Compendium 2 – R1.2 Application of Content

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

programs show that candidates are experiencing growth between completion of the formative and the summative evaluation. However, the growth was not experienced in the secondary program.

While candidates in the undergraduate initial program performed at the target level on the Praxis II Content Exam over the three data collection cycles, a downward trend was evident in the area of Social Studies. This is an area to be monitored. While the decrease may not be considered substantial, and there was even a one-year increase (2020-21), it is important to determine if there is a particular concept that is causing candidates difficulty.

next few cycles, specifically because the scores were below the target of 3.0.

Faculty will seek feedback from candidates after they complete the examination, focusing on any areas in the social studies section that candidates felt that they were not fully prepared to address or that the scores indicate a lag. Additionally, feedback will be gathered regarding what supports were impactful in the success of the candidate. Additionally, GSU will look at the subcategories of the Social Studies test to target areas with the greatest need to support student success. The History -

Social Studies

Dr. Cheyrl Ensley

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Initial Programs Standard Five Compendium 6

to Annual Measures

Education Curriculum has made changes to the social studies education curriculum. Non-tested and duplicate general education requirements have been eliminated in favor of courses that align to PRAXIS testing strands.

A system has been put in place to ensure candidates from all areas submit all required paperwork before entering the College of Education.

Students from all areas will enroll in ED 201 Advisee Report within the Curriculum and Instruction Department affording them one on one contact with an advisor in the C & I department.

Candidate knowledge of the Conceptual Framework will be a focus area to ensure that students are familiar with the framework and able to articulate the meaning.

The pictorial has been updated and will be displayed throughout the building. The pictorial will be included in syllabi. The department will complete a curriculum

Compendia

Standard One Compendium 3 – R1.3 Instructional Practice

Dr. Cheyrl Ensley

Standard One Compendium 4 – R1.4 Professional Responsibility

Dr. Cheyrl Ensley

Standard Two Compendium 1 - R2.1

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

mapping, emphasizing where dialog and instruction regarding the Conceptual Framework is expected to be included.

Standard Two Compendium 2 - R2.2 Clinical Expectations and Training

Remain consistent with stakeholder involvement

A spring 2023 meeting will focus on modifying and innovating field experiences now that schools are back faceto-face. Lessons learned from the Pandemic will be shared and discussed.

Dr. Cheyrl Ensley/OPLE Director

Grambling State University has established extensive partnerships with multiple districts throughout the state of Louisiana. However, due to the small size of the state and close-knit professional relationships that exist, these partnerships are often developed and continued through informal meetings and arrangements.

Moving forward, the College of Education will continue to hold a formal meeting with each district to review partnership arrangements annually.

Dr. Cheyrl Ensley/OPLE Director

Collaboration to provide support for clinical Policy regarding development, review, OPLE Director

Partnerships for Clinical Preparation

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

educators is outlined by the Residency Coordinator in the initial training meeting where information is provided. As the program is small and relationship based the Residency Coordinator provides one on one support for clinical educators as well as university supervisors.

Collaboration on the evaluation of clinical educators as it relates to roles and responsibilities can be seen through the resident and university supervisor evaluation process outlined in the Residency Handbook in Forms section. ***While collection has been occurring, analysis of data and dissemination of results due to the COVID19 pandemic.

As residency is a relatively new endeavor in partnership with the State of Louisiana, data regarding stakeholder feedback for the implementation and building of the program is still being gathered and

and revision of support will be outlined in Standard 5 to support data-based decisionmaking regarding support.

Policy regarding dissemination and review of data is outlined in Standard 5. OPLE Director

Our transition to faceto-face field experiences has started though limited. The EPP recertifies with the Danielson Group each year as evidenced in R2.1. The group has released a new rubric

OPLE Director

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Standard Two Compendium 3 - R2.3 Field Experiences: Depth and Breadth

Initial Programs Standard Five Compendium 6

to Annual Measures

used to inform continuous improvement model as evidenced in R2.1 and R2.2 Compendia Narrative. Following the COVID-19 pandemic where temporary remote measures were in place, there has not been a full cycle of implementation of face-to-face program field experiences following the redesign except for the Residency data presented with face-to-face observations occurring and collected utilizing the Danielson Rubric. The EPP is looking forward to moving past the pandemic and fully implementing face to face field experiences for candidates. Following an internal review of sufficient depth, breadth, diversity, coherence, and duration of field experiences that scaffold candidate experiences in preparation for residency in preparation for the CAEP Self Study, completion of course based of field experience is being captured by number of hours, however, data was inconsistent and

and a committee was created Fall 22 to begin realignment and identification of field experiences to determine a transition plan to the new Danielson Rubric that will follow the implementation of the new Watermark: SL&L Data Collection System.

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Taskstream was not being utilized effectively to store data. However, this has been acknowledged in Standards 1 and 5 and processes regarding intentional collection have begun.

With only two percent of teachers in the US being Black males, it is imperative that solutions be developed and implemented. The EPP has created an effective means to that end.

Focus Area 1: The EPP has established extensive partnerships with multiple districts throughout the state of Louisiana. However, due to the small size of the state and close-knit professional relationships that exist, these partnerships are often developed and continued through informal meetings and arrangements. Moving forward, the College of Education will continue to hold a formal meeting with each district to review partnership arrangements annually.

Focus Area 2: Remain consistent with stakeholder

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Standard Three Compendium 1-R3.1 Access and Equity

Dr. George Noflin

Initial Programs Standard Five Compendium 6

to Annual Measures

involvement through quarterly meetings.

Transition points stand alone for all programs. It will be critical to embed these into each specialized program. Digital Planning Sheets will help to protect the versioning of documents.

Focus Area: The dire shortage of teachers is further exacerbated by the extreme underrepresentation of Black males as classroom teachers in schools nationwide. With only two percent of teachers in the US being Black males, it is imperative that solutions be developed and implemented. The EPP has created an effective means to that end. It is an EPP Signature Program entitled, “Call Me MiSTER” (CMM) The program effectively addresses recruitment, retention, and graduation of Black Males, thus providing a unique approach which will be implemented comprehensively in all EPP programs. The program originated at Clemson University and the GSU model is uniquely developed to address teacher shortages in the state

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Standard Three Compendium 2-R3.2 Academic Achievement and Support

Dr. G eorge Noflin

Initial Programs Standard Five Compendium 6

to Annual Measures

of Louisiana. It is the only program in the nation that was granted federal funding and has effectively more than doubled its enrollment within the last year. The CMM model holds great promise as the tenets are in alignment with the EPP’s recruitment and retention goals.

The EPP Signature Program entitled, “Call Me MiSTER” (CMM) will be continued and monitored.

Focus Area(s):

There were some inconsistencies in some terms where data was not disaggregated by gender.

Transition points stand alone for all programs. It will be critical to embed these into each specialized program. Digital Planning Sheets will help to protect the versioning of documents.

Mrs. Tiffany Jackson

Standard Three Compendium 3-R3.3 Monitoring Academic and Non-Academic Factors

Initial Programs

Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Embed transition points into program DRFs in Taskstream Complete DegreeWorks training and implement to enhance candidate experience

Digitize Planning sheets to ensure ease of access for all

Faculty identified the need to consistently disaggregate data by gender to support the focus of the University’s mission. This will be a focus area moving forward.

Low Ns = Case Studies

The data from the 2020 Louisiana Dashboard, the most recent state generated data available, indicated that the highest scores for both undergraduate and MAT completers were in Demonstrated Teacher

We will work on developing knowledge and skills to allow completers to have greater direct impact on K-12 student learning. It is apparent that they have the skills to teach, so we will look at augmenting engaging and

Dr. Kathryn NewmanLead;

Dr. Harrison Jones, Mrs. Tiffany Jackson, Dr. Cheyrl Ensley

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Standard Four Compendium 1-R4.1 Completer Effectiveness

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Skill. The lowest scores for both undergraduate and MAT were in Impact on K-12 Students, although scores were higher for the MAT completers.

scaffolding skills methods to improve P12 student learning. In addition, the Louisiana Dashboard scores are not disaggregated by program. Consequently, the EPP will pilot a program to ask selected principals to evaluate completers at their schools using the Danielson Framework for Teaching to ascertain if there are differences in the effectiveness of candidat es according to area of certification, race, ethnicity or gender.

The data from the 2019-2020 Louisiana Scorecard: indicates there were several areas of focus for continuous improvement. For undergraduate and MAT completers, the focus areas of lowest scores were in Quality of Feedback and Candidate Performance where the EPP received a score of 2.0/4.0, and

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Initial Programs

Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Pilot data revealed that additional training on the Danielson rubric for supervisors and principals would be beneficial.

Ensure that all principals or supervisors of completers have standardized familiarity and training in using the Danielson rubric and the Special Education addendum according to completer to ensure reliability across observers and districts. Provide faceto-face and online training to principals/supervisors at least once a year and calculate reliability. Strive for reliability coefficient o f >.80 among participants.

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Cheyrl Ensley, Dr. Dagne Hill, Dr. Larry Proctor

This year an employer focus interview was added to complement survey data.

Continue to collect interview data about completer strengths and weaknesses in a rapidly changing teaching environment to triangulate survey

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Cheyrl Ensley, Dr. Dagne Hill, Dr. Larry Proctor

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

findings. Continue using focus group interviews to pinpoint specific areas of need, especially in response to ongoing covid environmental challenges.

Completer needs should be disaggregated by teaching environments.

Use data to increasingly pinpoint needs across the areas in which completers teach. Investigate to determine if completers in some areas are more successful or less successful in certain teaching environments. Review all assessment outcomes more closely across the overall teaching environments including urban, rural, suburban, regional, economic area, size of district, traditional, online, hybrid, etc., to pinpoint specific areas of need and strengths, both generally

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Cheyrl Ensley, Dr. Dagne Hill, Dr. Larry Proctor

Validity of the interview questions have not yet been established

Determine Validity of the Interview questions. Confirm

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Cheryl

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Initial Programs Standard Five Compendium 6 Continuous Improvement Linked to Annual Measures

validity of interview questions by using PK16 Council/Board or another representative sample of stakeholders. Use the Lawshe Content Validity formula.

The use of focus groups is a newly adopted practice. Ensure Ongoing Continuous Quality Improvement practices. Continue using focus group interviews to bolster areas in which the response rates are lower, occasions when the completers have moved out of the area, or when long term follow up data are not as complete. Securing additional data could further document that completers have the knowledge, skills and dispositions required to create educational settings that are equitable, inclusive, and welcoming for all diverse P-12 learners.

Continue using focus groups to pinpoint

Ensley, Dr. Dagne Hill, Dr. Larry Proctor

Dr. Kathryn Newman, Lead; Mrs. M elanie Monroe, Dr. Cheyrl Ensley, Dr. Dagne Hill, Dr. Larry Proctor

Dr. Kathryn Newman, Lead; Mrs. Melanie

Initial Programs Standard Five Compendium 6

candidates were included in the process.

specific areas of need, especially in response to ongoing Covid and post-Covid environments and include MAT completers in interviews or conduct interviews with them separately.

Using data to pinpoint needs across the areas in which complet ers teach— are completers in some areas more successful or less successful in certain teaching environments? This can be done by looking at all results more closely across the teaching environments urban, rural, suburban, regional, economic area, size of district, traditional, online, hybrid, etc., to pinpoint specific areas of need and strengths both generally and for specific areas.

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Monroe, Dr. Mary Ghongkedze, Dr. Larry Proctor

Initial Programs Standard Five Compendium 6

Continuous Improvement Linked to Annual Measures

Providing more practice in classes for completing pre-lesson assessments and diversifying instruction to meet the needs of students Provide more experience and training in working with diverse P-12 students.

Work with College of Arts and Science to provide more content knowledge to their candidates and provide more practice with the content knowledge, and the P-12 curriculums used in the schools (e.g., annotating lessons as advocated by the Resource Coordinator/Liaison).

Standard Five Compendium 1-R5.1 Quality Assurance System

One impediment to the timely and efficient collection and analyses of course-based key assessment data has been the university’s transition to a new learning

To address and correct this issue, the head of the department will continue to monitor the adherence to the existing policy to ensure candidates are

Dr. Danielle Williams

Initial Programs Standard Five Compendium 6

Improvement Linked to Annual Measures

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

management system (LMS). GSU previously used Moodle, which integrated with Taskstream and provided a seamless import of data from course- based, key assessments into Taskstream. The new LMS, Canvas, does not support the same integration. Therefore, students must submit key assessments twice: once in Canvas for evaluative purposes and again in Taskstream for assessment purposes. This has resulted in missing data in Taskstream.

uploading assessment artifacts promptly for scoring by faculty. Also, the assessment coordinator will train faculty on running program reports and augment permissions in Taskstream to increase their engagement in the Quality Assurance System. Lastly, the assessment coordinator and dean are in negotiations with Watermark to transition from Taskstream to Student Learning and Licensure, their new solution that boasts seamless LTI integration with Banner Web and Canvas. This implementation process is slated to begin in the fall and include a pilot in the spring of 2024.

Standard Five Compendium 2-R5.1 and R5.2 Representativeness of Data

Monitoring of candidates' satisfaction

of transition point requirements within programs can be improved.

To ensure that candidates have met the proficiency criteria for key course-based assessments at each transition point prior to beginning their clinical

Dr. Kathryn Newman

Standard Five Compendium 3-R5.2 and R5.4 Actionability of Results

Initial Programs

Standard Five Compendium 6

Improvement Linked to Annual Measures

practice, portal committee members should be given permissions to review student Taskstream folios for R esidency I applicants. The Office of Professional Laboratory Experiences (OPLE) director will inform any candidate that has scored lower than the required 3 on any assessment to resubmit. Resubmissions will be evaluated by the candidate’s program chair prior to admittance to Residency 1.

Transition Points - Edit transition points DRFs in Taskstream to make it more user-friendly for candidates and evaluators to standardize expectations across initial programs.

Grambling’s faculty is small, and many members teach in multiple programs. This can create

To improve and standardize recordkeeping procedures and capture data-

Dr. Cheyrl Ensley and Dr. Debbie Thomas

Initial Programs Standard Five Compendium 6

Improvement Linked to Annual Measures

issues with maintaining a schedule for formalized program meetings and capturing data from informal meetings.

informed decisionmaking that drives programmatic changes for continuous improvement, the EPP has created an EPP Minutes Form in Microsoft Forms that will capture key information in an Exc el file. This will ensure that this pertinent information is easily accessible to all on the SharePoint site and can be quickly formatted into a table for reporting purposes.

The EPP-created assessments have met the minimal 80% or .80 or above to establish content validity, and 75% or .75 or above to establish interrater reliability or agreement. However, several criteria within the instruments failed to meet the established threshold.

GSU faculty will review the comments from SME respondents and draft changes to the assessments. SMEs and other stakeholders will be invited to provide feedback and assist in the co -creation of revised directions and rubrics. After the assessments have been approved by the QAS Review panel, new assessments will be piloted during the 2023-2024 academic year. The current data

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Standard Five Compendium 4-R5.2 Reliability and Validity of EPP Assessments

Mrs. Tiffany Jackson and Dr. Kathryn Newman

Standard Five Compendium 5-R5.3 Stakeholder Involvement

Initial Programs Standard Five Compendium 6

Standard Five Compendium 6-R5.4 Continuous Improvement Linked to Annual Measures

For the CAEP Accountability Measures, GSU discovered as a part of the self-study that information on the website pertaining to completer data did not include our advanced programs.

quality review cycle will be amended to allow for an additional reliability and validity review following the completion of the pilot.

The College of Education will collaborate with the Alumni Center to update these measures to include the advanced programs.

Mrs. Melanie Monroe

Dr. Cheyrl Ensley, Ms. Rosiland Russell, Dr. Lacy Hitt