Standard Five Compendium 6 - RA5.4 Continuous Improvement Linked to Annual Outcome Measures

Advanced Programs Standard Five Compendium

6

Continuous Improvement Linked to CAEP Annual Measures

Standards Alignment: This packet, entitled Continuous Improvement Linked to CAEP’s Annual Measures, presents evidence for:

CAEP RA5.4 Continuous Improvement: The provider regularly, systematically, and continuously assesses performance against its goals and relevant standards, tracks result over time, and documents modifications and/or innovations and their effects on EPP Outcomes.

Description of Evidence

As a result of Grambling State University’s ongoing maintenance and improvement activities, the Data Day program review process, and the self-study process, numerous opportunities for continuous improvement have been identified, planned, and implemented.

The Evidence, below, presents Table 1 that includes outcome measures, and A link to a specific website that includes the public display of the CAEP Annual Reporting Measures as well as specific consumer information. A dedicated Table to Initial Programs includes a synopsis of improvements/changes noted in evidence throughout the self-study, basis for an improvement or change, and impacted CAEP component(s).

Evidence

CAEP’s outcome and impact measures are monitored and reported as indicated in the Quality Assurance Assessment Manual. Analysis of trends, comparisons with benchmarks, evidence of corresponding resource allocations, and future directions based on data can be found in all Compendia within the SelfStudy.

Outcome Measure Synopsis

Measure 1 (Advanced: Completer effectiveness. (RA4.1) Completer Effectiveness

Measure 2 (Advanced): Satisfaction of employers and stakeholder involvement. (R4.1|RA5.3)

CAEP Measure 3: Ability of completers to meet licensing (RA3.4)

Satisfaction of Employers and Stakeholder Involvement

The EPP are continuously engaged in efforts to improve our programs in order to improve teaching and learning for all children. Our goal is to produce excellent educators who engage in ongoing inquiry and intellectual work and are committed to the highest standards of professionalism, democratic citizenship, quality and high achievement in a context that affirms human diversity. EPP candidates are required to take and pass applicable Praxis Principles of Learning and Teaching

Table 1 Outcome Measures

Advanced Programs Standard Five

Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Table 1 Outcome Measures

Outcome Measure Synopsis

(PLT) tests and Praxis subject area test(s) to be recommended to the state of Louisiana for licensure. Our EPP uses and analyzes two Praxis data sets to obtain a full picture of how well our candidates can apply content and pedagogical knowledge and meet licensure requirements. The EPP requires that all initial and advanced teacher candidates complete and pass all state requirements prior to graduation. Data from Title II reports show that our completers have a 100% pass rate demonstrating that all of our completers meet state certification requirements.

Advanced Programs

2019-2020 (includes fall and spring terms No Graduates

Grambling Measure: Graduation Rates

2020-2021 (includes fall and spring terms) MED Curriculum and Instruction (Reading) 100%

2021-2022 (includes fall and spring terms) MED Special Education 100%

Measure 1 (Advanced): Completer effectiveness. (RA4.1)

Measure 2 (Advanced): Satisfaction of employers and stakeholder involvement. (R4.1|RA5.3)

Measure 3 (Advanced): Candidate competency at completion. (RA3.4)

Measure 4 (Advanced): Ability of completers to be hired (in positions for which they have prepared.)

CAEP ANNUAL REPORTING MEASURES: Below is a link to a webpage that publicly displays Grambling State University/College of Education Annual Reporting Measures as well as specific consumer information. CAEP reviewers are invited to click on this link to view the webpage in its entirety.

CAEP ANNUAL REPORTING MEASURES

This entire compendium was dedicated to displaying the use of data, evidence, and the input of internal and external stakeholders, to guide curriculum, assessment, and program improvements. At the time of the site visit, Site Visitors will be provided with an update on outcomes. Below are the findings and continuous improvements related to each CAEP Standard.

Focus Area: For the CAEP Accountabi lity Measures, GSU discovered as a part of the self-study that information on the website pertaining to completer data did not include our advanced programs. The College of Education will collaborate with the Alumni Center to update these measures to include the advanced programs.

Advanced Programs Standard Five Compendium 6

to

Annual Measures

M.Ed. in Curriculum & Instruction –

Reading:

M.Ed. in Curriculum & Instruction -Reading: While the skills are addressed in most of the courses, EPP plans to collect data regularly through standardized practices through this self-study. Unlike the undergraduate program, where they have three specific times to collect data, the graduate program does not.

M.Ed. in Special Education:

While candidates are familiar with and using the technologies available in the schools, we would like to provide more focus on assistive and augmentative technologies. Additionally, feedback from completers indicated that they would like more practice developing plans such as IEPs and ITPs, and leading stakeholder groups.

While the skills are addressed in most of the courses, EPP plans to collect data regularly through standardized practices through this self-study. Unlike the undergraduate program, where they have three specific times to collect data, the graduate program does not. EPP will develop beginning and end rubrics for graduates in the reading program.

M.Ed. in Special Education:

While candidates are familiar with and using the technologies available in the schools, we would like to provide more focus on assistive and augmentative technologies. Additionally, feedback from completers indicated that they would like more practice developing plans such as IEPs and ITPs, and leading stakeholder groups. Similarly, to the M.Ed. in Curriculum & InstructionReading, the EPP will develop

Dr. Cheyrl Ensley

Standard One Compendium 1 – RA1.1 Candidate Skills

Advanced Programs Standard Five Compendium

6

Continuous Improvement Linked to CAEP Annual Measures

more beginning and end rubrics for those in the program.

Reading

Reading

The program strives for continuous improvement in areas where there is lack or limited emphasis and is one of the major goals for this program. Although the students met 100% expectation, the area of change is meeting standard 4.1 at the highly effective level.

Master of Education-Special Education

Within the Master of Education Special Education program, candidate mean scores were continually within the effective: proficient to highly effective range on the four CAEP Standard A.1.1 skills and CAEP Standard A.1.2. The mean scores did increase slightly over the three data collection cycles.

Focus Area(s): The program strives for continuous improvement in areas where there is lack or limited emphasis and is one of the major goals for this program. Although the students met 100% expectation, the area of change is meeting standard 4.1 at the highly effective level. Candidates will need to articulate and demonstrate more insight on diversity and equity in the classroom. EPP plans to make some adjustments to strengthen the program as well as candidates' content knowledge and pedagogical skill on the elements of standard. It plans to implement a research approach that focuses on action research in various aspects of culturally responsive teaching, reflections of personal biases and microaggressions, collaborations with other school colleagues to embark on inquiry research on issues of inequity in both classroom and school will be infused across specific courses.

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Standard One Compendium 2 – RA1.1 Applications of Data Literacy

Dr. Cheyrl Ensley

Advanced Programs Standard Five

Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis

Focus Areas Responsible Outcome

Special Education

Focus Areas:

Focus Area 1: The third cycle data occurred during the Spring 2023 academic year. There is a need for more candidates in the program.

Focus Area 2:

Within the Master of Education Special Education program, candidate mean scores were continually within the effective: proficient to highly effective range on the four CAEP Standard A.1.1 skills and CAEP Standard A.1.2. The mean scores did increase slightly over the three data collection cycles. The program chair will monitor candidate performance on this assessment for additional data cycles to determine if adjustments should be made to the course content or instruction in order to further enhance research skills. Focus Area 3: Writing the Informed Consents by not only including all required information (PL 93-380), but also making the level match the comprehension level of the

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Advanced Programs Standard Five Compendium 6

Improvement Linked to CAEP Annual Measures

Focus Areas Responsible Outcome

subject(s). Although the Spring 2023 candidate mastered the APA format not only for bibliographic information but also for citing within the text, faculty believe that additional practice is still needed for all candidates in using APA format for citing research and formatting the research project.

Focus Area 1:

M.Ed. in Curriculum & Instruction-Reading:

M.Ed. in Curriculum & Instruction-Reading:

Review of candidate performance on this assessment highlights the importance of collaborative activities throughout all the Advanced programs. Candidates mostly performed at the effective proficient to effective emerging range.

M.Ed. in Special Education-Mild/Moderate: Candidates performed overall in the “Effective Proficient” range, but there were areas where candidates were evaluated as demonstrating skills at the “Effective Emerging” range.

Review of candidate performance on this assessment highlights the importance of collaborative activities throughout all the Advanced programs. Candidates mostly performed at the effective proficient to effective emerging range. We would want them to be at the highly Proficient range. The goal is to encourage more collaboration with teachers from other schools, so during the Annual spring conference they could present their findings and create awareness of best practices. We plan to convince principals, with whom we have a wonderful relationship to grant permission to candidates to attend, conferences. Focus

Dr. Cheyrl Ensley

Standard One Compendium 3 – RA1.1 Collaborative Activities

Advanced Programs Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Area 2:

M.Ed. in Special EducationMild/Moderate:

There were only two (2) cycles of data so a focus has to be on getting more candidates into the program. Successful collaboration is paramount in special education. Candidates are expected to be active and collaborative partners when working with students with exceptional learning needs. Candidates performed overall in the “Effective Proficient” range, but there were areas where candidates were evaluated as demonstrating skills at the “Effective Emerging” range. This was most evident in working with the curriculum to make it accessible to all students in the general education classroom. We will focus upon those areas, and in working with candidates to engage in more annotations of lesson planning before teaching it to develop teaching strategies that meet district guidelines yet make the curriculum more accessible and supportive to all students, ev en when working with highly structured lesson plans.

Advanced Programs

Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Focus Area 3:

While candidates participated in planning meetings such as IEPs, data from RA Standard 4 indicated that candidates wanted more experiences with taking the leadership in developing thos e documents. The observations in 2018 and 2019 were done with one observer. In the future, at least two observers should work together to ensure that observations have interrater reliability or should be recorded so that interrater agreement can be assure d.

Standard One Compendium 4 – RA1.1

Data suggests that graduate candidates need improvement in completing APA citations in research.

There have been some graduate candidates who have struggled to complete their proposal during the course.

More emphasis is needed on the APA format of research proposals including citing within the body of the proposal and building the short literature review.

Determining why some advanced candidates are able to complete the proposal in one semester, and others struggle.

Dr. Cheyrl Ensley

Standard One Compendium 5 –

Low numbers that are currently in the program, but the numbers are slowly rebuilding after the pandemic.

Praxis II: Special Education: Core Knowledge and Mild to Moderate Applications (5543): More practice with case histories and scenarios requiring the

Dr.

Cheyrl Ensley

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Use of Research

RA1.2 Provider Responsibilities

Advanced Programs Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

Graduate candidate scores for the Praxis II: Special Education: Core Knowledge and Mild to Moderate Applications (5543) are lower than other content exam scores.

application of knowledge in a testing session. Stated in the Standard R4 interview, more practice with creating and practice leading IEP meetings.

More focus on collaboration with stakeholders, knowledge of community resources for behavioral interventions, increased opportunities for making the general education curriculum more accessible, as well as creating opportunities for collaboration between students in the classroom.

Standard Two Compendium 1 - RA2.1 Partnerships for Clinical Preparation

While we have MOUs with the local districts and schools where many of our advanced candidates are teaching, we have relied upon the MOUs with the school districts to be “blanket” for the initial and advanced candidates to document partnerships for our advanced and online candidates.

More formalized, documented meetings are needed to continue to grow Grambling’s rich partnerships with our stakeholders.

There is a need for specialized advisory boards for advanced programs .

Report, the Master of Education programs did not have a formalized field and clinical handbook.

As with the initial programs, we realized that we were also relying on close-knit professional relationships, and informal meetings rather than formal meetings. Therefore, the decision stated in the initial R2.1 compendium to rely on more formal meetings is valid here as well.

We realized that we needed Advisory Boards at the Advanced level to specifically collaborate with the EPP in the areas of Reading Specialist and Special Education. Those boards were created this spring and will have initial meetings this summer. These smaller boards will provide

Dr. Cheryl Ensley/OPLE Director

Advanced Programs Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis

Focus Areas Responsible Outcome

specific actions to be reported to the PK -16 advisory board.

At the time of the Self- Study The handbook is being created this summer and will be available at the time of the CAEP Site Visit.

M.Ed. in Special Education Focus Area 1:

Low numbers: Some of the courses have not been taught in several years due to low enrollment and covid, but we are rebuilding.

We lack the required number of applications of data for some of the Special Education field experience data cycles. We expect to have more for many of these by the end of Fall 2023.

Standard Two Compendium 2 - RA2.2 Clinical Experiences

There have been several instructors listed for ED 530 and the field experience for the course is b eing updated Spring 2023 to reflect current standards.

M.Ed. in READING

The faculty realized that there was more of a focus on literacy reading standards and wanted to highlight writing and language in the assessments. Consequently, there were some modifications to focus on the “Action Research” in ED 506 and add the writing process aspect and Diversity and Equity Awareness to ED 504 assessments.

There is a need to use reading and special education advisory boards to review the field experiences in each program for validity and relevance to the educational setting bi-annually.

We need to include more handson IEP/ITP development practice for those in the Special Education program.

Changes in faculty from retirements resulted in some not having access to key assessments associated with specific courses. As a result, key assessments are now required to be attached to the syllabi and submitted at the beginning of the semester.

OPLE Director

The new syllabus for ED 504 reflects the changes. More emphasis is to improve candidates’ collaboration on a

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Standard Three Compendium 1 – RA3.1 Recruitment

Advanced Programs

Standard Five Compendium 6

to CAEP Annual Measures

Responsible Outcome

Low enrollment numbers can affect data analyses.

common identified problem with other teachers in the districts. We are planning soon to have a course specifically for Dyslexia as the State places emphasis on teaching dyslexia in all schools.

M.Ed. in Curriculum & Instruction- Reading:

Increase the number of advanced candidates in our program through collaboration with the Office of Student Affairs, and specifically Enrollment Management. We must market the program more aggressively throughout the state and nation. We will have to look at short-term and longterm strategies for viability and capitalize on program strengths and flexibility.

Diversity among candidates is limited.

M.Ed. in Curriculum & Instruction- Reading:

Increase the number of diverse candidates in the areas of gender and ethnicity. Anecdotal observations and research into the “crisis of boys” suggest that promoting more males into the field of literacy could have a

Dr. George Noflin

Advanced Programs

positive effect on the academic achievement of P -12 males.

M.Ed. in Special Education:

Similar to Focus Area #1, Increase the number of advanced candidates in our M.Ed. program in Special Education through collaboration with the Office of Student Affairs, and specifically Enrollment Management. Even though this is a shortage area nationwide, we must market the program more aggressively throughout the state and nation. We will have to look at shortterm and long-term strategies for viability and capitalize on program strengths and flexibility.

M.Ed. in Special Education:

Increase the number of diverse candidates in the areas of gender and ethnicity. Anecdotal observations and research into the “crisis of boys” suggest that promoting more males into the field of special education could have a positive effect on the academic achievement of P -12 males by reducing the disproportionality of males and

Advanced Programs Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis Focus Areas Responsible Outcome

particularly males of color in special education programs.

Standard Three Compendium 2 – RA3.2 Candidates Demonstrate Academic Achievement and Ability to Complete Preparation Successfully

Online programs are suffering with low enrollment.

GSU wants to focus on increasing the number of candidates in the cohort to increase sustainability by developing its online programs and further outreach to neighboring districts.

In transitioning from traditional face-to-face to totally online, an advisor noted that occasionally, an online candidate was not monitored to the same degree as a traditional one. Steps have been put in place to increase online candidates' access to advising and support.

Dr. George Noflin

Standard Three Compendium 3 – RA3.3 Monitoring and Supporting Candidate Progression

Online candidates need additional support. Ensuring that all online candidates have the same supports and access to monitoring and support as traditional face-to-face and hybrid candidates.

Use and training of DegreeWorks has not yet begun. Both advisors and candidates will be trained to use and monitor DegreeWorks (the electronic database system coming online this semester)

When all are trained in how to use DegreeWorks , monitor and advise in the system, the older paper based and electronic system of recordkeeping can be retired. DegreeWorks will allow both candidates and advisors to instantly see progress, next steps and needs to complete the degree.

Focus Area 3: Digitize Plan of Study sheets (into fillable pdfs) or explore the use of DocuSign. These are still primarily paper- based, and to get

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Dr. George Noflin

Advanced Programs

Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis

Focus Areas Responsible Outcome

the plans of study signed, they must be routed electronically to each person, signed, scanned and then sent forward. When they are signed by all necessary parties, the candidates receive a copy to keep as part of their documentation. Fillable pdfs and/or use of DocuSign will streamline the process.

Standard Three Compendium 4 – RA3.4 Competency at Completion

Standard Four Compendium 1 – RA.4.1 Satisfaction of Employers

Newly created interview process yielded just one employer representing only one of the two programs.

See Standard One Compendium 1-RA1 for focus areas.

Focus Area 1: Determine the validity of the focused interview questions.

Focus Area 2: Continue the focused interviews and conduct bi-annual focused interviews with supervisors of M.Ed. completers in both programs.

Focus Area 3: Begin to look at ways to provide additional and/or varied leadership opportunities for candidates in Advanced programs. This could also address a completer concern raised in the focused interview) .

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Cheyrl Ensley, Dr. Dagne Hill, Dr. Larry Proctor

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.

Advanced Programs Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

A formal validity study will be administered to see if the questions and format are appropriate for a dvanced programs.

The focused interview is new and needs to undergo a validity study.

While both indicated general satisfaction, one completer that was interviewed noted that seeing sample IEPs was not enough

In the survey. Completers from the M.Ed. in Curriculum & Instruction reading program were generally satisfied, the completer from the M.Ed. Special Education indicated overall dissatisfaction despite noting that much was learned.

We will add at least one assignment within the Special Education program sequence that requires candidates to lead others through the creation of an entire Individualized Education Plan and/or Individualized Transition Plan. Review the courses in the Special Education sequence to see if more can be done to enhance skills across all areas of Special Education

The completer from the Special Education program wanted more training in the IEP process (to assist in leading the evaluative process)

One impediment to the timely and efficient collection and analyses of course-based key assessment data has been the university’s transition to a new learning management system (LMS). GSU previously used Moodle, which integrated with Taskstream and provided a seamless import of data from course- based, key assessments into Taskstream. The new LMS, Canvas, does not support the same integration. Therefore, students must submit key assessments twice: once in Canvas for evaluative purposes and again in Taskstream for

To address and correct this issue, the head of the department will continue to monitor the adherence to the existing policy to ensure candidates are uploading assessment artifacts promptly for scoring by faculty. Also, the assessment coordinator will train faculty on running program reports and augment permissions in Taskstream to increase their engagement in the

Standard Four Compendium 2-R A4.2 Satisfaction of Completers

Dr. Kathryn Newman, Lead; Mrs. Melanie Monroe, Dr. Mary Ghongkedze, Dr. Larry Proctor

Standard Five Compendium 1-RA5.1 Quality Assurance System

Dr. Danielle Williams

Advanced Programs

Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

Compendia Findings/Data Analysis

assessment purposes. This has resulted in missing data in Taskstream.

Focus Areas Responsible Outcome

Quality Assurance System. Lastly, the assessment coordinator and dean are in negotiations with Watermark to transition from Taskstream to Student Learning and Licensure, their new solution that boasts seamless LTI integration with Banner Web and Canvas. This implementation process is slated to begin in the fall and include a pilot in the spring of 2024.

Standard Five Compendium 2-RA5.1 and R A5.2

Representativeness of Data

Students who have take n courses out of sequence have not always been flagged for failure to meet proficiency criteria.

To ensure that candidates have met the proficiency criteria for key course-based assessments at each transition point prior to beginning their clinical practice, program leads should be given access to review student Taskstream folios. Graduate program leads will inform any candidate that has scored lower than the required composite score of 3 on any assessment to resubmit. Resubmissions will be evaluated by the candidate’s professor in the associated course.

Standard Five Compendium 3-RA5.2 and R A5.4

Actionability of Results

Grambling’s faculty is small, and many members teach in multiple programs. This can create issues with maintaining a schedule for formalized program meetings and capturing data from informal meetings.

To improve and standardize record-keeping procedures and capture data-informed decisionmaking that drives programmatic changes for continuous improvement, the EPP has created an EPP Minutes Form in Microsoft Forms that will capture key information in an Excel file.

Dr. Kathryn Newman

Dr. Cheyrl Ensley and Dr. Debbie Thomas

Advanced Programs

Standard Five Compendium 6

Continuous Improvement Linked to CAEP Annual Measures

This will ensure that this pertinent information is easily accessible to all on the SharePoint site and can be quickly formatted into a table for reporting purposes.

Standard Five Compendium 4-RA5.2 Reliability and Validity of EPP Assessments

Some newly revised surveys have not yet undergone a formal validity study.

GSU will continue to follow its schedule for reviewing EPPcreated assessments to establish content validity and inter-rater reliability percentages that meet and/or exceed the requirements to meet CAEP sufficiency of evidence.

Mrs. Tiffany Jackson and Dr. Kathryn Newman

Standard Five Compendium 5-RA5.3 Stakeholder Involvement

Mostly low numbers.

Graduate program leads will follow their proposed Reading and Special Education Advisory Board calendar to involve members in the decision- making process. Focus on writing (M.Ed. in Reading), and more work with IEPs etc. (SPED).

Mrs. Rosalind Russell and Dr. Lacy Hitt

Standard Five Compendium 6-R5.4 Continuous Improvement Linked to Eight Annual Measures

For the CAEP Accountability Measures, GSU discovered as a part of the self-study that information on the website pertaining to completer data did not include our advanced programs.

The College of Education will collaborate with the Alumni Center to update these measures to include the advanced programs.

Mrs. Melanie Monroe

Template for the Presentation of Evidence by Dr. Michele Brewer and Dr. Amber Vraim is licensed under Attribution 4.0 International "College of Education Office of Technology, Assessment, and Compliance: Template for the Presentation of Evidence." Copyright 2020 by Wilmington University.