Providing Hopkinsville Community College with updates and new developments related to the assessment of student learning
SLO Soundbites “Soundbite: „ minimum of sound to a maximum of sense.”
COMMUNICATING EFFECTIVELY, EVALUATED The Student Learning Outcomes Communicate Effectively: Writing Assessment Team met January 27, 2012, and read 96 writing samples. Fifty-six of the samples, or 58 percent, met the standard by scoring at least a 3 (out of a possible 4), which, according to the current scoring rubric, is Satisfactory. Although this is an improvement from the previous assessment period, it still falls short of our 70 percent goal. Before and after the assessment session, members of the team discussed both the assessment process and potential ways to improve student writing. One of the greatest challenges with assessing student writing through our process has been gathering writing samples that can be assessed accurately by the rubric we are currently using. Either we need to coach faculty on what is an appropriate writing sample to submit or we need to make our assessment rubric more flexible. Many of the samples submitted are examples from essay exams, journaling, or other more informal assignments. By their very nature, these artifacts are weak writing samples, and, when we use a fairly formal writing rubric to assess them, the writing samples usually fall short. The Assessment Team feels these types of assignments are not appropriate representations of formal student writing and from now on faculty should submit more formal assignments, such as, research papers, essays, article reviews, lab reports, case studies, etc. Another possible action the assessment team discussed is to create rubric samples that work for different disciplines. We could share these rubrics with discipline faculty, and, when writing samples are turned in by these disciplines, the assessment committee could use discipline-specific rubrics—instead of using one rubric (Continued on page 2)
INSIDE THIS ISSUE C.E. Rubric ...............................2 C.E. Assignment Ideas………… 2 Examining Relationships ........3 E.R. Rubric...............................3 T.C. Assignment Ideas .............4 Thinking Critically ...................4 E.R. Assignment Ideas.............4
SPECIAL POINTS OF INTEREST Assignment ideas Faculty reflections on assessment process
February 2012 Volume 1, Issue 1
COMMUNICATE EFFECTIVELY WRITING A STRONG ASSIGNMENT
Essay #2: Taking a Stand: High School Exit Exams Due date for peer edit: 10/13 (must be at least 3 pages and typed) Final draft due 10/15 (Please turn in with rough draft and editing sheet)
Your Task: Using the articles and editorials provided to you as resources, write a persuasive argument either for or against the exams.
Paper must: -Have a clear, underlined thesis explaining which side you support -Explain/summarize the issue at hand -Give at least two reasons for your stance -Explain at least one aspect of the other side’s argument -Use at least two quotes from provided sources
Remember: -Use MLA format -Cite all quotes, ideas that are not your own, and statistics -Include Works Cited list at the end of essay
You may also consider: -Your own personal experience or the experience of someone you know who is in high school/works at a high school
(From Modesto Junior College’s Student Learning Outcomes Handbook - http://cai.cc.ca.us/ workshops/ SLOFocusOnResults.doc )
COMMUNICATE EFFECTIVELY, CONTINUED (Continued from page 1) for all types of writing. Our SLO requirement that seeks artifacts only from students with 33 general education hours might not provide an appropriate sample of student writing to analyze. For example, a pre-nursing student already with 33 general education hours is often the student with poor academic skills, whereas, stronger students may have entered the nursing program with half that many hours. In this case, the 33 hour requirement might cause us to miss some of the best students and over-represent the weaker students. Since many of our students are weak in paragraph development (topic sentences with support), essay development, and grammar and proofreading, should there be some kind of benchmark of success for students who test into developmental writing, possibly using the COMPASS as a posttest? The Assessment Team would like to continue using the assessment process as a qualitative measure of student writing capability. While HCC faculty members give students many interesting writing prompts, there is room for improvement in this area. Many of the prompts the Team reviewed could have done a better job providing clear directions and assessment expectations. Along with the improvement measures mentioned below, a PD session geared toward creating effective writing prompts would probably be of use to HCC faculty in all disciplines. The HCC English faculty will continue to review the new ENG 101 exit exam— first piloted during the fall of 2010. The exit exam includes both an objective test and an essay portion. The objective test places additional emphasis on correct use of Standard Written English (SWE), which, according to learning outcomes assessment results, is a major weakness of student writers at HCC. The essay portion of the exam assesses the student’s mastery of the writing process, essay format and organization, and critical thinking skills. The exam represents a significant portion of the final course grade, reinforcing the idea that mastering these written communication skills is essential to success in college, in the workplace, and in the student’s personal life. Assessment of student writing across the curriculum has revealed the need for faculty across all disciplines to send students with writing projects to the HCC Writing Center for tutoring help. As a result of this directive, the Writing Center hours
(Continued on page 6)
EXAMINING RELATIONSHIPS AN EXAMINATION
EXAMINE RELATIONSHIPS (Rubric)
Assessing the Student Learning Outcome (SLO) entitled “Examine Relationships” is accomplished by all reviewers reading each artifact and rating it using the rubric shown to the right. The group then discusses the artifacts collectively, noting exceptionality, mediocrity, and weakness. The artifact assignment is then viewed as to its quality for representing an opportunity for students to demonstrate the ability to examine relationships in diverse and complex environments. In the evaluation of artifacts from the fall of 2011, committee members scrutinized whether the assignments allowed students to demonstrate an awareness of relationships in diverse and complex environments. During the course of its examinations, the committee noticed three issues. First, as the assessment process for “Examine Relationships” now stands, it appears that HPC Faculty fully comprehend the nature of this activity. The quality of submitted artifacts has improved over the past three semesters, with more assignments targeting the upper levels of the outcome statement’s rubric. This trend indicates an alteration in faculty perceptions of SLO assessment and perhaps instruction as well. Second, it remains vital for the instructor to write prompts that explicitly allow students to demonstrate their level of competence when examining relationships. On occasion, there were artifacts whose directions were vague or too general. Such instructions provided little opportunity for the student to discover a relationship within the assignment materials, much less examine one in a diverse and complex environment. Finally, in order to avoid these types of artifacts in the future, there must be targeted professional development to better equip instructors to devise effective prompts. The committee will lead this training by pulling examples from artifacts that exhibit superior and less than adequate verbiage, and presenting this information to the faculty as soon as possible. Ted Wilson, Brett Ralph, and Thomas Howell - On behalf of the Evaluate Relationships Committee
Outcome Statement: A student will demonstrate an understanding of relationships within or among diverse environments. Rubric: Score: 4 Able to evaluate relationships in diverse & complex environments Evaluation: judge, criticize, appraise, discriminate, compare, justify, or contrast Score: 3 Comprehension & application of relationships in diverse & complex environments Comprehension/ Application: identify, interpret, convert, demonstrate, summarize, generalize, explain, discover, modify, predict, operate, or solve Score: 2 Able to demonstrate knowledge of the existence of relationships in diverse & complex environments Knowledge: name, reproduce, list, label, define, recall, describe, or state Score: 1 Unable to recognize relationships in diverse and complex environments.
CRITICAL THINKING A STRONG ASSIGNMENT
Using the long essay format discussed in class and described in your syllabus, answer the following question in a logical, cogent, and grammatically correct manner. Remember to use at least one example—ones you think work best—from each time period mentioned below and to connect them in a coherent narrative. From the end of the Civil War to the early twenty-first century, the historical theme of centralization versus decentralization of authority has guided this nation’s development. In the semester’s final essay, your task is to reveal the centralization aspect of this thesis found in the five major phases of American history covered in this class. Using the time periods of “Radical Reconstruction,” “Populism and Progressivism,” “The Great Depression and the New Deal,” “The Civil Rights Era,” and “The Conservative Reaction,” demonstrate Americans’ tendency to concentrate authority.
REFLECTIONS ON ASSESSING CRITICAL THINKING Our Critical Thinking Committee this spring was faced with a number of challenges. The short term challenge of reading the artifacts, assessing them and commenting on them was the immediate task. The long term goal is to reflect upon our experience and suggest changes in the way we assess and report so that our faculty can hone the teaching of critical thinking and make pedagogical changes as needed to improve the outcome of critical thinking. We discussed a number of artifacts from a variety of disciplines ranging from accounting, criminal justice, philosophy and history. One example we looked at was a report—although we found that students sometimes did well in the three rubrics—there was no explicit mention of being critical in the report. The excellent submissions by students showed evidence of critical thinking at times despite lack of clarity in assignments; however, sometimes students just strung a report together without assessing what they were reporting on. Although they seemed to fulfill the assignment, they did not assess the work against standards. Summarizing is a skill and an important one for students, but it is just the jumping off point for assessing. The ideal artifact assignment will use words like analyze information, apply standards and interpret results so that students will not just summarize. The lowest scoring submissions perhaps could be pushed to avoid mere reporting. One suggestion
(HCC Faculty Submission)
Wordle from faculty articles.
from one of the committee members was to include the teacher’s grading rubric with the artifact—this too could help the lower scoring students by guiding their assignment in a more focused manner. The committee also discussed difference of assessments. There was both a quantitative and qualitative aspect. Some were suspicious of the objectivity of the quantitative analysis. One mathematically inclined member of the committee offered to crunch the numbers. The committee seemed to agree that the most important numerical score would not be the mean number but rather to look at the range both to identify the excellent and the deficient. Two ques-
“How can we pare down the number of these, so that we can look at them thoughtfully rather than be overwhelmed by too much information?”
tions seem to me to be important. 1) Are some assignments in need of honing? 2) What can we do for students to bring up the lower scores? These were our immediate concerns; however, as we reflected on the longer term goals we noted a number of different concerns. First, we noted the need of spelling out what an ideal artifact for measuring student critical thinking would look like. Second what kinds of things demonstrate an ideal response to the critical thinking rubric? The two questions are closely interrelated. A few reflections emerged. First, we noticed that some of the artifacts were examinations—as such they were written in some haste. Although it seems that we do want students who have the ability to “think on their feet”—it might perhaps be better to evaluate them when the mode is more leisurely. Second, the ideal assignment should not use words like explain or describe—this encourages simple regurgitation and so assignments that are more clearly worded can guide students and also allow others looking on the evaluations to more clearly focus their scope. As to ideal responses, it is quite easy (and gratifying) to see students organizing, analyzing, applying standards and interpreting their findings. It is easy to say, “I know it when I see it.” There is a sense in which years of grading can hone a sense of smell so that some essays “fragrance” appears in the first words. (Continued on page 6)
COMMUNICATING EFFECTIVELY, CONT’D
(Continued from page 2)
A STRONG ASSIGNMENT
were expanded in 2011 and a documentation/tracking process made available for faculty. Also, during the spring of 2012, a permanent, professional tutor was hired in the Writing Center to meet the increased demand.
Using readings from class, describe the promises for prosperity promised by the free trade doctrine. Has the implementation of free trade ideals produced the promised changes? Explain the social ramifications of globalization for the middle class.
All faculty members will be encouraged to help students develop writing strategies appropriate for the discipline. Faculty members will also be encouraged to provide guided revision opportunities inclusive of the tutoring available on campus. To help in this effort, the HCC English faculty will develop a series of PD sessions and resources designed to help all HCC faculty implement discipline-specific writing strategies in their classes.
—————————— Compare and contrast cell mediated and antibody mediated immunity. —————————— In this exercise, you determine your daily basal metabolic rate, voluntary muscular activity, and specific dynamic action per day. These are used to estimate your total energy requirements per day in kilocalories (kcal). You then calculate your total daily kcal intake. By comparing these two figures, you can determine whether or not your present diet should result in your maintaining, losing, or gaining weight. (HCC Faculty Submissions)
ASSESSING CRITICAL THINKING, CONT’D (Continued from page 5)
Explaining what the “smell” of a good essay is like however is rather difficult to spell out across the disciplines. Although we appreciated getting together across the disciplines sometimes it seemed that we needed help from our colleagues. Given that we had a very heavy workload of artifacts and a large representation of disciplines a larger group would have been helpful. And that leads into the final observation about the process—the sheer number and volume of the artifacts. Although appreciating that so many faculty have contributed artifacts, it also seems that information overload is a difficulty. How can we pare down the number of these, so that we can look at them thoughtfully rather than be overwhelmed by too much information? This question and others mentioned would be a good jumping off point for discussion among the faculty. --Ken Casey, on behalf of the Think Critically Committee