Problem Solving with Reflective Judgment
Overview Tusculum College is re-visioning its approach to educating students. Beginning in 2010 there will be greater institutional emphasis and support behind practices that are learner-centered and intended to develop students’ ability to solve problems through the application of Reflective Judgment skills. This document is a summary of the plan.
The process to get to this point At the Fall Faculty Workshop in August 2007 the Provost kicked off the planning process by providing an overarching description of the task and initiating focus groups charged with identifying student learning needs. A Steering Committee comprised of faculty, staff, and students from multiple sites and programs collected the feedback, reviewed assessment data about student learning, solicited additional feedback through surveys and, after analyzing the emergent themes, proposed three potential student learning areas that Tusculum might address. By campus-wide vote, Problem Solving with Reflective Judgment was selected in May 2008. The Steering Committee proceeded to spend academic year 2008-2009 conducting literature reviews on Reflective Judgment and the practice of teaching through “problems” rather than the more standard approach of disseminating information. In May 2008, the Board of Trustees received an update on the topic as it was becoming defined and understood, and a consultant from the Southern Association of Colleges and Schools Commission on College (SACS-COC) provided comment on our efforts to this point. In the summer and fall of 2009, the Steering Committee oversaw the development of a plan in which Problem Solving with Reflective Judgment would begin to affect the Tusculum curriculum in two broad phases. In the first two years of this plan, students would encounter coursework in lower level classes that had been modified to involve complex problems and coaching on Reflective Judgment. In the third and fourth year of this plan, students would take what they had practiced in the classroom, into the world and begin to address problems outside the classroom with Reflective Judgment. The process of developing this plan occurred over nearly 30 months and involved faulty and staff from across Tusculum’s campuses. During this time there were regular cycles of brainstorming and idea solicitation, development of proposals, presentation, feedback, and integration of that feedback. The result is a plan to enhance student learning that is not only integrated with our civic arts tradition, but also connected to research on effective educational practice. It will serve to meet the needs of our students who will face many challenges as citizen-leaders. The mind is not a vessel to be filled but a fire to be kindled.
~Plutarch
The need Accountability requires that from time to time institutions evaluate their progress toward goals, and perhaps modify their efforts. Additionally, our students’ learning needs have substantially changed as the world has grown ever more complex. Only a few decades ago it may have been said that an educated person knew a certain body of facts. However, the rapid creation of new knowledge means that what is known and relevant in a given discipline is always growing and changing, quickly becoming obsolete. Awash in information, what students now need is the ability to recognize relevant, credible information which they then use to support logical conclusions or make smart decisions. For both these reasons - the periodic reexamination of where we are in relation to our goals and the changing needs of students - we find ourselves reassessing our efforts and aligning more closely with the Association of American Colleges and Universities’ call to teach “students to find and evaluate evidence and to take into account both context and competing perspectives as they form judgments about significant questions” (AACU, 2008, p. 5). In short, our students need Reflective Judgment (RJ). Reflective Judgment Recognizing the limits (uncertainty) of knowledge and yet being able to construct a reasonable knowledge claim or solve a problem with a defensible solution, all while holding this view tentatively and recognizing that with better tools or a different perspective that the “truth” of the situation may change. Although Tusculum College has only recently embraced Reflective Judgment as an educational initiative, the Office of Institutional Research has provided data which support our need to improve in this area.
•
The MAPP test of critical thinking (2004-2005 and 2007-2008) shows that few of our students are proficient in critical thinking as they enroll at Tusculum; a retest of these same students four years later indicates this persists as a weakness as they graduate.
•
The CAAP test of critical thinking (2001-2008) also indicates our students score below the national average as they enter, and they conclude their education with roughly the same level of critical thinking skills.
•
The Reasoning about Current Issues test (RCI), administered during the spring and fall of 2009, is a test of RJ. Initial results fail to support the view that our present efforts positively and meaningfully effect an upward trend in RJ.
•
Our Commons program aims to develop and measure critical thinking in students. Although the assessment procedures for this learning outcome need refining, early indications are that our students perform near the middle of range.
Related: Indirect Measures of Student Learning and Campus Culture
•
The National Survey of Student Engagement (NSSE), administered in 2007, reveals that our seniors perceive Tusculum to be behind its peers in the area of senior theses and culminating projects.
•
Finally, results from the Noel-Levitz Student Satisfaction Inventory (2008-2009) suggest our students want more intellectual growth, a campus committed to academic excellence, and good classroom instruction.
Our plan The quality of our thinking is believed to progress across time. To illustrate, children believe in Santa Claus and often uncritically accept the words of those in authority. Adults tend to question information they are given. Our plan will harness this developmental cognitive process and maturation. In the first phase of the plan, students will hone their reasoning skills by encountering course-specific problems posed to them in certain RJ-enriched classes. These courses will incorporate (a) problem-based learning, (b) case studies, and/or (c) ethical dilemmas into instruction. In the second phase of the plan the scaffolding and props will be lessened and students will apply these same reasoning skills to real problems outside the classroom. Our plan to develop students’ ability to solve problems with Reflective Judgment will unfold from “the classroom to the world.” In year one of our plan (2010-2011) students will receive RJ-enriched instruction in the following courses: Orientation Various Commons Social Science Courses ENGL 111: Composition, Research and Rhetoric EVSC 111: Environmental Science
In year two of our plan (2011-2012) we will build upon our initial efforts by adding RJ-enriched instruction to the following courses: Residential CMNS 251: Theory and Practice of Citizenship
GPS EDUC 200: History, Philosophy and Principles of Education MGMT 232: Foundation in Management Skills
Concurrently, Residential students will encounter an enriched co-curricular experience in which Resident Advisors are developed as Reflective Judgment peer-mentors and infuse residence life with periodic discussion about solving problems using Reflective Judgment. The Arts and Lecture series may also present another occasional venue for co-curricular Reflective Judgment opportunities.
Problems
efine d situations thought of as ill-d be y ma s” em bl ro “P mation, which lack essential infor or dilemmas which y, about complete certaint th wi d lve so re cannot be may disagree, an d ople an d experts pe le ab on as re which resolution (King sort of solution or me so ire qu re ich wh ). & Kitchener, 1994 of facts. But textbo oks are full r Ou is: th er id Cons me regarded as tion in them beca ma or inf e th re fo be blems” the discipline’s “pro “fact,” these were d experts an d questions. An dilemmas, puzzles dust in a an d kicke d up the ed gu ar , ed ch ar rese consensus settle d nt process before me dg Ju ive ct fle Re blems” is how king through “pro or W . er sw an around an e. our fields advanc
In the second phase of our plan, our attention will turn to developing reflective judgment by solving problems in the world. There are many different situations that might be considered ill-structured problems. For instance, prioritizing one’s time, making wise parenting choices, and planning a kitchen renovation all involve biased claims and mercurial data, culminating into more and less logical decisions.
In light of the purposes of higher education and the unique civic arts character of Tusculum College, we have decided to focus on three different types of problems: (a) problems that exist because of a lack of information, requiring research or a creative project to solve, (b) social problems, requiring practical wisdom and action and (c) problems in the workplace, which can be addressed through an internship. Toward this end, our plan anticipates beginning conversations across the campus and working with the appropriate committees in the fall of 2011 toward the goal of modifying program requirements to include (a) independent research or a creative project, (b) a program-specific service learning course, or (c) an internship that involves solving a work problem with RJ. Consider that we are already doing some of this... A few people would say modifying our programs so that students can engage in solving actual problems through RJ might be too great an undertaking for this little institution. Presently, they might be correct; however, we are already doing some of this. And to ensure that this becomes a realizable goal for all Tusculum departments and students, our plan has a strong emphasis on faculty development during a May Institute, ongoing support for faculty through Teaching Circles, and symposia, with online dialogue and consultation to fit the schedules of adjunct faculty. Additionally, the resources of Center for Civic Advancement, the Hobbie Center, and the Career Services Director will be enlisted to assist faculty as departments embark upon these proposed changes. In years three through five of our plan (2012-2015) we propose to build upon phase one by modifying program requirements so that they include either (a) independent research or a creative project, (b) participation in a program-specific, service learning project emphasizing the use of RJ, or (c) an internship in which an ill-structured problem is identified, researched, and resolved.
Practical wisdom is derived from the Greek “phronesis,” which is, roughly, the ability to look at a problem situation, consider information, and determine a course of action that does everyone some good, AND the ability to be smart about executing it.
Resources Mindful of this present challenging economic environment, Tusculum College nonetheless believes strongly in the importance of student learning, and so has allocated funds for this new initiative. Tusculum anticipates spending approximately $300,000 across the five year horizon of his plan. This figure, which is generally in line with other institutions our size, will support course releases for the QEP Director, the May Institute development opportunity, RJ resources, assessment supplies, and to promote curricular and co-curricular student activities which involve RJ. Although the execution of the plan does not depend upon external grants, such grants will be sought when available in order to expand this effort to improve student learning and sustain it beyond the capacities of Tusculum’s budget. Tusculum is committed to student learning and enabling students to solve problems through Reflective Judgment.
However, our most important resource is not financial, but human. Tusculum has dynamic and effective people across the campus. We appreciate them for their expertise and contributions. Our proposal to substantively change learning at Tusculum is multi-faceted. To meet this challenge, your colleagues and friends below have agreed to serve on the QEP Implementation Team (QEP-IT), working as a team, but with certain areas of emphasis.
Focus: Implementation and Evaluation
•
Bill Garris: QEP Director
•
Tony Narkawicz: (Director of Institutional Research): RJ Assessment
•
Jeff Lokey:
•
Suzanne Richey (Director of Communications): Publicity
•
Melinda Dukes (Associate Vice President): Liaison with Provost’s Office
(Director of Commons Program): Integration into Commons
Focus: Faculty Development and Course Re-design
•
Michelle Freeman (Teaching and Learning Circle Coordinator): RJ Teaching Circles
•
Angela Keaton (Hobbie Center Director): RJ and Service Learning
•
Amanda Waddell (Career Services Director: RJ and the Workplace
•
Marsha Griffith (Assistant Professor of Library Science) RJ, Information Literacy and GPS
•
Corinne Nicolas: (Associate Professor of English): RJ and GPS Student Development and Co-curricular
•
Jonita Ashley-Pauley (Associate Dean of Students): QEP and Student Life
•
Stephanie Rhea: (Senior Student Life Coordinator): RJ in the Residence Halls
Multiple measures, multiple methods; gauging progress and informing the plan Tusculum College will use numerous measures to both evaluate the plan’s effectiveness and guide its progression. First, in early 2010, syllabi from courses identified on page 3 will be collected and reviewed by the QEP Implementation Team (QEP-IT) for evidence that the course design promotes RJ. Then in the winter of 2011, following our first efforts at infusing RJ into coursework, the QEP-IT will examine syllabi for the same courses and note whether or not the re-designed courses appear to promote RJ. The QEP-IT will discuss their findings and consider if changes need to be made to year two of the plan. Second, courses identified on page 3 will assess students’ RJ ability through a new rubric starting in the fall of 2010. Data will be collected each semester and during the winter of each year, and the QEP-IT will interpret the data and modify the plan as needed. These first two assessments will provide timely feedback on our efforts. Third, our sophomores will take the CAAP test of critical thinking in 2012 and again, as seniors, in 2014. This will enable us to compare gains made with this cohort to the gains made on this test by the cohort which took this test as sophomores in 2004-5 and as seniors in 2006-07. Independent of this reference group, we will also be able to note the changes in critical thinking affected by our modified pedagogy and focus on RJ. Whereas our previous CAAP testing failed to show an improvement in the percentile rank scores of our students, we would like to see a change of 15 points in our students’ percentile rank between their sophomore and senior years on account of our intensive efforts. The QEP-IT will consider the results of this test as they become available and make changes to the plan as warranted. Fourth, beginning in 2010, our students will take the Reasoning about Current Issues test (RCI) as freshmen during block one, and during their junior and senior years at Tusculum. Currently our baseline testing using the RCI shows that scores by class vary from 4.26 to 4.55, and that there is only a weak positive trend as people advance from their first to final year at Tusculum. Our goal is that by 2012-13 our juniors will score 20% higher than our freshmen and by 2013-14 our seniors will score 30% higher than our freshmen. The QEP-IT will review the data as they become available and modify the plan as needed. The Office of Institutional Research will continue to administer the National Survey of Student Engagement (NSSE) and the Noel-Levitz Student Satisfaction Index. The QEP-IT will consider this information and what it says about the plan’s effectiveness as increasing student engagement particularly with culminating senior projects and the broader academic culture on campus. In short, the plan intends to facilitate improvement in students’ Reflective Judgment skills, and this will be measured through rubrics, the standard test of Reflective Judgment (the RCI), tests of critical thinking (as a proxy), and other measures of the campus culture. The results will guide the unfolding of the plan and also assess our effectiveness.
Graphic Depicting the Plan Phase I: Problem Solving with Reflective Judgment In the Classroom • Course level implementation • Focus on courses within the Commons • Residential Advisors and other co-curricular events enrich life outside of the classroom for Residential students • Emphasis on both Residential and Gateway courses May Institute: intensive workshops
• Focus on faculty development
Teaching and learning circles: ongoing support Reflective Judgment Resource Center Online readings, discussion, consultation (GPS)
Phase II:
Problem Solving with Reflective Judgment In the World
• Proposed program level implementation • Focus on infusing Tusculum resources (CCA, Hobbie Center, Career Services) more richly into programs, and using resource leads to consult more with programs • High level of coordination to insure students practice solving real-world problems with Reflective Judgment
Internships
Research / Creative Project
Aided by Career Services (Amanda Waddell)
QEP Director (Bill Garris)
Service Learning as Problem Solving Aided by the Hobbie Center and CCA (Angela Keaton)
Terms and Definitions as Used in our Plan If you cannot - in the long run - tell everyone what you have been doing, your doing has been worthless. ~ Erwin Schrodinger
Deductive Reasoning: A type of logic in which conclusions must follow from the premises. A sound deductive argument has true premises AND the conclusion follows from these premises.
Epistemology: The philosophical discourse about the nature of knowledge, what is truth, and how one comes to know what is true.
Inductive Reasoning: A type of thinking in which examines data or makes observations and then infers a conclusion or principle from the data. Inductive arguments are described as strong or weak, depending upon the quality of the conclusion’s connection to the previous data.
Practical Wisdom: One of Tusculum’s distinctive values, this refers to the ability to reflect upon a situation, determine what a good action or solution for all would be, and then to act smartly or wisely in order to carry out that action or solution.
Problems: Ill-defined situations and/or dilemmas which lack essential information, which cannot be resolved with complete certainty, about which reasonable people and experts may disagree, and which require some sort of solution or resolution (King & Kitchener, 1994).
Reflective Judgment: This is a type of thinking in which one recognizes the limits (uncertainty) of knowledge and yet constructs a reasonable knowledge claim or solves a problem with a defensible solution, all while holding this view tentatively and recognizing that with better tools or a different perspective that the “truth” of the situation may change. In short this is a critical thinking skill that says, “yes, maybe we will never know THE truth, but I can assemble the better information and put together a strong argument for ____”
Reflective Judgment Model: A model that describes developmental changes in our thinking, particularly about the nature of knowledge (epistemology). This model posits that our understanding about the nature of knowledge progresses through three broad developmental stages outlined below.
Pre-reflective: The truth is out there. It is self-evident (“everyone knows that...”) or authorities have the truth. Quasi-reflective or relativism: No one can really know the truth. Scientific inquiry is flawed, and everyone has an opinion. It is not really possible to know. Therefore, all positions are equal.
Reflective: In this stage it is recognized that while absolute truth might be beyond our grasp, it is possible to gather good evidence and put together an argument or solution. This is sometimes referred to as “commitment in relativism.”
... sounds like critical thinking to me ...sounds like information literacy to me
Overlap and Distinctiveness
References Association of American Colleges and Universities (2008). Our student’s best work: A framework for accountability worthy of our mission. Retrieved July 1, 2009 from http://www.aacu.org/publications/ pdfs.StudentsBestreport.pdf King, P., & Kitchener, K. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey Bass.