Page 1

Evaluating Readiness in Your Online Course

by Barbara J. Keinath, Ph.D., Metropolitan State University and Lesley A. Blicker, M.M.A, Metropolitan State University

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric.html.


Evaluating Readiness in Your Online Course

Lesley Blicker, M.M.A., Metropolitan State University Barbara J. Keinath, Ph.D., Metropolitan State University A Bit of History • At Metropolitan State University, the College of Management’s approach to implementing its Emerging Curriculum grants called for us to determine if online course sites were “student-ready.” • Student-ready was what we called a WebCT course site that would provide good structure for learning in an online course. • The definition of “student-ready” gelled over time into the rubric. At first, we used the rubric to expedite feedback to instructors and ensure consistency in our site review. • We now provide the rubric to instructors as a guide for their site development. • We use the rubric across the University for all courses developed within Teaching Online training. • In the future, we expect to use the rubric to encourage continuous improvement of course sites, thus steadily improving quality in online courses. Definitions • Rubric—as used here, a structured evaluation tool that enables reviewers to provide feedback efficiently and consistently. (According to Oxford English Dictionary (OED Online) “a descriptive heading or title; a designation, category. Also, an injunction, a general rule”) • Continuous Improvement—never-ending improvement; can be incremental or breakthrough types of changes that improve student learning and satisfaction with the learning process • Quality o Fitness for intended use (user-defined criteria) o Conforming to specifications (manufacturing-based definition) o Meeting or exceeding expectations (customer-driven definition) ©Barbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

2


Clarification/Assumptions • The rubric is intended to guide review of an online course site, not the content of an online course. • The structure of an online course and the site navigation are the focus of the rubric. An Observation • Although developed independently, the Metropolitan State rubric is highly congruent with other models for assessing and assuring quality in online courses. • Common elements: o Clear learning path for students o Learning community o Student to instructor interaction o Variety in assignment types and evaluation methods o Feedback to students o Availability of learning resources o Appealing appearance o Efficient and transparent site navigation Some Uses for the Rubric • Guide instructors as they develop online courses • Serve as a development checklist • Provide feedback to course developers • Determine site readiness and usability • Suggest quality standards for an institution’s online courses • Provide a method for peer review of online course sites • Encourage continuous improvement of courses Please direct questions, comments or inquiries to:

barbara.keinath@metrostate.edu lesley.blicker@metrostate.edu To view rubric online, please visit: http://www.metrostate.edu/col/rubric_2.pdf

©Barbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

3


Evaluating Readiness in Your Online Course

by Barbara J. Keinath, Ph.D., Metropolitan State University and Lesley A. Blicker, M.M.A, Metropolitan State University

The Practice Review Elements: 1. 2. 3. 4. 5. 6.

Course Design Assessment of Student Performance Learner Support/Feedback Learning Community Pedagogical Practices Use of Technology

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

4


Course Element

Course Design

Framework

Course Information and Objectives

Context for Course Content

Not Student-Ready

StudentReady

Student-Ready With Modification

Framework for course delivery is apparent. Course modules are identified, described, & connected. Progression through the course is apparent. A format for progression through modules from introduction to evaluation is provided.

Some evidence of a framework and modules are apparent. Beginnings of course progression is apparent. Format for progression through modules from introduction to evaluation is in development.

All necessary information is apparent, easily located, and clearly stated. This includes a course syllabus that contains: a course description, the course requirements, the course/learning objectives, a course grading or student assessment scheme, dates of the course, course policies, and instructor information.

Site contains most necessary information although items may not be as easily located or clearly stated as they could be.

Site is missing or lacking in critical information. Items are not easily found or stated clearly.

Limited context is provided for course content. Content format may be somewhat inconsistent, requiring students to interpret a new approach in each module.

Site provides no context for course content. Students left to rely mainly on syllabus and textbook only.

Site provides context for course content in a progressive manner in clearly identified/manageable segments or modules. Content format is consistent from segment to segment.

Comments

Framework for course delivery is not apparent. Course modules are not outlined. How students progress through modules from introduction to evaluation is less apparent or unclear.

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

5


Course Design

Course Element

StudentReady

Student-Ready With Modification

Not Student-Ready

Appearance and Presentation of Material

Site’s appearance is appealing. A high amount of work has been completed as evidenced by minimal scrolling, placement of important information at the top of pages, appropriate use of white space on a web page, consistent style of icons throughout the site, and a color scheme that is pleasant and easy on the eyes.

Site’s appearance and layout is mostly appealing due to presence of some but not all of the following: minimal scrolling, placement of important information at the top of pages, appropriate use of white space on a web page, consistent style of icons throughout site, and color scheme that is easy on the eyes.

Navigation

Site navigation is transparent to students. Web pages are logically linked for both novice and experienced users. Instructions provided to orient the user. Ample navigational aids provided to move from page to page. Ample instructions provided for the uses of various elements in course site. Content can be provided in formats accessible to persons with disabilities. Students with disabilities can be accommodated according to their disability through the availability of alternative delivery methods in the online context.

Site navigation is mostly transparent and logical to students. Instructions are provided to orient the user. Links and movement between web pages is mostly logical or obvious to user. Some instructions about uses of elements of course and navigational aids have been made available. Site contains some evidence of efforts taken to accommodate a variety of disabilities through alternative delivery methods, if needed.

Site’s appearance and layout need work due to any combination of: lack of minimal scrolling, lack of using tops of pages for placement of important information, lack of sufficient white space on a web page, inconsistent style of icons throughout site, and poor choice of color schemes, or one that is hard on the eyes. Site navigation is not transparent. Items appear to have different names in different places. There is no apparent site orientation. Few or no clues regarding using various elements of course site have been provided. Pages are not necessarily linked in logical fashion.

Accessibility

Comments

Site contains little or no mention of accommodating students with disabilities or little or no evidence of availability of alternative methods of delivery. Students with disabilities are not accommodated in the online course.

©Barbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

6


Course Element

Assessment of Student Performance

Type and variety of assessments used

Assignments

Quizzes/Exams

StudentReady

Student-Ready With Modification

Not Student-Ready

Evidence of the use of assessment techniques which are provided in various formats so as to address differences in learning styles. Assessments are congruent with learning objectives and activities (learning activities support student achievement). How grades on individual assessments translate to a course grade is clear. Site contains evidence of active or student-directed learning via assignments. Expectations and due dates are clearly defined. Assignments support learning objectives.

Some evidence of the use of assessment techniques which are provided in various formats so as to address differences in learning styles. Assessments may not be completely congruent with learning objectives and activities. How grades on individual assessments translate to course grade is not clear.

Assignments and/or assessments appear to be all of a similar nature; they may not reach out to differences in learning styles. How grades on individual assignments translate to course grade is not clear.

Site contains some evidence of active or student-directed learning via assignments. Expectations and due dates mostly clearly defined. Assignments may or may not support learning objectives.

Site contains little or no evidence of active or student-directed learning via assignments. It is not clearly evident that assignments support learning objectives. Expectations are unclear and due dates may or may not be present.

Course used a modest amount of varied and frequent selfassessment methods to guide learners. Quizzes or exams mostly are congruent with learning objectives.

Course uses little to none varied and frequent self-assessment methods to guide learners. Quizzes or exams are not congruent with learning objectives.

Course uses varied and frequent self-assessment methods to guide learners. Quizzes or exams are congruent with learning objectives.

Comments

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

7


Course Element

Learner Support/Feedback

Feedback Methods

Student Grades

Learning Support/ Resources

StudentReady

Student-Ready With Modification

Not Student-Ready

Site is set up to allow instructor to provide frequent and timely feedback to students throughout the semester. Feedback methods include: email and/or use of private discussion areas; provision of feedback in assignment and quiz tools; acknowledgement of receipt of assignments. Feedback is individualized with sufficient detail to be useful to the student. The Student Grade tool is utilized so that students may track their own course achievement at any time. Grade book entries are consistent with the syllabus. Grades can be entered in a timely fashion. A method is available for students to determine their own midterm grade equivalency.

Multiple methods for provision of feedback may be available, but feedback could be more individualized and detailed so as to be of greater help to the student. Feedback is not always prompt.

Site is absent of timely feedback methods. Site contains no evidence of multiple methods for providing feedback. Feedback is not individualized or detailed enough to be useful to student.

The Student Grade tool is utilized. Grade book entries are not consistent with the syllabus. Grades are not always entered in a timely fashion. Students are not entirely able to determine their mid-term grade equivalency through online grade book. Site contains evidence of some resources being made available, but they are not yet fully fleshed out or made readily accessible.

The Student Grade tool has not been set up or is not utilized well enough for students to track their own achievement at any time.

Students are provided a full set of resources to support their online learning experience, which are easily accessible in site. These may include: technical services and help desk services and tutorials on the course software, access to student services including library services, writing assistance, computer lab assistance, etc.

Comments

Site lacks learning support resources.

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

8


Learning Community

Course Element

StudentReady

Student-Ready With Modification

Not Student-Ready

Interaction and Participation

Variety of avenues for establishing a sense of community among learners has been incorporated into the overall design of the course. Several opportunities are or can be provided for students to engage in collaborative learning activities. Tools such as discussion board, chat rooms, student presentations, group activities, and student home pages are well-utilized for effective student-to-student learning activities. Expectations regarding student interaction in the course are clearly defined.

Modest to moderate avenues for establishing a sense of community among learners has been incorporated into the overall design of the course. Opportunities for students to engage in collaborative learning activities are limited to case studies. Tools such as discussion board, chat rooms, student presentations, group activities, and student home pages have not been fully utilized for effective student-tostudent learning activities. Expectations regarding student interaction in the course have not been fully defined.

Few to no avenues for establishing a sense of community among learners are incorporated into the overall design of the course. Site is absent of opportunities for students to engage in collaborative learning activities. Tools such as discussion board, chat rooms, student presentations, group activities, and student home pages have not been fully utilized for effective studentto-student learning activities. Expectations regarding student interaction in the course have not been fully defined.

Comments

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

9


Learning Community

Diversity

Site is appropriate for English-language learners, free from cultural bias, and minimizes use of idioms. Where visuals are used, they are diverse in nature and not restricted to any single cultural group, unless warranted by specific content. Icons used are universal and easily interpreted.

The site contains some evidence of efforts to use culturally unbiased language and language that is appropriate for English-language learners. Some visuals are diverse in nature and most icons are universal and easily interpreted.

Site may contain evidence of several or all of the following: text reflects some cultural bias, text is consistently too advanced for Englishlanguage learners and contains numerous culturally specific idioms. Visuals reflect a single cultural group.

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

10


Course Element

Pedagogical Practices

Instructional Design/Activities

StudentReady

Student-Ready With Modification

Not Student-Ready

Instructional design facilitates active or student-centered learning. Students are or can be given opportunities for practice, reflection, and engagement with the course content. Site has a presence of innovative approaches to delivering content. Pedagogy is focused both on content and the learning process. Teaching practices allow for experiential applications from students’ lives. Teaching strategies are congruent with learning objectives.

Instructional design facilitates limited active or studentcentered learning. Students are or can be given modest to moderate opportunities for practice, reflection, and engagement with the course content. Site has a limited presence of innovative approaches to delivering content. Pedagogy is focused more on content than on the learning process. Teaching practices allow for a modest amount of experiential applications from students’ lives. Teaching strategies are mostly congruent with learning objectives, but limited in variety.

Instructional design does not facilitate active or student-centered learning. Students are given few to no opportunities for practice, reflection, and engagement with the course content. Site is absent of innovative approaches to delivering content. Pedagogy focus is unclear. Teaching practices do not allow for experiential applications from students’ lives. Teaching strategies are not particularly congruent with learning objectives.

Comments

©Barbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

11


Course Element

Use of Technology

Use of inherent web course tools

Use of Instructional Media

StudentReady Course maximizes the use of web course tools available, in support of the learning objectives, possibly including: email, discussion board, calendar, grade book, assignment area, student presentations, chat rooms, whiteboard, online note-taking, links, self-tests, quizzes, surveys, and references. Tools implemented are used effectively. Graphics and media used have clear instructional purposes which reinforce content. They require little download time and are accessible in all standard browsers. Media content not accessible in all browsers is made available in other formats. Audio and video files do not require excessive download time. Resource links are made available for students to test their software and to download necessary plug-ins.

Student-Ready With Modification

Course makes partial use of the web course tools available, in support of the learning objectives. For example, the course may have made effective use of email, the discussion board, the grade book, quizzes and self-tests. However, students could benefit from the implementation of additional tools such as the assignment area, student presentations, chat rooms, whiteboard, online note-taking, links, quizzes, surveys, and references. What has been implemented is used effectively. Graphics and media probably have clear instructional purposes which reinforce content, however files either require greater download time or are not accessible in all standard browsers. Additionally, media content, while effective when viewed, is not made available in other formats for browsers unable to open the file. Resource links are made available for students to test their software and to download necessary plug-ins.

Not Student-Ready

Comments

Course makes use of the most basic web course tools such as the grade book, email, and discussion board. Discussion board may not be used as effectively as it could be. Other tools are not incorporated.

Graphics and media used do not have clear instructional purposes to reinforce the content. They require a fair amount of download time and are not accessible in all standard browsers. Media content is not made available in other formats. Resource links are not made available for students to test their software and to download necessary plug-ins.

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

12


Reviewer’s General Comments:

ŠBarbara J. Keinath, Ph.D. and Lesley Blicker, M.M.A, Metropolitan State University, Minneapolis, Minnesota, 2003. Rubric can be accessed at: http://www.metrostate.edu/col/rubric_2.pdf. Revised 11.10.03

13

Site Readiness Rubric  

Site readiness rubric

Read more
Read more
Similar to
Popular now
Just for you