Perinpasingam, T.(2006). An evaluation of a website to be used as an Instructional Tool in an English Language Classroom.Presentation of paper at Taylor’s 1st Teaching and Learning Conference held on the 2nd September, 2006.
Abstract:
As students find different learning resources on the Internet, they become active participants in their quest for knowledge. Incorporating Internet in the classroom provides students with more opportunities to structure their own learning. The Internet also engages students in interactive learning. This paper reports the results of the evaluation of a selected website used as an instructional tool in an English Language classroom. The evaluation process was divided into formative evaluation which involved two language lecturers and two instructional technologists. Later a summative evaluation was carried out among a group of five students. Three main elements for the evaluation were based on Kristof and Satron (1995), who suggested that an interactive learning system can be separated into information design, interactive design and presentation design. Specifically, the findings suggest that the content of the selected website was engaging and well organized. The exercises were interactive and enhanced inquiry oriented learning. Furthermore, the students also mentioned that it encouraged interaction among their peers as well as their lecturer. It may be concluded that this selected free web- based lesson is suitable to be used as an instructional tool in an English language classroom. Introduction: Technology can be integrated into various teaching styles. According to Gagne (1985) teaches are suitable human models to stimulate alteration in attitudes. For these reasons, the web based learning provides students and teachers various opportunities to make language learning more effective and enjoyable. The purpose of this study is to evaluate whether a web based lesson in an English language classroom will offer comprehensible information in multimedia form designed to fit the learning to cater for different individual learning styles. Furthermore, it is also the mode of communication beyond the computer screens. In addition, it is a form of interactive learning with meaningful feedback and guidance. Research Question: Specifically, this study aims to answer the following question. 1. What is the criteria involved in evaluating a website to be used as an instructional tool in an English language classroom? Significance of the study: This study was to enact the introduction of web-based lesson as a tool to enhance interactive teaching and learning particularly in Malaysian classrooms. It was able to offer helpful tips to determine which facets and traits that was important in evaluating a teaching and learning material. Furthermore, it also offered another alternative or option
to teachers so that these educators do not solely depend on printed materials and CDROMs, which are inflexible and costly.
Limitation of the study: Since the study is confined to a certain group of the students, their views may not be generated to other student’s views. Related Literature Review: The World Wide Web is known to promote new learning resources and opportunities in a classroom, allowing the teachers and students to explore the world of knowledge with a click of the mouse. According to Owston (1997), one of the major benefits of using the web for teaching and learning is that it favored student centered learning as opposed to the traditional teacher centered learning. The two issues discussed here are the effectiveness of online teaching as well as roles played by a web educator. Below are five “I� methods of effective online teaching, suggested by Don Olcott (1999). a).Interaction occurs when there are communication between the students and the instructor, students with other students and also interaction between the students and the contents of the course. The interaction is present here because the presentation of print material is linked to the Internet for interaction purpose. Therefore the interaction can help improve their performance, due to depth of discussion conducted. b). Introspection refers to interpretation, revision and demonstrating the understanding of concepts. E-mails, graphic even audio and video can be effective technologies to encourage introspection. c). Innovation refers to integration of facts, concepts, theories and practical application of knowledge. d). Integration like using interactive quizzes, print exercises and role-play can create a setting in which integration can occur. These collaborative activities can easily be accomplished in an asynchronous environment. e). Information refers to the knowledge and understanding that is a pre-requisite for students to move to the next level of learning. He further adds that instructors should design courses with learning objectives in mind, building learning activities that allow students to reach those objectives. The technological tools and instructional module should be chosen to match the objectives in order to achieve the learning outcomes.
“Any technology can disseminate information, the instructor must create interactive opportunities and access whether students have the basic knowledge to move forward to the next level of learning.” (Olcott,1999) In conclusion, Olcott’s discussion makes it clear that online teaching can be used to deliver a course which can improve student’s performances because it allows learning from various forms of media such as images, interactive learning tools like interactive tools as well as communicative tools like email and forum. These features of online learning or distance learning can help improve retention of information, increase motivation through sharing of information which leads to higher order thinking.
The Role of a Web Educator: In the 21st century classroom, the role of the teacher is to facilitate learning towards the vast information available and help them develop their skills in critical thinking, problem solving and decision making. According to Jonassen (1996), an online learning course improves student’s performance and supports two way communications, not only among the teachers but also among their peers. This form of communication helps increase interest and motivation among learners. Hence the web based educators can take on several roles like facilitators guide, mentor or coach. The creation of a new learning environment for the web base learning allows hands on facilitation while students are learning. Evaluation of a module: Evaluation may answer two complementary roles. In one context the aim is anticipating or formative which is to improve. According to Bhola (1990), a formative evaluation is conducted typically for development or improvement of a program or product. On the contrary, summative evaluation is to assess concrete achievement also seen as a process of acknowledgment. This form of evaluation views more than one learner’s performance. It also reveals how well a group did on learning task that practice specific learning materials. According to Barba (1998), formative evaluation techniques include one to one testing, small group testing and pilot study. The success of the lesson depends well on its design and structure. It is also important to include a clear objective and learning outcome with various sub topics. Evaluation was done in order to ascertain the effectiveness of this web-based lesson to support interactive teaching and learning which will shift the focus of teaching and learning towards student centered learning.
Hence a semi-structured interview was conducted with two educational technologist, two lecturers and five students focusing on the areas suggested by Kristof and Satron (1995) which covered information design, interaction design and presentation design. Information design: It covered the objective and the learning outcome of the lesson, analysis of the content which covered areas like structure and organization of information. Furthermore, it also included the types of instruction given whether it was clear or simple. Interaction Design: This area focused on the navigation activities like scrolling of bars, navigation bars and hypertext. Apart from that, it focused also on communicative tools like emails, discussion board or bulletin boards. The features which were also highlighted are links, glossary and picture gallery. Presentation Design: It covered screen layout, background colour, font size also use of graphics and animations.
Research Methodology: The evaluation of the web based lesson involved a formative evaluation and a summative evaluation. A formative evaluation was conducted by interviewing two educational technologist and two English lecturers. The interview questions were derived from three elements of design in interactive learning suggested by Kristof and Satron (1995). The three areas were information design, interactive design and presentation design. Later, a summative evaluation was conducted by interviewing a group of five students. The semi-structured question are shown in Appendix A, B and C. The respondents were probed to provide more detailed feedback about the web based lesson during the interview. Instrumentation and Data Collection Procedure: In the formative evaluation process, semi- structured interview and observation techniques were used to gather data. The semi-structured interview schedules are shown in Appendix A, B and C. The Instructional Materials: A free selected website was used as an instructional tool to teach English among Diploma students. The main reason for selecting this website was to diversify the teaching and learning process. Hence, the teaching and learning process changed from a teacher centered oriented to a learner centered oriented.
Evaluation Procedure: For the formative evaluation two educational technologists who are instructional designers were interviewed, followed by the interview of two English lecturers who were subject matter experts. They were briefed about the selected website prior to the interview. For the summative evaluation, students were guided to work with the selected website lesson. They were also asked to complete a task and later were interviewed. The interview questions for the educational technologists can be seen in Appendix A, where else the interview questions for the English lectures and the students are available in Appendix B and C respectively. Data Analysis: The evaluation was based on Kristof and Satron’s (1995) interactive learning system. The three elements were information design, interactive design and presentation design. Two experts in instructional design and two experts in the content did the formative evaluation of this selected website. The two experts in instructional design had the knowledge and skills in instructional design. They evaluated the instructional structure, interaction, (navigation and usability) and presentation aspects. The content experts were the two English language lecturers. They evaluated the content or information design. They are English lecturers who have knowledge, pedagogical and curricular aspects in teaching and learning English at the tertiary level. They evaluated the information aspects, which emphasized the objective and learning outcome, content delivery, activities or exercises and also language. Table 4.1: Responses from the educational technologists about the instructional structure Expert Comment Rizal
·
Azura
· · ·
Suitable and easy to understand No learning outcomes Clearly written but no learning outcomes Structured and the information is chunked accordingly
The two educational technologists mentioned that the instructions were well structured, clear and easily understood also suitable for the targeted learners. One of the experts mentioned that the information in this module was chunked accordingly, making it easy for the learners to understand the lesson. On the contrary, both of them highlighted that no learning outcomes were included.
Table 4.2: Responses from the instructional technologist on navigation Expert Comment Rizal · Navigation is easy and user friendly Azura · Navigation is easy because the links are available on the top and bottom of the page. Both of them agreed that they did not have any difficulties with the navigation because it was easy and user friendly. On the other hand, another respondent mentioned that the links were available at the top and bottom of the page. Therefore, it saved scrolling time. Rizal: “I did not have any problems with the navigations. It is easy and user friendly”. Table 4.3: Responses from the educational technologist on usability. Expert Comment Rizal · Critical reading activity encourages higher order thinking. It is relevant. · Advice page is appropriate. Azura · Relevant and appropriate for the age group. As mentioned above, (table 4.3), both of them agreed that this module was suitable for the targeted learners. One of them mentioned that the “Critical Reading” section encouraged higher order thinking among students because it supported inferencing. Hence it helped the learning process by further enhancing their knowledge about different reading styles. “Advice” section was very useful because it enabled the students to comprehend different types of reading styles also to further explore the topic. Table 4.4: Responses from the educational technologist on the presentation. Expert Comment Rizal · Neat screen design · Suitable background colour · Font type is clear and easy to read · No animations are used. Azura · Suitable screen design · Background colour appropriate · Font type is fine and easy to read · No animations available.
Based on the feedback received from the two respondents as in table 4.4, it revealed that the screen design was neat, organized and suitable. Hence it was able to attract the students to explore this selected website. The background color was suitable. The type of font used was appropriate and easily read. Hence it was suitable for the targeted learners. On the contrary, there were no animations inserted to attract the attention of the viewers. Rizal: “The screen design is appropriate because it is neat and organized. The background colour is suitable and limited to two to three colours. The font size is suitable. No animations are used, I think a few can be inserted to attract the viewers”. Table 4.5: Responses from the English lecturers on information design Expert Comment Azmi · Objectives are clear and relevant. · Instructions are good and clear. · Content is suitable. · Exercises are relevant and provides immediate feedback. . Language is simple and clear. Kim · Objectives are clear. · Instructions are simple and easy to understand. · Content is relevant · Exercises are suitable but better if structured according to proficiency level. Interactive with immediate feedback. · Language is simple and direct. The findings shown in table 4.5 revealed that the content experts found that the objectives of the web based lesson were clearly stated and related to the curriculum. The content of the lesson was suitable for the targeted learners. The instructions were simple and direct. The language used was simple and easy. The exercises were engaging and it provided immediate feedback. However, the exercises can be further structured according to different proficiency level. Kim: “The students will definitely enjoy the lesson because the instructions are simple and clear. The activities are suitable but it would better if it is structured according to different proficiency level. Since it is interactive learning approach, I believe that it will capture the student’s interest because it allows for immediate feedback.
Five Diploma students were involved in the summative evaluation. The findings were divided into three different sections. The first section was information design, which included the objectives and learning outcomes, content and learning materials, language and exercise or activities. The second section focused on interaction design, which included interaction features like the links related to reading such as advice, efficient reading skills, strategies for reading academic reading texts and critical reading, third and the final section involve the presentation design which focuses on screen design, background color, font, graphics as well as animation. Table 4.6: Responses from the students on the objectives and learning outcomes. Student Comment Siti · Objectives are clear and easy to understand. No learning outcomes Vincent · Objectives are suitable and direct. No learning outcomes. Lee · Objectives are clear and it help me to focus on the specific topics. Learning outcomes not available. Keshav Anita
· Objectives are well organized. No learning outcomes. · Objectives are suitable and it is helpful to further explore the lesson. Learning outcomes not available.
Based on the information obtained from the students (table 4.6), the objectives were clearly written, easily understood and suitable for Diploma students. Two students agreed that the objectives helped them to be focused. However all five respondents agreed that no learning outcomes were available. Lee: “The objectives are clearly stated and it enable me to stay focus on specific topics. But no learning outcomes available.”
Table: 4.7 Responses from the students on the content and learning Materials. Student Comment Siti · Clear and easy to understand · Simple language used and the instructions are easy to follow Vincent Lee
Keshav Anita
· Factual and easy to understand · Simple instructions to follow · Content is engaging and interactive · Instructions are straightforward · Content is clear and structured. · Content is simple and easy to follow · Instructions were direct
As revealed in (table 4.7), the students agreed that the content was clear, easily understood and engaging. Furthermore, the students also mentioned that the instructions given were simple and easily understood because the language used in this lesson were simple and direct. Anita: The content is simple and easy to follow. Apart from that the instructions are direct”. Table 4.8:Responses from the students on the activities Student Comment Siti · Simple, interesting and fun Vincent · Fun and direct. Lee · Exciting and challenging Keshav · Interesting and the answers can be obtained immediately and assess their performance Anita · Exciting and interactive. Based on the feedback received from the students, the activities in this website were interactive, challenging and fun. One of the student also mentioned that the exercises allowed for immediate feedback. Hence, this website had changed the teaching and learning process towards student centeredness. Keshav : “ The activities are interesting and fun. In addition the answers can be obtained immediately. Furthermore one can assess their performance”.
Table 4.9: Responses from the students on navigation Student Comment Siti · Easy because I’m familiar with the Links available Vincent · Easy to use Lee · Easy to use. Links are available at the top and bottom of the screen. Keshav · Easy to use Anita · The navigation buttons are user Friendly Based on the feedback received from the students (table 4.9), all the students mentioned that they did not have any difficulties with the navigations. On the whole the navigations were easy and user friendly. One of the students mentioned that the links were available at the top and bottom of the page. Hence, it reduced scrolling time. Table 4.10: Responses from the students on Advice Student Comment Siti · Introduced the information in · an orderly manner. Vincent
·
Lee Keshav
· ·
Anita
·
Able to identify with terms or concepts introduced interesting and useful Engaging and information is chucked accordingly Interesting and easily Understood
Feedback obtained from the students (table 4.10), were very encouraging because all the students agreed that it was very interactive and engaging way to learn reading skills. One student mentioned that he was able to identify with the terms or concepts introduced. On the whole, the students enjoyed learning using this form of media. Lee: “It is engaging and the information is chucked accordingly”.
Table 4.11: Responses from the students on Efficient reading skills. Student Comment Siti · Simple and well structured Vincent · Exercises are relevant and helpful Lee · Immediate feedback for answers Keshav · Helpful, suitable and simple Anita · Suitable and able to relate to the topics Based on the feedback obtained from the students (table 4.11), all the students agreed that this lesson was suitable. Two of the students mentioned that it was helpful because they could relate it back to the skills introduced. Siti: “The information given is simple and well structured. There are also numerous exercises to try out”. Table 4.12: Responses of the students on Strategies for reading academic reading texts. Student Comment Siti · Introduces the reading strategies in an orderly manner Vincent Lee Keshav Anita
· · · ·
Information provided in point form Good and appropriate Numerous exercises to practice Helpful and relevant
The findings above revealed (table 4.12), that the strategies for reading academic reading text were suitable and relevant because the information was well organized and there were numerous exercises to practice.
Table 4.13: Responses from the students on Critical Reading Student Comment Siti · List of question given relevant. Vincent · Able to apply the skills on various reading materials. Lee · Example given was a helpful guide. Keshav · Different colour shade was used in the example passage was a good indicator for highlighting the types words selected. Anita · The exercise can help me comprehend the concepts better It was found that Critical Reading (4.13), was very exciting and engaging. Two students mentioned that the examples given were relevant and a helpful guide in further understanding especially the types of vocabularies used. One of the student mentioned that he was able to apply this reading skill across other reading materials. Therefore, it supported independent learning because it helps students to comprehend the material at their own pace. Lee: “The example given is very useful because I was able to do the subsequent exercise without much of a problem”. Table 4.14: Responses of the students on Screen layout and background colour Student Comment Siti · Suitable background colour Vincent
· ·
Lee
·
Keshav
· · ·
Anita
· ·
Suitable screen layout preferred a colored Background Screen layout is attractive Suitable background color Screen layout is appealing Use a lighter shade color for the background Suitable screen layout preferred a colored Background
Table 4.15: Responses from the students on font, graphics and animations Student Comment Siti · Font type and clear and easily read · No animation or mind map used. Vincent · Font type and size are suitable and easily read · No animation used. Lee · Font type, size and color were suitable and l unvarying. · No animation used Keshav · Font type, size and color were suitable · No animation used Anita · Font type, size and color were appealing and easily read. · No animation used
Based on the feedback obtained from the students as shown in table 4.14, all the students were very optimistic about the screen design. It was appealing to them. Apart from that, most of the students suggested to use colored background. They felt that it might make the lesson more attractive and appealing. Keshav: “The screen layout is appealing but using a lighter colour shade for the background colour may be more attractive”. According to the student’s response from (table 4.15), it was found that the type and size were clear, suitable and easy to read. All the students highlighted that no images or animations were inserted. Deriving from the observation conducted, it revealed that all the students were eager, curious and also focused during the teaching and learning process. Therefore, this experiment was carried out without much distraction. Siti: “Font type and size is clear and easily read, but no images no animation or mind maps used to deliver the information”. Generally, the feedback obtained from the instructional technologist, lecturers and students regarding this lesson were optimistic and affirmative.
Discussion, Conclusion, Implication and Suggestions for Future Research The aim of this study was to get the educational technologist, lecturers and students view and evaluate this selected website which was developed by Andy Gillett from School of Combined Studies, University of Hertfordshire, Hatfield, United Kingdom in 1996.This was done to ascertain the appropriateness, suitability as well as to view the shortcomings of the lesson. Hence, the feedback received from the instructional technologist, lecturers and students were given due consideration to allow this web based lesson to be carried out as an instructional tool in an English language classroom. Formative evaluation and summative evaluation was conducted. The formative evaluation involved the educational technologist and English lecturers because their opinions were relevant in order to view the appropriateness of this learning material for the targeted learners. According to Dick and Carey, (1990) formative evaluation was a process to gather data and help upgrade the instruction in order to produce a more efficient learning material. The summative evaluation involved the students or the targeted learners. This was done in order to evaluate the learning material because their feedback will be needed to further furbish this lesson in order to be carried out in a classroom. This finding was supported by Bhola, (1990), that summative evaluation was conducted to ensure that it has achieved the minimum standards of knowledge, skills and attitudes among the intended learners. Information Design: Findings received from the respondents reveal that the objectives were clearly stated. The clearly written objectives would enable the targeted learners to recognize the scope of this lesson and also help them to stay focused in order to achieve what is expected. The reading skills of this website were the only focus for this research. The links in the reading section which were analyzed included Advice, Efficient reading skills, Strategies for critical reading and critical reading. All the respondents agreed that content was well structured and fulfilled the curriculum needs. They also added that the instructions were direct and easily understood.
The lecturers highlighted that this exercises in this selected website allowed for immediate feedback and independent learning which shifted the teaching and learning process from teacher centeredness towards student centeredness. This is consistent with Owston (1997), that online lessons allowed for student centeredness as opposed to teacher centeredness because it allowed the learners to learn at their own pace and immediate feedback obtained from the exercises given. In addition, it also allowed for the use of their hands and minds. Therefore, this kind of activity encouraged exploration as well as promoted inquiry-oriented learning. Furthermore the instructional technologist and lecturers mentioned that the content were well organized and chunked accordingly. This is consistent with Shaumbaugh, R. N. & Magliaro, S. G. (1997) who stated that the information provided to the learners begins with a simple concept and moves towards a more complex one. Hence this flow of information facilitates learning and reduces disruptions among learners. The respondents mentioned that the language used was simple and easy to understand. On the whole, all the respondents agreed that the exercises were interesting and challenging. As a result, this engaging learning activity enabled the learners to explore this lesson better. This finding was consistent with Olcott (1999) who mentioned that an engaging online material helps students to comprehend the lesson better as it caters for different learning styles. Interaction Design: All the respondents agreed that they did not have any problems with navigations. Hence the respondent did not have any disruptions during the navigation, which further encouraged them to explore the lesson. All the respondents agreed that the features included in this online lesson were relevant and interesting. These findings were supported by Shaumbaugh R.N. & Magliaro S. G. (1997) who stated that a module should be comprehensible to the learners, helpful in learning and the teachers as well as the students are able to draw relationships between the concepts and content delivered. Presentation Design: Most of the respondents said that the screen layout was simple and appealing. There were differences in opinion with regards to the background color of the module between the instructional technologist and lecturers because they suggested using white color as the background colour for the module; where else the students suggested using lighter color background. All the respondents agreed with the font size used. They all agreed that it was clear and easy to read. The obvious suggestion by all the respondents is to insert animation. This is to diversify the delivery of information as well as to capture the viewer’s attention.
Conclusion and Implications: This study focused on evaluation of a selected free website. It addresses three elements of design process which are the information design, interaction design and presentation design which was introduced by Kristof and Satron, (1995) whom suggested that an interactive learning system includes these three elements. The overall responses received from the all the respondents were very positive. The educational technologist and lecturers found this online lesson attractive, benefiting and engaging. The strengths of this selected website included the following like the content was well organized and chunked appropriately with achievable objectives. Furthermore it had interesting and challenging activities, which helped to promote higher order thinking. Therefore, all the respondents were optimistic and affirmative towards this lesson. In conclusion, this selected web based lesson is suitable to be executed in classrooms. Yusof, (2000) stated that information technology is part and parcel of today’s society. As information and communication has become an integral part of our daily lives, it is essential that educators and students of this millennium learn to manage and use the information accordingly. Apart from that educators should be inclined to deliver their lessons from various types of media like web based lessons and not solely rely on print base or CD base versions. Teachers should also be more pro–active in using the new technologies available in order to create a more interactive and engaging lesson. Suggestion for further research In this study, the selected website was evaluated so that it can be used as an instructional tool in an English Language classroom. Apart from that it also supports interactive and engaging teaching and learning. Possible area for future research is to conduct a case study to see if the students’ performances in English language improve after being exposed to this kind media in their classrooms. Another area of study is to see whether it can be implemented in other subjects like Mathematics or science.
References: Barba,R. H.. (1998).DIT 186, Using Instructional Media. Retrieved on 11th August 2005 Website:http://www.2.sjsu.edu/depts/it/edit186/mod13.htm Bhola, H. S. 1990)Evaluating "Literacy for development" projects, programs and Campaigns: Evaluation planning, design and implementation, and utilization of evaluation result.Hamburg, Germany: UNESCO Institute for Education; DSE [German Foundation for International Developement]. xii, 306 pages. Retrieved on 5th April 2006.Website http://www.sil.org/lingualinks/literacy/ReferenceMaterials/GlossaryOfLiterac yTerm/WhatIsEvaluaton.htm Boaz, Mary, Elliot, Betty; Foshee, Don, Hardy, Darcy, Jarmon, Carolyn; Olcott, Don Jr.(1999).Teaching at a Distance: A handbook for Instructors.Available at: http://eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/11/99/0e.p df . Retrieved on 4th April 2006.
Dick, W and Carey, L. (1990). The Systematic Design of Instruction (3rd Edition). USA : Harper Collins Publishers Gagne, R.M. (1985). The conditions of learning (4th edition). New York: Rinehart & Winston. Grasha, A.(2004).Teaching Styles and the WWW. Retrieved August25,2004 from Center for Teaching and Learning, Indiana State University.Available at http://web.indstate.edu/ctl/5styles.html Retrieved on 15th March 2006. Jonassen, D, (1996). Computer in the classroom: Mindtools For Critical Thinking. Kristof, R and Satran, A. (1995). Interactivity by Design. Mountain View, CA:Adobe Press. Owston, R.D. (1997) . The World Wide Web: A Technology to Enhance Learning. Available at : http:// www.edu.yorku.cal~ owston/ article.html Retrieved on March 30th 2006.
Shaumbaugh, R. N. & Magliaro, S. G. (1997). Mastering the Possibilities: A Process Approach to Instructional Design. Boston: Allyn and Bacon. Yusof, S.(2002). Information technology training gets a boost. Retrieved on 5th May 2005 http://www.web4.infotrac.galegroup.com/itw/in