IJLTER.ORG Vol 20 No 2 February 2021

IJLTER.ORG

p-ISSN: 1694-2493 e-ISSN: 1694-2116

International Journal of Learning, Teaching And Educational Research

Vol.20 No.2

International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 2 (February 2021) Print version: 1694-2493 Online version: 1694-2116

IJLTER International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 2

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.

Society for Research and Knowledge Management

International Journal of Learning, Teaching and Educational Research The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal which has been established for the dissemination of state-of-the-art knowledge in the fields of learning, teaching and educational research.

Aims and Objectives The main objective of this journal is to provide a platform for educators, teachers, trainers, academicians, scientists and researchers from over the world to present the results of their research activities in the following fields: innovative methodologies in learning, teaching and assessment; multimedia in digital learning; e-learning; m-learning; e-education; knowledge management; infrastructure support for online learning; virtual learning environments; open education; ICT and education; digital classrooms; blended learning; social networks and education; etutoring: learning management systems; educational portals, classroom management issues, educational case studies, etc.

Indexing and Abstracting The International Journal of Learning, Teaching and Educational Research is indexed in Scopus since 2018. The Journal is also indexed in Google Scholar and CNKI. All articles published in IJLTER are assigned a unique DOI number.

Foreword We are very happy to publish this issue of the International Journal of Learning, Teaching and Educational Research.

The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal committed to publishing high-quality articles in the field of education. Submissions may include full-length articles, case studies and innovative solutions to problems faced by students, educators and directors of educational organisations. To learn more about this journal, please visit the website http://www.ijlter.org.

We are grateful to the editor-in-chief, members of the Editorial Board and the reviewers for accepting only high quality articles in this issue.

We seize this opportunity to thank them for their great collaboration.

The Editorial Board is composed of renowned people from across the world. Each paper is reviewed by at least two blind reviewers.

We will endeavour to ensure the reputation and quality of this journal with this issue.

Editors of the February 2021 Issue

VOLUME 20

NUMBER 2

February 2021

Table of Contents Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human Cognition Schema of Psychology Students ......................................................................................................................... 1 Guadalupe Elizabeth Morales-Martinez, Janneth Trejo-Quintana, David Jose Charles-Cavazos, Yanko Norberto MezquitaHoyos, Miriam Sanchez-Monroy Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers .............................. 22 Arlinda Beka, Ganimete Kulinxha The Efficacy of Habits of Mind in the Inculcation of Self-Directed Learning Skills in Pre-Service Teachers ........... 36 Gideon Petrus Van Tonder, Byron John Bunt, Alvine Petzer, Hermann Davin Bosch, Nicolaas Van Deventer, Amanda Gerber, Lizelle Van Schadewijk The Impact of Professional Development, Modern Technologies on Lecturers’ Self-Efficacy: Implication for Sustainable Science Education in Developing Nations .................................................................................................... 61 David Agwu Udu, Benson Ikechukwu Igboanugo, John Nmadu, Chidebe Chijioke Uwaleke, Benjamin Chukwunonso Okechineke, Adaora Phina Anudu, Precious Chisom Attamah, David Onyemaechi Ekeh, Mercy Ifunanya Ani A Competence Model to Assess and Develop Designing Competence Assessment Tool .......................................... 81 Do Tra Huong, Nguyen Thi Dieu Linh Cultural Language Learning Approach (CLLA): An Approach for Learners’ Integrated Language Use Acculturation ....................................................................................................................................................................... 104 Hermayawati .

The Impact of Using VAKT Strategy on Oral Reading and Reading Comprehension Skills of Elementary Students with Dyslexia ....................................................................................................................................................................... 121 Ayed H. Ziadat The Influence of Architecture Students’ Learning Approaches on their Academic Performance in Two Nigeria Universities .......................................................................................................................................................................... 137 Gabriel Sen, Albert Adeboye, Oluwoye Alagbe Blended Learning in Rural Primary ESL Classroom: Do or Don’t ............................................................................... 152 Melanie Jerry, Melor Md Yunus Developing Interaction in ESL Classes: An Investigation of Teacher-Student Interaction of Teacher Trainees in a Sri Lankan University......................................................................................................................................................... 174 Abdul Majeed Mohamed Navaz Teaching with Relevance: Saudi Students’ Perceptions of a Foundation Course in Communication Skills .......... 197 Philline M. Deraney Students’ Soft Skills, Hard Skills, and Competitiveness (SHC): A Suggested Model for Indonesian Higher Education Curriculum........................................................................................................................................................ 218

Hadiyanto ., Noferdiman ., Syamsurizal ., Muhaimin ., Ina Krisantia Decision-making by Heads of Academic Department using Student Evaluation of Instruction (SEI) ................... 235 Mohammed Saleh Alkathiri Fostering Media Literacy Skills in the EFL Virtual Classroom: A Case Study in the COVID-19 Lockdown Period ............................................................................................................................................................................................... 251 Marina Bilotserkovets, Tatiana Fomenko, Oksana Gubina, Tetiana Klochkova, Oksana Lytvynko, Maryna Boichenko, Olena Lazareva Analysis of Future Teachers’ Perceptions on the Evaluation of Learning: A Case Study of Chile .......................... 270 José Hernández Sepúlveda, Rodrigo Panes Chavarría, Karla Rosalia Morales Mendoza Career Readiness of Graduating Office Technology and Management Students in Polytechnics in Southeast Nigeria .................................................................................................................................................................................. 290 Lynda Chineze Ezechukwu, Oliver Okechukwu Okanazu, Muhammadu Madu-Saba Babalulu, Ayoola Abimbola Arowolo, Chijioke Jonathan Olelewe Information Technology Integration Perception on Ghanaian Distance Higher Education: A Comparative Analysis ................................................................................................................................................................................ 304 Albert Arthur Qua-Enoo, Brandford Bervell, Paul Nyagorm, Valentina Arkorful, John K. E. Edumadze ESL Teachers’ Challenges in Implementing E-learning during COVID-19 ................................................................ 330 Brenda Anak Lukas, Melor Md Yunus

1

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 1-21, February 2021 https://doi.org/10.26803/ijlter.20.2.1

Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human Cognition Schema of Psychology Students Guadalupe Elizabeth Morales-Martinez and Janneth Trejo-Quintana Cognitive Science Laboratory, IISUE, National Autonomous University of Mexico, Mexico City, Mexico https://orcid.org/0000-0002-4662-229X https://orcid.org/0000-0002-7701-6938 David Jose Charles-Cavazos TecMilenio University, Mexico City, Mexico https://orcid.org/0000-0002-3445-9026 Yanko Norberto Mezquita-Hoyos Autonomous University of Yucatán, Yucatan, Mexico https://orcid.org/0000-0001-6305-7440 Miriam Sanchez-Monroy Tecnologico Nacional de Mexico-Instituto Tecnologico de Merida, Yucatan, Mexico https://orcid.org/0000-0001-5263-1216

Abstract. This study measured the structural and organizational changes in the knowledge schema of human cognition in response to the learning achieved by 48 students enrolled in the second year of a psychology degree. Two studies were carried out based on the Chronometric Constructive Cognitive Learning Evaluation Model. This article deals only with the first one, which consisted of a conceptual definition task designed in line with the Natural Semantic Network technique.

Participants defined ten target concepts with verbs, nouns, or adjectives (definers), and then weighed the grade of the semantic relationship between the definers and the target concepts. The data indicate that the initial knowledge structures had been modified towards the end of the course. The participants’ human cognition schema presented changes in terms of content, organization, and structure. This evidence supports the idea that the acquisition and transformation of the schemata learned in academic environments may be observed through cognitive science indicators.

Keywords: cognitive evaluation; knowledge schema; learning; NSN; psychology students

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

2

1. Introduction Assessing academic learning is one of the most significant challenges for educators in the twenty-first century. This is supported by William (2011), who asserted that assessing learning is a central activity in the instruction-learning process. There is a great diversity of learning measurement tools, especially given the development of new technology, which has opened up new possibilities in this field. However, there is still no consensus on the most convenient way to assess student learning. This problem means that although there is a diversity of tools with which to measure academic learning, there is no agreement about the best way to determine what and how much content a student has learned during a course. El-Yassin (2015) remarked that there is no right or wrong way to evaluate student learning since each instrument can inspect a specific learning aspect. In addition, William (2011) pointed out that although the sequence of presentation, quality, and even teaching in a class is the same for all students, they understand what they learn in the classroom differently and may even learn different things to what they are taught.

This variability in students’ academic learning has long been considered a barrier to teaching rather than a source of enrichment within the classroom. In this regard, William (2011) discussed how for many years, those involved in the educational field assumed that the quality of instruction alone would be enough for students to learn, and failure to learn in spite of effective instruction was attributed to the students’ cognitive characteristics. Currently, the educational community is beginning to raise awareness about the role played by an individual student’s needs and cognitive characteristics in the design of teaching-learning sequences.

Regarding the above, in the 1980s, Messick (1984) stressed that the interpretation of achievement measures should be carried out in the context of the style of instruction and learning to reduce errors in the interpretation of academic performance and students’ functioning within specific learning environments.

Although this proposal sounds obvious, Messick explained that measuring learning in such an all-encompassing way is rarely feasible due to the complexity of the information that needs to be extracted at different levels of student life.

In general, learning assessment can be very complex due to the broad spectrum of factors involved. According to Muskin (2015), the evaluation of learning implies using a means to determine what a person knows in conceptual or procedural terms. In this regard, Messick (1984) pointed out that school learning not only involves the content that a student can store in their memory, but also how the student structures or restructures their knowledge and cognitive skills according to their level of academic development (beginner, intermediate, or advanced).

Messick (1984) suggested that any learning measurement should take account of the state of academic development of each student to establish the cognitive functioning level at which the learning assessment will be carried out. For example, Messick proposed that with students in an initial learning phase, the objective should be to acquire information. At this level, information-retrieval

http://ijlter.org/index.php/ijlter

3

recognition assessments could be used. In contrast, at a more advanced level, student learning should manifest itself in the restructuring of schemata and the flexible use of schemata to solve problems. However, Messick saw the application of such a proposal as very forward-looking rather than being based on the reality of developing performance tests.

Currently, most evaluation instruments are focused on performance measurement. In this regard, Banister (2004) pointed out that in psychology, the most commonly used instruments to measure learning are exams, practical tests, and empirical dissertations. These kinds of tools are used as summative assessments of student performance. The tests provide valuable information about aspects of students’ knowledge of the information evaluated in the test.

However, they are not planned to have implications for the design of instruction techniques (Arieli-Attali, 2013). Exams have been criticized for being indirect measures that do not take into account context and that are more oriented towards obtaining a product rather than understanding the learning process (Sadeghi & Rahmati, 2017).

Summative assessments are useful in this sense as they are used for what they were designed. However, when the main objective is to provide information on the processing of the information inputs that students receive in the classroom, rather than on the performance (the output from the process), then the necessary use of alternative tools to measure the cognitive processes of assimilation and accommodation of information as a result of learning becomes evident.

Nevertheless, scientific exploration of the use and impact of evaluation tools to assess cognitive changes and provide useful indicators to correct or promote the restructuring of a learned schema is still an underexplored field.

One way to approximate this learning-evaluation challenge is to include cognitive psychology tools to measure the human mind. This scientific discipline has high potential to evaluate skills (Embretson, 1999) and the formation of knowledge structures, and can thus be applied to different aspects of the learning process. For example, Marzano’s Learning Dimensions Model identifies five kinds of thinking involved in the learning process: a) attitudes and perceptions, b) acquisition and integration of knowledge, c) extending and refining knowledge, d) the meaningful use of knowledge, and e) mental habits (Marzano & Pickering, 1997).

The measurement of these dimensions can be approximated with the paradigms and research techniques involved in human cognition science. For example, the research techniques used to explore human memory can be extrapolated to studying the cognitive mechanisms involved in dimensions b, c, and d of the Learning Dimensions Model.

Arieli-Attali (2013) stated that the idea of including advances in cognitive science to develop new forms of measurement or complement psychometric means of evaluation is not new. Initiatives have been emerging since the last century to link advances in cognitive psychology to the measurement of abilities. For example, the Air Force Human Resources Lab carried out the Learning Abilities Measurement Project (LAMP) (Kyllonen & Christal, 1988), which sought to

http://ijlter.org/index.php/ijlter

4

identify indicators of student learning and achievement, taking into account measures for processing capacity, speed of processing, knowledge, and skills. The results of this seminal effort demonstrated that cognitive measures could successfully predict performance in learning tasks and even do so with greater precision than some instruments already available. Later initiatives such as the Cognitive Design System (CDS) (Embreston, 1999) or Evidence-centered Design (ECD) (Mislevy, Steinberg & Almond, 2003) have continued to promote the concept of using cognitive tools within the assessment of learning.

The Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM)

by Lopez and Morales (Lopez et al., 2014; Morales-Martinez & Lopez-Ramirez, 2016; also see Morales-Martínez, 2020; Morales-Martinez et al., 2017; MoralesMartinez, Lopez-Ramirez & Lopez-Gonzalez, 2015) is a recent initiative to promote the use of cognitive measurement tools to evaluate academic learning.

This evaluation model is based on applying the laws and principles for how the human mind selects, stores, and retrieves information.

From cognitive psychology, the human mind is seen as a producer of cognitive structures called schemata. These mental structures are formed with the knowledge that people store in their memories. Schemata possess properties relating to their flexibility and stability. In the educational field, the students form schemata from materials learned on a course or in a career. These schemata can remain or be modified over time, depending on how students store, organize and structure their learning.

Keeping the above idea in mind, Lopez (1989) proposed an academic-failure-rate predictor system based on evaluation techniques derived from the Theory of Human Information Processing (HIP) and the Theory of Parallel Distributed Processing (PDP). Lopez attempted to show that the study techniques from these areas allow the properties of learned-knowledge schemata to be observed and measured in the same way that general knowledge schemata can be observed. He tested this idea in his doctoral thesis, by designing and applying the Semantic Analyzer of Schemata Organization (SASO). This system allowed him to explore knowledge schemata in human memory (Lopez, 1996; Lopez & Theios, 1992).

Later, Lopez et al. (2014) used this model to create a new system by which to evaluate learning. This learning-evaluation system was the origin of the Cognitive Evaluator (known in Spanish as EVCOG), which is a computerized system that assesses academic learning, and which gave rise to the C3-LEM developed by Morales-Martinez & Lopez-Ramirez (2016; also see Morales-Martinez et al. 2017, Morales-Martinez, Ángeles-Castellanos et al. 2020).

The C3-LEM (Figure 1) offers an alternative way to measure various aspects of mental representation of the knowledge students learn in academic courses. For example, this model allows indicators on the schematic organization of knowledge to be obtained. Arieli-Attali (2013) pointed out that measuring the conceptual understanding advances of students during a course can provide useful information to support the design of teaching and learning strategies that help students learn the knowledge and skills necessary to adapt to an

http://ijlter.org/index.php/ijlter

5

environment whose economy is based precisely on information and knowledge management.

Figure 1. Phases and components of the C3-LEM Note: From “Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course,” by Morales-Martinez, ÁngelesCastellanos et al., 2020, International Journal of Learning, Teaching and Educational Research, 19(9), p. 346 (https://doi.org/10.26803/ijlter.19.9.18). Copyright 2020 by the authors and IJLTER.ORG.

Figure 1. illustrates the phases and components that make up the C3-LEM. In general, this evaluation model promotes the combined and intertwined use of mental representation techniques, computational simulation tools, and chronometric cognitive measurement techniques to assess the modifications in the organization and mental structure of knowledge, as well as the dynamics and temporal changes in the learned schemata (Morales-Martinez, ÁngelesCastellanos et al., 2020; Morales-Martinez, Lopez-Perez et al., 2020).

C3-LEM studies are based on the EVCOG procedure, which consists of two phases: constructive cognitive evaluation and chronometric cognitive evaluation (Figure 1). Together, these two approaches provide indicators of students’ cognitive mechanisms in terms of their ability to select, elaborate on, and build knowledge from the information obtained from an academic course. This article focuses on using the constructive cognitive evaluation of knowledge since it illustrates the first step for evaluating learning with C3-LEM. The objective is to contribute empirical evidence on the usefulness of cognitive techniques for measuring organization and structural changes in students’ knowledge schemata due to the learning process in a human cognition course.

1.1. Constructive Cognitive Evaluation of Knowledge Schemata Learned during an Academic Course The constructive cognitive evaluation of learning involves measuring the knowledge schema’s properties through a mental representation technique and computer simulations. The central idea is to observe the conceptual changes that occur in the student’s memory due to the learning process.

http://ijlter.org/index.php/ijlter

6

Typically, the first step consists of applying the Natural Semantic Network (NSN) technique at the beginning and the end of the academic year (see the Methodology section), although any other technique that allows organization indicators and conceptual structure to be extracted can be used. Figueroa, Gonzalez & Solis (1976) proposed the NSN as a mental representation technique to explore meaning formation. According to Figueroa-Nazuno (2007), the construction of meaning depends entirely on the person who constructs it. The person elaborates and interprets knowledge through a constructive and reconstructive process of memory. So, from this conceptualization of cognitive functioning, the formation of meaning goes beyond free association.

Mental representation studies based on the C3-LEM have provided evidence that students construct or reconstruct their declarative knowledge schemata as a result of the learning obtained during a course. For example, Morales-Martinez, LopezPerez et al. (2020) applied the NSN technique to measure the knowledge schema arising from a course on the Computational Theory of Mind. They observed that students enter the course with a pre-schema. However, no conceptual organization could be identified between the pre-schema nodes. After the course, the students had assimilated new concepts, eliminated some information nodes, and established an organization amongst the conceptual nodes they had learned during the course. These results agree with Bower’s (1975) seminal idea that the acquisition of declarative schemata embraces the incorporation of new information nodes.

Moreover, the studies using NSN have been able to identify limitations in the knowledge structures of students, relating to each individual’s level of academic development in terms of the subject they are learning. Morales-Martínez, Mezquita-Hoyos et al. (2018) noted that students who did not achieve passing grades on the computational usability course had fractured knowledge schemata at the end of the course. Morales-Martinez, Angeles-Castellanos et al. (2020) reported similar data in their cognitive diagnostic study on the structure and organization of the human anatomy knowledge schema amongst first-year medical students. The data from this study pointed to fractured cognitive structure in the schema and difficulties with conceptual organization.

Some reasons for schematic fragmentation include the relevance weight given to the different topics within a course or a lack of emphasis on establishing the relationships or connections between the topics reviewed during the academic course (Morales-Martinez, Ángeles-Castellanos et al., 2020). Fragmented knowledge structures are also observed in students starting a course to review a new topic (Morales-Martínez, López-Pérez et al., 2020; Urdiales-Ibarra et al., 2018).

Information integration strategies influence the formation or correction of integration limitations in knowledge structures such as those mentioned above.

In this regard, Morales-Martínez, Mezquita-Hoyos et al. (2018) reported that engineering students with a fractured schema at the end of their course managed

http://ijlter.org/index.php/ijlter

7

to integrate information from the computational usability schema after attending a corrective course on the subject.

In general, NSN provides information on how the student’s mind organizes and structures knowledge schemata according to the learning experiences during academic courses. Few studies exist which have used the C3-LEM approach to explore the knowledge domain in psychology. Specifically, the topics covered to date using C3-LEM relate to the Piagetian Theory schema and the Computational Theory of Mind (e.g., Morales-Martínez, López-Pérez et al., 2020). The results of these studies suggested that students start the courses with vague but pre-organized ideas about the knowledge that they will review throughout the course. At the end of the course, students with passing grades had acquired new information nodes in the cognitive structures related to their knowledge.

Additionally, they had established new relationships between concepts and reconstructed or reorganized their schemata based on their learning experiences.

However, more investigations offering empirical evidence on the learning properties of knowledge schemata in psychology are necessary to build a solid theory about the behavior of schemata in this field of knowledge. The present study contributes new information on the organization and schematic behavior of the knowledge structures acquired in one of the most relevant fields of psychology science, human cognition.

2. Methodology 2.1. Study Overview This research measured the state of knowledge on the human cognition schema amongst students enrolled in the second year of a psychology degree at the beginning and end of a course. The state of knowledge refers to the set of cognitive properties (organizational, structural, temporal, and dynamic) that characterizes students’ knowledge schemata in any academic course. For example, at the beginning of a course, students present less semantic richness than at the end of the course. In addition, throughout the course, students judge the semantic relevance of concepts in different ways. Moreover, the recognition pattern for schematic words is different at the beginning, during, and at the end of the course.

Thus, this study explored the changes in the organization and structure of the human cognition schema experienced by students as a result of the learning acquired during a cognition course. The authors designed an NSN study that included a conceptual definition task related to the human cognition schema.

2.2. Participants The participants were 48 second-year psychology students enrolled in a course on human cognition. Their ages ranged from 19 to 34 years old (M = 20.3, SD = 2.58).

Overall, 79% (38) were women and 21% (10) were men. The authors selected participants using a convenience sampling technique. Potential participants were included in the study only if they took part voluntarily and signed the informed consent. Participants who did not finish the two application phases or did not follow the instructions were excluded from the study.

http://ijlter.org/index.php/ijlter

8

2.3. Study Design The study design was based on the EVCOG sequence proposed in the C3-LEM. The researchers designed a mental representation study based on the modified NSN from Lopez and Theios (1992) and Lopez (1996). The objective was to measure the cognitive properties of the content, organization, and structure of the human cognition schema.

2.4. Instruments and Materials To build the NSN instrument, the researchers selected ten target concepts from the Protocol for the Collection of Target Concepts and Central and Deferred Definers (Morales-Martinez, 2015). This protocol guides the teacher or knowledge domain expert in terms of identifying the most relevant conceptual targets for the course. The resulting ten concepts were considered to be the evaluated schema concepts. The ten conceptual targets selected by the teacher were: cognition, cognitive psychology, perception, attention, consciousness, memory, representation of knowledge, reasoning, problem-solving, and decision-making.

The researchers used EVCOG software to design and apply the cognitive studies of mental representation. Additionally, this software allowed the capture and analysis of data based on the C3-LEM (Morales-Martínez, López-Pérez et al., 2020).

2.5. Procedure In this study, the constructive cognitive evaluation of learning comprised the application of a task based on the NSN technique at the beginning and end of the course. First, the researchers invited students who were enrolled in a course on human cognition to participate in the research. Subsequently, the students who agreed to participate received information about the study and their rights as participants, and gave their informed consent. After this, they performed an exercise to familiarize themselves with the task. Finally, the NSN study was applied.

During the NSN study, each participant observed the target concepts one by one on a computer screen. The task was to define the targets using verbs, nouns, adjectives, and pronouns as definers. The production criterion for definers was that they had to be directly related to their course content on human cognition.

Phrases, articles, and prepositions were not allowed to be used for the definitional task. The participants had 60 seconds to define each target. Subsequently, they rated each definer using a scale from 1 to 10; 1 meant that the evaluated definer chosen was not very related to the target concept, and 10 indicated that the definer was significantly related to the target concept. The time to complete the entire task varied from 15 to 20 minutes, depending on each participant.

3. Data Analysis In this study, the authors undertook three analyses of the NSN data. The first analysis was a traditional mental representation analysis using the EVCOG system. This software allows several NSN values, proposed by Figueroa et al.

(1976) and described by Lopez (1996) and Lopez and Theios (1992), to be

http://ijlter.org/index.php/ijlter

9

computed. This analysis involved various elements which are described below.

The indicators for the analysis included, firstly, semantic richness (J value), generated for each target concept through the total number of different definers.

Secondly, semantic relevance (M value) was obtained from the score consciously given by the participants for each target definer, expressed as the sum of all the weights assigned by the participants to each definer. The ten most relevant defining concepts were also identified to build the meaning of the target concept of the network. This group of definers is known as the SAM group (Semantic analysis of M value or SAM) and is made up of the ten definers with the highest M values for each target concept. Another indicator was semantic distance (FMG value) between the given definer and the target concept that was defined. This is computed using the percentage range corresponding to the M value of each of the definers obtained for the SAM group in relation to the highest M value obtained in the group. Finally, semantic density (G value) was calculated.

The second analysis was undertaken using the EVCOG system. This software allowed the extraction of the association matrix. This matrix is called the SASO connectivity matrix, which is calculated using a Bayesian formula proposed by Lopez and Theios (1992). According to these authors, this equation is a modification of that by Rumelhart et al. (1986). Lopez and Theios’s equation is given below: WIJ = -1n{[p(X = 0 & Y = 1) p(X = 1 & Y = 0)]*[p(X = 1 & Y = 1) p(X = 0 & Y = 0)]-1} [1] This equation calculates the co-occurrence probability amongst pairs of concepts (X and Y) throughout the NSN. Firstly, p(X = 0 & Y = 1) refers to the joint probability that Y appears but X does not appear in a SAM group. Similarly, p(X = 1 & Y = 0) denotes the joint probability that X appears but Y does not appear in a SAM group, and p(X = 1 & Y = 1) was computed in the same manner. The calculation of p(X = 1 & Y = 1) involved the hierarchical modulation of M values in the SAM groups.

The SASO connectivity matrix was used to feed the Gephi software to obtain a graphical representation of the accommodation of schema concepts (see Figure 3).

Finally, the authors used STATISTIC software (version 7) to apply a multidimensional scaling on the NSN data. To this end, the authors considered the co-occurrence of definer concepts for each target concept.

4. Findings/Results 4.1. Lopez and Theios’s Analysis of NSN Data The NSN data obtained before (Table 1) and after (Table 2) the course were analyzed based on the procedure described by Lopez and Theios (1992).

http://ijlter.org/index.php/ijlter

10

Table 1. SAM groups for the human cognition schema obtained from the participants before the course F 9 3 5 3 5 4 4 5 1 3

F 9 1 5 2 1 5 2 1 4 2

F 1 1 1 1 1 4 4 1 5 1

F 2 9 2 1 2 1 5 3 3 3

Cognition Definer M IRT Cognitive process 158 18 Mind 115 23 Memory 110 27 Learning 88 32 Thought 79 20 Attention 64 32 Perception 44 38 Capacity 40 41 Processing 36 33 Brain 35 31 J-value: 218 G-value 12.30 Attention Definer M IRT Cognitive process 129 24 Focus 91 27 Capacity 73 31 Stimuli 66 19 Concentrate 58 23 Memory 38 24 Senses 36 38 Selective attention 33 20 Perception 31 47 Cognition 30 72 J-value: 205 G-value: 9.90 Representation Definer M IRT Schemata 95 22 Image 81 20 Symbols 43 15 Models 39 21 Mental 36 15 Perception 33 32 Information 31 31 Object 28 27 Memory 27 39 Concepts 27 37 J-value: 175 G-value: 6.80 Decision making Definer M IRT Choice 104 17 Cognitive process 73 34 Options 58 30 Evaluation To reason Solutions Capacity Reasoning Analysis Thinking J-value: 212

53 42 38 34 33 31 27

F 9 3 5 5 3 4 4 1 1 1

F 3 9 1 5 4 3 1 1 2 3

F 3 9 1 5 3 1 1 5 4 2

Cognitive psychology Definer M IRT Cognitive process 159 22 Mind 147 18 Memory 141 36 Thought 93 26 Learning 75 24 Attention 73 35 Perception 72 37 Study 69 16 Behavior 64 29 Science 47 18 J-value: 258 G-value: Consciousness Definer M IRT Mind 78 23 Cognitive process 55 30 Mind state 45 11 Thought 44 38 Attention 42 27 Brain 40 44 Vigil 39 29 Internal 33 42 Cognition 31 41 Reasoning 30 23 J-value: 174 G-value: 4.80 Reasoning Definer M IRT Thinking 117 16 Cognitive process 72 20 Human 47 22 Thought 46 25 Analysis 45 33 Logic 43 19 Consciousness 32 33 Capacity/Ability 31 27 Information 29 30 Interpretation 27 46 J-value: 200 G-value: 9.00

F 2 2 2 9 1 4 3 1 4 1

F 1 1 1 3 9 1 1 1 4 1

F 3 9 3 2 5 5 3 2 2 5

Perception Definer M IRT Senses 97 26 Interpret 67 35 Stimuli 63 23 Cognitive process 54 33 Feel 51 24 Attention 43 22 Brain 36 54 Observe 34 22 Information 33 34 Reality 26 41 J-value: 217 G-value: 7.10 Memory Definer M IRT STM 127 26 LTM 105 29 Store 94 14 Learning 91 25 Cognitive process 88 28 Memories 86 24 Remember 67 14 WM 65 30 Information 63 32 Retrieve 59 33 J-value: 282 G-value: 6.80 Problem solving Definer M IRT Reasoning 64 18 Cognitive process 56 28 Thinking 52 18 Reason 51 16 Memory 46 27 Capacity 44 19 Analysis 41 28 Choice 39 36 Options 38 41 Thought 38 32 J-value: 192 G-value: 2.60

39 18 31 33 26 40 50 G-value: 4.60

Note: J = semantic richness, G = semantic density, F = occurrence frequency, M = semantic weight, IRT = inter-response time

http://ijlter.org/index.php/ijlter

11

Table 2. SAM groups for the human cognition schema obtained from the participants after the course Cognition F Definer M IRT 10 Cognitive process 259 13 3 Information 98 28 2 Mind 89 22 7 Memory 80 33 1 Cold cognition 79 31 3 Attention 68 33 1 Psychology 63 43 1 Hot cognition 54 29 1 Human 49 42 1 Processing 49 32 J-value: 373 G-value: 21.00 Attention F Definer M IRT 1 Filter 211 22 10 Cognitive process 190 20 1 Selective attention 124 23 1 Divided attention 95 31 2 Stimuli 93 39 1 Attenuation model 82 25 1 Sustained attention 81 23 1 Capacity 78 28 4 Perception 71 25 1 Focus 68 28 J-value: 360 G-value: 14.30 Representation F Definer M IRT 1 Schemata 261 23 2 Mind 198 19 1 Concepts 142 26 10 Cognitive process 99 25 7 Memory 98 33 1 Images 74 20 1 Imagine 73 14 2 Knowledge 59 24 2 Semantics 43 42 1 Absence 41 18 J-value: 332 G-value: 22.00 Decision making F Definer M IRT 1 Choice 222 14 1 Alternative 112 19 10 Cognitive process 105 29 1 Evaluation 94 19 1 Experience 80 25 2 Reasoning 79 37 2 Decision 78 18 1 Options 62 29 7 Memory 56 32 1 Normative theories 23 35 J-value: 331 G-value: 5.90

F 10 1 1 2 7 1 3 1 3 4

F 3 10 1 7 4 2 1 1 1 2

F 1 1 1 1 1 10 1 3 7 1

Cognitive psychology Definer M IRT Cognitive process 230 26 Science 167 20 Neisser 113 23 Cognition 92 30 Memory 79 31 HIP 78 29 Information 76 31 Representation 63 41 Attention 62 36 Perception 61 32 J-value: 378 G-value: 16.90 Consciousness Definer M IRT Attention 134 25 Cognitive process 127 24 Become aware 94 17 Memory 83 35 Perception 67 36 Knowledge 62 11 Unconscious 49 43 Explicit 48 42 Reflector 41 36 Cognition 41 49 J-value: 279 G-value: 9.30 Reasoning Definer M IRT Reasoning 239 20 Conclusion 216 25 Inductive 201 20 Syllogism 175 26 Analogical 110 27 Cognitive process 109 23 Premises 86 24 Information 78 25 Memory 66 34 Logic 60 26 J-value: 344 G-value: 17.90

Perception F Definer M IRT 1 Sensation 182 13 1 Interpret 152 22 1 Threshold 125 29 2 Stimuli 116 27 10 Cognitive process 110 32 1 Direct perception 84 26 1 Illusion 69 30 1 Senses 62 24 4 Perception 55 24 1 Gestalt 49 44 J-value: 336 G-value: 13.30 Memory F Definer M IRT 1 Store 286 20 1 Retrieve 258 30 1 LTM 232 32 1 STM 231 27 1 SM 230 26 1 Encoding 147 29 10 Cognitive process 112 22 1 WM 96 34 1 Implicit 77 25 2 Semantics 74 41 J-value: 411 G-value: 21.20 Problem solving F Definer M IRT 1 Objective 114 30 10 Cognitive process 103 19 1 Problem 100 25 2 Reasoning 87 30 1 Goal 81 17 1 Heuristics 77 37 7 Memory 76 48 2 Decision 70 38 1 Strategies 56 33 1 Initial state 55 29 J-value: 311 G-value: 5.90

Note: J = semantic richness, G = semantic density, F = occurrence frequency, M = semantic weight, IRT = inter-response time

Table 1 shows that the definers (cognitive process, mind, memory, short-term memory (STM), thinking, long-term memory (LTM), choice, senses, schemata) with the highest M in each SAM group before the course were mostly general. At the end of the course, however, most of the concepts with the highest

http://ijlter.org/index.php/ijlter

12

M in each SAM group were specific (cognitive process, sensation, filter, attention, store, schema, reasoning, objective, choice), as shown in Table 2. Besides, when comparing Tables 1 and 2, it can be observed that the students at the end of the course included new definers or information nodes, rearranged some definers, or eliminated concepts in the definitions of some targets. For example, the following definers for cognition: thought, capacity, perception, learning and brain were removed, and definers such as information, cold cognition, psychology, hot cognition, and human were included (Figure 2).

Figure 2. Conceptual changes in the target cognition

In general terms, cognitive process (M value = 159) was the definer with the greatest semantic weight in the entire network before the course (Table 1), whilst after the course, it was store (M value = 286) (Table 2). Additionally, cognitive process was

http://ijlter.org/index.php/ijlter

13

the definition with the highest appearance frequency at the beginning of the course (F = 9) and also at the end of the course (F = 10). The M value average for cognitive process at the beginning of the course was 93.77, whereas at the end of the course, it had increased to 144.4.

4.2. Gephi Analysis of NSN Data The researchers carried out a graphical analysis of the changes in the organization and structure of the NSN using the Gephi system (Bastian, Heymann & Jacomy, 2009). Gephi is open-access software which explores the properties of networks.

At the beginning of the course, the participants’ knowledge schema on human cognition was made up of four large modules of concepts (Figure 3). The first (blue) included memory-related definers (memories, learning, remembering, storing, retrieval, working memory (WM), short-term memory, long-term memory, information).

The second group (purple) consisted of definers related to cognitive psychology as a science (science, study, cognition, mental state, observing, wakefulness, feeling, internal, reality, interpretation, reasoning, attention, senses, mind, thought, cognitive process, stimuli, processing, brain, behavior). The third grouping (orange) embraced definers related to decision-making (solutions, reasoning, thinking, evaluation, analysis, ability, options, choice, consciousness, human, logic). The fourth group of definers (green) was made up of concepts relating to cognitive processes (memory, selective attention, concentration, symbols, perception, focus, image, schemata, models, mind).

After the course, the participants rearranged the human cognition schema into seven conceptual modules (Figure 3). The first module embraced definers associated with perception (orange) (senses, interpretation, illusion, sensation, threshold, direct perception, Gestalt). The second module (light green) included definers related to consciousness and attention (sustained attention, divided attention, selective attention, capacity, attenuation model, unconscious, filter, focus, realize, reflector, explicit). Module 3 (pink) grouped concepts related to problemsolving (initial state, strategies, problem, goal, heuristics, objective). Conceptual group 4 (dark green) encompassed definers on decision-making (alternative, options, choice, evaluation, experience). Module 5 (purple) concentrated concepts related to three objectives: cognitive psychology, cognition, and mental representation (schemata, images, absence, concepts, cold cognition, imagine, mind, processing, hot cognition, human, reasoning, cognition, stimuli, decision, semantics, memory, psychology, memory, Neisser, cognitive process, mental representation, HIP, science, attention, knowledge). Module 6 (brown) included definers on reasoning (deductive reasoning, premises, conclusion, logic, inductive, analogical, information, syllogism). The last module (blue) involved definers related to memory (sensory memory, short-term memory, long-term memory, retrieve, store, encoding, working memory, implicit).

Additionally, the Gephi analysis pointed out changes in the conceptual organization. The conceptual connections of definers had changed at the end of the course. To illustrate these changes, observe in Figure 3 that at the beginning of the course, cognitive process was a central definer concept in the primary schema that participants brought about human cognition, although it did not have a

http://ijlter.org/index.php/ijlter

14

connection with all the schema modules. At the end of the course, the concept of cognitive process retained its quality as a central conceptual node yet now also fully connected with all the targets and all the conceptual modules.

Figure 3. Gephi analysis of the NSN data obtained before and after the course

http://ijlter.org/index.php/ijlter

15

4.3. Multidimensional Scaling of NSN Data The researchers applied multidimensional scaling to the NSN data to examine the general structure of the human cognition schema. The analysis showed changes in the arrangement of target conceptual nodes due to the learning achieved during the course (Figure 4).

Figure 4. Multidimensional scaling analysis of the target concepts

http://ijlter.org/index.php/ijlter

16

The multidimensional scaling graph shows that the participants started the course without a specific structure in mind for the objective concepts, whilst at the end of the course, they had rearranged the objective concepts based on two dimensions. The first related to categorizing cognitive processes in terms of basic and higher order cognition (horizontal axis). Although the definition of the second dimension is not clear, in general, this dimension seems to be related to the use of knowledge structures (vertical axis). Note that the target concept for reasoning does not appear alongside targets such as problem-solving or decision-making, even though all of these processes involve making use of knowledge structures from memory.

5. Discussion This study has explored changes in the knowledge schema due to the learning process during a course on human cognition taken by second-year psychology students. First, the authors determined whether a human cognition schema existed before the course. The NSN and Gephi analyses indicated that the participants entered the course with a previous-knowledge schema or a knowledge pre-schema (see Table 1 and Figure 3). The existence of a knowledge pre-schema has been observed in other studies (e.g., Morales-Martinez, LopezPerez et al., 2020); however, the organization and structure are rudimentary. This finding suggests that students generally have a vague schema about the knowledge they will acquire in their courses, and it is based on this schema that they reorganize and reconfigure the information they will learn in class.

As psychology teachers, the authors have observed that the use of general schemata and previous learning to begin a new knowledge schema is a common phenomenon observed in the classroom. Students generally comment that they have come across certain information about the topic. It was therefore not unexpected that the participants in this study commented that they were slightly familiar with the topics. They had reviewed readings on cognitive processes in other courses, although this had not been from the perspective of the field of cognitive psychology.

The authors hypothesize that students use their previous learning experiences to form a general schema or make inferences about information related to the course in which they are enrolled. In this way, they have a conceptual basis from which to form a more sophisticated outline of the information they cover during the course. From a cognitive perspective, students can use or create a rudimentary cognitive structure that allows them to guide the reorganization and restructuring of their knowledge based on the new information inputs that they acquire through the course. If the cognitive structure is sufficiently broad and general, it will be flexible enough to undergo modifications due to the new learning experiences.

Interestingly, although the initial schema with which the participants in this study entered the course was very general, their schema was not fractured as has been observed in other courses where students start on a topic for the first time (e.g., Urdiales-Ibarra et al., 2018). This result may be because the participants in this study had reviewed cognition materials the previous year when taking different

http://ijlter.org/index.php/ijlter

17

courses, meaning that they had had previous information about the topic. At the start of their degree, the participating students were enrolled in a course where they reviewed some of the concepts included in the course on human cognition and obtained a passing grade on this initial course. Thus, they had general and pre-organized ideas about the meaning of some important target concepts in NSN studied in this research. Other studies have indicated that students who do not obtain a passing grade for a course have a fragmented schema at the end of the course compared to those who end the course with a passing grade (MoralesMartinez, Angeles-Castellanos et al., 2020; Morales-Martinez, Mezquita-Hoyos et al., 2018).

In this study, at the end of the course, the authors explored the changes that had taken place in the participants’ pre-knowledge schema of human cognition due to the learning acquired through the course. The analysis of the organization of the schematic knowledge indicated that the participants had established new relationships between the concepts. This result is consistent with Bower’s (1975) idea that the acquisition of declarative schemata necessarily involves incorporating new information nodes and new connections between these nodes.

The reader can compare the definers included in Tables 1 and 2 and observe that at the beginning of the course, for some target concepts, some of the definers were global concepts on the topic of human cognition. Meanwhile, at the end of the course, the definers were more specific and theoretically closer to the target evaluated. For example, for the initial conceptual definition of human cognition (Figure 2), half of the concepts were categorical (memory, thought, attention, perception, learning), and the other half were schematic (cognitive process, capacity, mind, brain, processing). At the end of the course, however, the participants included a greater number of schematic-type definers (cognitive process, information, mind, cold cognition, psychology, hot cognition, human, processing).

The change in predominance from categorical to schematic relationships in knowledge structures suggests that the participants had developed more sophisticated schemata. That is, instead of using as many exemplification schemata, their perception had changed and they were using more probabilistic schemata. It is possible that, when students start learning a knowledge domain, learning by exemplification dominates most of their knowledge acquisition process. As participants in this study acquired new knowledge and refined it, they began to use or establish other semantic relationships amongst the concepts. It would be useful to carry out further research to explore this phenomenon since there has been no discussion of this issue in previous research with C3-LEM to date (e.g., Morales-Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018).

Another modification in the knowledge organization, which is of note, was the change in the degree of generality with regard to the human cognition schema. At the beginning of the course, the participants formed some groups that included general definers and even incorporated information from other knowledge

http://ijlter.org/index.php/ijlter

18

schemata. For instance, module 2 of the Gephi analysis shows that before the course, participants included definitions of various target concepts (cognitive psychology, cognition, mental representation, perception) in the same group of concepts and included definitions of other knowledge schemata learned for other topics. For example, participants recovered conceptual nodes from the behaviorism field as stimuli instead of inputs or behavior instead of cognitive patterns (Figure 3).

The previous results indicate that at the end of the course, the participants were able to extend and refine their knowledge about human cognition, thus placing them at level three of Marzano’s Dimensions of Learning Model (Marzano & Pickering, 1997). On the other hand, according to Messick (1984), the participants in the present study would be in an intermediate stage of academic development in terms of the development of the knowledge schema on human cognition because indicators observed included not just the retrieval of information but a restructuring of their schema. In congruence with this idea, the analysis of the structural changes in the knowledge schema indicated a reconfiguration of the schema structure by the end of the course. In this regard, Figure 3 shows how the initial schema’s definers were arranged into four large modules, whilst the definers for the final schema were restructured into seven conceptual modules.

Changes in the configuration of the schematic structure have been observed in other studies that have used the C3-LEM (Morales-Martinez, Lopez-Perez et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018).

From the point of view of cognitive psychology, changes in schematic configuration patterns are an indication of learning. In this study, the changes to the schema’s configurational arrangement suggest that participants had rebuilt their structures based on the new meanings that they had acquired during the course.

The multidimensional analysis (Figure 4) on the target concepts showed that at the beginning of the course, the participants did not have a clear idea of how the course’s target concepts could form a wholly organized knowledge schema. At the end of the course, the participants organized the ten target concepts into two dimensions, the first one relating to the cognitive nature of the processes (basic vs. higher order cognition) and the second associated with the use of knowledge structures. Although some concepts such as reasoning were not correctly located in this second dimension, in general terms, this result suggests that the participants had understood the structure of knowledge underlying the course’s thematic organization, using the information implicit in the same target concepts.

Since this is a seminal intent of introducing a new way to analyze the results from C3-LEM, more evidence about this phenomenon is needed to explore and explain this kind of implicit cognitive change in the knowledge schema as a learning product.

In summary, the study results indicated that there were changes in the organization and structure of the human cognition knowledge schema of the participants. They had reconfigured their old four-module schema on human

http://ijlter.org/index.php/ijlter

19

cognition into a new one which included seven modules. The participants included new information nodes, eliminated conceptual nodes that belonged to other disciplines, and established new relationships between the old and new concepts.

6. Conclusions In conclusion, the results of the present investigation have implications in three areas. At a theoretical level, the study generated empirical evidence that supports the idea that students enter courses with prior knowledge of the subject they are going to study. For example, the study data indicated that the participants possessed a macro-schema of human cognition at the beginning of the course. This finding is relevant because it suggests that cognitive techniques such as those contemplated in the C3-LEM can help diagnose preconceived ideas. It opens up the possibility of correcting inaccurate information held by students when starting a course. The measurement of this type of pre-schema would empower the teacher to decide whether it is necessary to demystify some information or whether modifications are required in the application of the established work program to provide continuity or correct the knowledge structures held by students when starting the course.

In addition, the results demonstrated that the learning process involves the assimilation of new information and the elimination of specific conceptual nodes, as well as the restructuring of schematic information. Furthermore, evidence from the NSN study indicated that this type of technique can provide information on students’ academic development level in a course. This finding has important implications at the applied level. For example, how a student configures their knowledge can also be taken as an indicator of mastery of the course knowledge.

Consequently, the C3-LEM could be a valuable tool in the formative assessment of students. However, since the sample in this study was small and only addressed one domain of knowledge, new explorations must be carried out in other fields, such as the area of exact sciences (e.g., mathematics, chemistry, physics), to calibrate the scope and implications of this evaluation model in the design of new forms of educational evaluation and intervention.

Finally, at a methodological level, the study’s data supported the idea that mental representation studies from the C3-LEM perspective may help assess cognitive changes in the organization and structure of knowledge schemata.

7. References Arieli-Attali, M. (2013, October 20−25). Formative assessment with cognition in mind: The cognitively based assessment of, for and as learning [Paper presentation]. 39th Annual Conference: Educational Assessment 2.0: Technology in Educational Assessment.

Tel Aviv, Israel. https://www.iaea.info/conference-proceedings/ Banister, P. (2004). Assessment as a tool for fostering key skills. Psychology Learning & Teaching, 3(2), 109–113. https://doi.org/10.2304/plat.2003.3.2.109 Bastian, M., Heymann, S., & Jacomy, M. (2009, May 17−20). Gephi: An open source software for exploring and manipulating networks [Paper presentation]. Association for the Advancement of Artificial Intelligence, Third International AAAI Conference on

http://ijlter.org/index.php/ijlter

20

Weblogs and Social Media. San Jose California, United States. https://gephi.org/publications/gephi-bastian-feb09.pdf Bower, G. H. (1975). Cognitive psychology: An introduction. In W. K. Estes (Ed.), Handbook of learning and cognitive processes: Vol. 1. Introduction to concepts and issues (pp. 25– 80). Lawrence Erlbaum Associates.

El-Yassin., H. D. (2015). Integrated assessment in medical education. Journal of Contemporary Medical Sciences, 1(4), 36–38. http://www.jocms.org/index.php/jcms/article/view/51 Embretson, S. E. (1999). Cognitive psychology applied to testing. In F. T. Durso (Ed.), Handbook of applied cognition (pp. 629−660). John Wiley & Sons.

Figueroa-Nazuno, J. G. (2007, October 24−25). El significado de las Redes Semánticas Naturales: Y la tradición oral, 20 años después [The meaning of natural semantic networks: And oral tradition, 20 years later] [Paper presentation]. Primer Simposium Internacional: Cognición y Representación del Conocimiento.

Monterrey, Nuevo Leon, Mexico.

Figueroa, J. G., Gonzalez, E. G., & Solis, V. M. (1976). An approach to the problem of meaning: Semantic networks. Journal of Psycholinguistic Research, 5(2), 107−115. https://doi.org/10.1007/BF01067252 Kyllonen, P. C., & Christal, R. E. (1988). Cognitive modeling of learning abilities: A status report of LAMP (Learning Abilities Measurement Program). In R. Dillon & J. W. Pellegrino (Eds.), Testing: Theoretical and applied issues. Freeman.

Lopez, R. E. O. (1989, December 1−6). Sistema predictor de indice reprobatorio (SPIR) [Failure rate predictor system (SPIR)] [Paper presentation]. IX Coloquio de Investigación.

ENEP Iztacala, Mexico city, Mexico.

Lopez, E. O. (1996). Schematically related word recognition (Publication No. 9613356) (Doctoral dissertation). University of Wisconsin-Madison, Madison, Wisconsin.

ProQuest Dissertations & Theses Global.

Lopez, E. O, & Theios, J. (1992). Semantic analyzer of schemata organization (SASO).

Behavior Research Methods, Instruments, & Computers, 24(2), 277−285. https://link.springer.com/content/pdf/10.3758/BF03203508.pdf Lopez, E. O., Morales, G. E., Hedlefs, I., & Gonzalez, C. J. (2014). New empirical directions to evaluate online learning. International Journal of Advances in Psychology, 3(2), 40−47. https://doi.org/10.14355&ijap.2014.0302.03 Marzano, R. J., & Pickering, D. J. (1997). Dimensions of learning: Teacher’s manual (2nd ed.).

ASCD. Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21(3), 215–237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary research and perspectives, 1(1), 3−62.

Morales-Martinez, G. E. (2015). Protocolo para la recolección de conceptos objetivo y definidores centrales y diferidos (PRECODECD): Un sistema de codificación de conceptos extraídos de las redes semánticas naturales [Protocol for the collection of objective concepts and central and deferred definers (PRECODECD): A coding system for concepts extracted from natural semantic networks] (Unpublished manuscript). Institute of Research on the University and Education, National Autonomous University of Mexico, Mexico.

Morales-Martínez G. E. (2020). Sistema de evaluación cognitiva constructiva cronométrica del aprendizaje en línea y presencial [Online and face-to-face learning’s constructivechronometric cognitive assessment system] (Manuscript submitted for publication). Institute of Research on the University and Education, National Autonomous University of Mexico, Mexico.

http://ijlter.org/index.php/ijlter

21

Morales-Martinez, G., & Lopez-Ramirez, E. (2016). Cognitive responsive e-assessment of constructive e-learning. Journal of e-Learning and Knowledge Society (Je-LKS), 12(4), 39−49. http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/1187 Morales-Martinez, G. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired mental representations as an alternative measurement method technique to innovate e-assessment. European Journal of Educational Research, 6(4), 455−464. https://www.eu-jer.com/EU-JER_6_4_455_Morales-Martinez_etal.pdf Morales-Martínez, G. E., Lopez-Perez, R. M., Garcia-Collantes, A., & López-Ramírez, E. O. (2020). Evaluación constructiva cronométrica para evaluar el aprendizaje en línea y presencial [Chronometric constructive assessment to assess online and face-to-face learning].

Tecnología, Ciencia y Educación, 15(1), 105−124. https://www.tecnologia-ciencia educacion.com/index.php/TCE/article/view/371 Morales-Martinez, G. E., Lopez-Ramirez, E. O., & Lopez-Gonzalez, A. E. (2015). New approaches to e-cognitive assessment of e-learning. International Journal for eLearning Security (IjeLS), 5(2), 449−453. https://doi.org/10.20533/ijels.2046.4568.2015.0057 Morales-Martinez, G. E., Ángeles-Castellanos, A. M., Ibarra-Ramírez, V. H., & ManceraRangel, M. I. (2020). Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course. International Journal of Learning, Teaching and Educational Research, 19(9), 341−362. https://doi.org/10.26803/ijlter.19.9.18 Morales-Martinez, G. E., Mezquita-Hoyos, Y. N., Gonzalez-Trujillo, C. J., Lopez-Ramirez, E. O., & Garcia-Duran, P. J. (2018). Formative e-assessment of schema acquisition in the human lexicon as a tool in adaptive online instruction. In R. Lopez-Ruiz (Ed.), From natural to artificial intelligence: Algorithms and application (pp. 69−88).

IntechOpen. http://doi.org/10.5772/intechopen.81623 Muskin, J. A. (2015). Student learning assessment and the curriculum: Issues and implications for policy, design and implementation (Current and critical issues in the curriculum and learning). UNESCO International Bureau of Education. http://www.ibe.unesco.org/sites/default/files/resources/ipr1-muskinassessmentcurriculum_eng.pdf Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and sequential thought processes in PDP models. In J. McClelland, D. Rumelhart, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 2. Psychological and biological models (pp. 7−57). MIT Press.

Sadeghi, K., & Rahmati. T. (2017). Integrating assessment as, for, and of learning in a largescale exam preparation course.

Assessing Writing, 34, 50−61. https://doi.org/10.1016/j.asw.2017.09.003 Urdiales-Ibarra, M. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Carrillo-Colon, J. E. (2018). Biology schemata knowledge organization and meaning formation due to learning: A constructive-chronometric approach to concept mapping usability.

Creative Education, 9(16), 2693−2706. https://doi.org/10.4236/ce.2018.916203 Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/doi:10.1016/j.stueduc.2011.03.001

http://ijlter.org/index.php/ijlter

22

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 22-35, February 2021 https://doi.org/10.26803/ijlter.20.2.2

Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers Arlinda Beka Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo https://orcid.org/0000-0002-9871-783X Ganimete Kulinxha Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo https://orcid.org/0000-0003-4461-8841

Abstract. Self-reflections, especially those done through the usage of portfolios, are an essential component of teaching. Portfolio reflections allow pre-service teachers to set professional goals. This research was conducted using qualitative methodology, and all semi-structured interviews were organised with students and graduates from the Early Childhood Department-Faculty of Education, University of Prishtina.

The research results show that a portfolio provides future teachers with the opportunity to organise their work better while enabling cooperation between pre-service teachers, which ergo, directly affects their professional competencies. Owing to its benefits, portfolio should be integrated by the Faculty of Education of the University of Prishtina as an integral part of its work with students.

Keywords: portfolio; pre-service preschool teachers; teacher professional development

1. Introduction Self-reflection is one of the most important competencies that a teacher should possess. Reflection directly enables the identification of things that not only are a strong part of professional work but that also need to be advanced and to allow a higher level of professionalism. In terms of taking individual responsibility for thought and action processes, self-reflection is a required condition for assessment and regulation (Dilger, 2007). After completing a given task, one tries to reflect on the process and the work accomplished; however, sometimes one’s attention is drawn to the sections that have been the most successful or most challenging, leaving out many other aspects that may be equally important for enhancing the quality of one’s work and for continuing professional development.

Portfolios are a proper way of recording our accomplished work and the challenges and successes encountered during the process. The record of such data ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

23

allows one to go back in time to see what one went through, how one did one’s activities, how one conceived things or what challenges were faced during a given period. The portfolio technique is viewed as particularly huge with regard to selfintelligent learning. The technique forms a significant part in the advancement of polished skill through reflection and self-reflection (Riebenbauer, Dreisiebner & Stock, 2017).

Through the use of portfolios, pre-service teachers can succeed in developing a professional working plan, reflect on their work, develop additional skills in the use of technology for professional development and have a clear review of their work (Beka & Gllareva, 2016). Portfolios for self-reflection in pre-service teacher training provide vital input for trainers since they represent both the strengths and weaknesses of students (Cimermanová, 2019).

Portfolios play a significant role in not only the process of preparing teachers for employment but also in their licensing as teachers and coaches and their statelevel performance appraisals. Being reflective is a key aspect of any educational experience. It involves the continuous educational growth of both professional educators and students (Webster & Whelen, 2019).

However, portfolio is often regarded as a collection of facts and documents that testify teachers’ professional development and achievement. In some cases, it is considered as an important part where the teacher must present the work planned for certain periods, including the reflection on commitments which he/she has accomplished. Therefore, the teachers make the portfolio in terms of the purpose that it will serve them. Similarly, during pre-service studies, future teachers are sometimes required to have more reflections on their portfolio. In some cases, their portfolio is also required to include facts, arguments and goals that students aim to achieve during their studies (Stock & Winkelbauer, 2012).

2. Development of portfolios and the arguments The use of portfolios in pre-service teachers’ professional development and professional assessment has become widespread around the world. As a result, several contemporary studies and research projects are being conducted by various scholars and educational scientists regarding the importance of portfolios, their meaning and their relevance to teachers’ professional development. It is often thought that portfolio has begun to be affirmed in the last two decades as a contemporary approach to professional development. Portfolio, as a concept in education, has been developed in the last decades of XX century in the United States of America and the German-speaking countries (Slepcevic-Zach & Stock, 2018).

For a clear understanding of the concept of portfolio usage among teachers whether in pre-service or in-service, we need to determine the purpose of the portfolio and the way in which it acts as a professional aid to the teacher.

According to various authors, there is a discrepancy between the portfolio and the way it should be filled in, i.e., whether the portfolio will focus more on reflection or on the documentation of professional work.

http://ijlter.org/index.php/ijlter

24

In his paper, ‘Do Portfolios Have a Future?’, Driessen (2017) divides portfolio into two categories, namely reflective portfolios and comprehensive portfolios.

Reflective portfolio is primarily intended to help students improve their reflective abilities. It requires students to write a reflection on an aspect of their learning or career that is significant to them. Reflections can be found in comprehensive portfolios, but their material is much more varied than that of reflective portfolios (Driessen, 2017). Baumgartner (2009) classifies portfolio into the three following types depending on their purpose: reflection portfolio, development portfolio and presentation portfolio.The e-portfolio is a flexible tool that can be used for several purposes, including learning, evaluation/assessment and employability/career growth (Ciesielkiewicz, 2019). Trávníčková and Puhrová (2020) state that portfolios can be encountered in the following terms: ‘professional teaching portfolio’, ‘professional development portfolio’, ‘professional practice portfolio’ and even the relatively new term of ‘eportfolio’.

All the above types of classifications from different authors can be considered as being similar due to their common elements. The reflective portfolio, which is considered a crucial part of teachers’ professional development, is common in the classifications. The aim of comprehensive portfolios is to assist students in their learning and to evaluate their progress. Reflections can be found in comprehensive portfolios, but their material is much more varied than that of reflective portfolios (Driessen, 2017).

Since education methods, such as reporting, contemplation and representation of one’s learning activity, were major issues in the 1980s, the portfolio idea might have been a pillar in the German debate about educational reform (Häcker, 2009).

Like in Germany, in almost all European countries and beyond, the portfolio is beginning to be considered a significant part of pre-service and pre-service teacher work. Nowadays, portfolios are seen as “rich, contextual, highly personalised documentaries of one’s learning journey.” (Jones & Shelton, 2011, p.21).

There are numerous reasons why a portfolio is considered an important tool for teacher development and professional development. Portfolios are considered beneficial for pre-service and in-service teachers “as they promote autonomous learning and autonomous learners are often motivated and reflective learners.” (Fernández, 2014).

In almost all professions, especially the teaching profession, reflection is a crucial part. According to Lynch (2000), reflection is considered to be an important part of a person’s skill set. Through reflection, each teacher undergoes a long and complicated process to reach his/her desired stage of professional development.

Within the scope of the portfolio-work, not only the result (item) is critical but mostly the method that led to this improvement (Kerr, 2007). Therefore, the documentation of professional experiences through the use of a portfolio is an asset for every pre-service teacher. Usually, the portfolio’s documentation is structured and deliberately designed as “they contain purposefully organised documentation that demonstrates specific knowledge, skills, dispositions, and accomplishments achieved

http://ijlter.org/index.php/ijlter

25

over time. Portfolios represent connections made between actions and beliefs, thinking and doing, and evidence and criteria. They are a medium for reflection through which the builder constructs meaning, makes the learning process transparent and learning visible, crystallises insights, and anticipates future direction.” (Jones & Shelton, 2011, p.22) Despite the different definitions of portfolio which depend on their purpose, objective and nature, certain theories consider portfolio as a personal ownership, personal record structured collection of material and so on. According to Paris and Ayres (1994), “The overarching purpose of portfolios is to create a sense of personal ownership over one’s accomplishments because ownerships engender feelings of pride responsibility and dedication (p.10).”

McKimm (2014) defined teaching portfolios in her book ‘A Handbook for Teaching & Learning in Higher Education’ as “A teaching portfolio (sometimes referred to in this chapter as a ‘portfolio’) is a personal record of achievement and professional development as a university teacher. It is a carefully selected and structured collection of material that may demonstrate a level of attainment, a range of skills and activity, and/or progression.(p.473)” Kayler (2004) states that: “Teacher education institutions need to make the often-invisible professional growth of classroom teachers more visible to other teachers and programs as a way to foster and expand the teacher knowledge base (p.267).”

According to Fernández (2014), portfolios play a significant role in stimulating and inspiring teachers as well as students. Learning becomes more effective and motivating with a portfolio since the students take ownership of their learning process and develop strategies to plan, monitor and evaluate their learning.

Teachers benefit from portfolios in their professional development since portfolios promote autonomous learning, and autonomous learners are often motivated (Fernández, 2014). Portfolios can also be considered as a source of data, whether qualitative or statistical, and are an important tool for evaluation.

In terms of assessment, Belgrad, Burke and Fogarty (2008) consider that portfolios (and e-portfolios) can become excellent tools for both, formative and summative assessments. Martin-Kniep (1999) considers portfolios as an instrument that recounts the account of the educator whose objective is to make guidance and evaluation rehearses something very similar and focused on critical learning results. The portfolio uncovers the educator’s extraordinary capacity to apply educational plan and appraisal-related ideas and abilities to the advancement of exercises and evaluations that target obviously characterised results. The instructor has provided all drafts of appraisals, enabling the user to notice the changes and upgrades made to apply plan standards and better adjust educational program, guidance and evaluation. The teachers can utilise their portfolios to report their expert history, the inquiries and worries that drive their

http://ijlter.org/index.php/ijlter

26

work, their endeavours to plan or carry out better or diverse learning encounters for understudies, their investigation of basic issues with at least one understudy or the settings that encompass them, their expert work with partners, local area individuals or bosses and their examination of and reflection on proficient assets or critical writings. Notwithstanding the specialised parts of portfolios, it should be remembered that the “essential crowd for an educator portfolio is the instructor who readies the portfolio. Simultaneously, the portfolio can help instructors screen the change from the exercise as-wanted to the exercises educated, to the exercise as-learned. At last, portfolios can help screen educators’ endeavours to fuse new information and abilities into their expert collections. Albeit quite a bit of this work is a private undertaking, educators, as different experts, need to take part in a discussion with their partners to really comprehend and approve their work. Portfolios can fill in as a springboard for such conversations.” (Martin-Kniep, 1999, p.15) This is very well explained in ‘The Portfolio Connection: Student Work Linked to Standards’, in which Belgrad et al. (2008) states that: “Portfolios as the outcome of a continuous, student-centred process help students learn in a variety of ways. One remarkable by-product of the portfolio process in the classroom is increased student awareness of and responsibility for learning objective (p.15).”

The portfolio presents pre-service teacher’s information that is directly related to the personality, mindset, worldviews, mission and vision approaches that preservice and in-service teachers create for themselves, whether on a personal or professional level. Portfolio creation represents the nature of the individual’s personality, his/her level of knowledge and the goals that he/she wishes to achieve in his/her professional field.

This approach will help teachers in their awareness of professional development.

They will manage to present the goals for their work and the path that they will follow. Moreover, portfolios will act as an open statement that they will follow to be more successful. In this way, they manage to develop critical thinking about themselves and their profession and begin to be more creative regarding their plan to achieve their professional goals. They also become aware of a sustainable professional investment that will serve their work in the long term. The portfolio also presents the skills that pre-service teachers have built up during their studies and becomes a bridge between them and the work that they will aspire to have, which enables teachers to have personal and professional competence.

Through reflective learning, pre-service teachers can learn from their previous experiences either in cognitive or practical terms, enabling them to make decisions for change and professional development. Pre-service teachers can also share information with their peers and obtain feedback from their colleagues, thus having an excellent opportunity to be open to others, share ideas and collaborate in the professional field.

http://ijlter.org/index.php/ijlter

27

Creating a portfolio during the study period enables pre-service teachers to establish a culture of presenting ideas, approaches and understandings in the personal and professional field. By establishing strategies for presenting their work in a broader professional environment, they continue to present their work in the same way when they look for a job or even after they are hired. For teachers in the portfolio, a service is a great tool that can serve them for the evaluation performed either by their institution or even at a wider level.

Therefore, in terms of teacher professional development, regardless of in-service or pre-service teacher training, portfolio is a considerable tool for documentation, assessment and evaluation of instructors’ work. They can be pre-designed and pre-structured to collect and document information as required for a specific task or profession, i.e., pre-service or in-service teaching. Portfolios enable teachers to document the evolution of different areas of their work, such as curriculum and assessment development, professional development, inquiry/research and teaching and learning in general.

When used by students, portfolios are developed to display materials that represent the abilities and accomplishments during a specified period. According to Henderson, Davis & Day (2015), students’ presentation of materials over an extended time helps to chronicle their improvement and competence that cannot be measured by paper and pencil tests. One of the primary purposes of a portfolio is to provide a more reliable and comprehensive picture of student growth and achievement.

By using pre-service teachers’ portfolios, instructors will have better opportunities for their personal and professional development and will create a professional culture for their work with children. In Kosovo, as in several other nations, one of the vital competencies of instructors is to reflect on the work they do, to archive their completed exercises and to show the goals they wish to attain in a given period. Typically, the employment of a portfolio plays a critical role in teacher instruction in Kosovo or any other nations because it makes a difference in creating the key competencies for educator teachers and pre-service instructors (Beka, 2015).

According to the implemented laws and educational policies, all the pre-service and in-service teacher trainings in Kosovo are organised by public universities, particularly by their faculties of education (Ministry of Education, Science and Technology [MEST], 2011). The Faculty of Education of the University of Prishtina, being the oldest one, carries out the most significant role in teacher training. It provides pre-service and in-service teacher programs from undergraduate to doctoral level (University of Prishtina, 2004).

As part of the Strategic Development Plan of the Faculty of Education, improvement of quality teaching programs and teacher professional development are considered crucial factors for enhancing our education system (Faculty of Education, 2013). Professional portfolios of pre-service teacher are becoming now an integrated part of the teaching curriculum in all of its study programs. The Faculty of Education has had in the past few years several in-service training

http://ijlter.org/index.php/ijlter

28

programs for teachers’ professional development besides the regular study programs. Those programs have been supported by the MEST and different donors-education partners aiming to advance teachers’ professional development. Examples of these programs are: Advancement of Qualifications of Teachers Project, Capacity Building for Teacher Mentors, Re-Qualification of Pedagogical Student Graduates, and so on. So far, in- service training programs have not been part of the systematic policy of Teacher Professional Development (TPD). Rather, they were a need for the re-qualification of teachers in their teaching degree or in the new teaching strategies (MEST, 2015). With the new strategic documents in place, in-service training is expected to play a vital role in TPD, leading to teachers’ career advancement (MEST, 2016). A professional portfolio is a required tool from the pre-service studies as it will present ideas, professional goals and the ways of achieving these goals in a period of time. It will help teachers to self-reflect on what they have learned and how to improve their professional practice. Reflecting on their accomplished work is significant for the role and the well-being of the activities that they should perform on campus and in internships. However, not all students find the portfolio relevant to their professional development. Junior students treat it more as a task that they should complete during their studies, whereas senior students understand the importance of the portfolio and view it as a tool that will help them to enhance the quality of their work and to promote continuous professional development.

At the same time, senior students consider the portfolio as a way to achieve the goals that they set for professional development.

3. Research methodology This research paper presents the current state of usage of portfolio as a tool for teachers’ professional development in pre-service teacher training at the Faculty of Education of the University of Prishtina. The research was conducted using qualitative methodology. Interviews were organised with students and graduates of the Early Childhood Department-Faculty of Education, University of Prishtina.

Students were randomly selected; every fifth student was selected for each year, while alumni were purposely selected due to their easier access and the possibility of their inclusion in this research. Five students were selected for each year of studies, starting with the second year of studies, third and fourth, as well as five graduates. A total of 20 respondents were included in this survey.

The interviews were organised with each student and graduate separately. Each of the interviews lasted approximately 45-50 min. Students and graduates have voluntarily participated in the interviews. The difference between sophomores and third- and fourth-year students was that sophomores felt more limited in giving answers due to their beliefs about the portfolio and its relevance – reflection, while third- and fourth-year students felt more confident in giving answers since they already had a clear portfolio effect on their professional development as it was their reflection on the quality of the work they did. The graduates were clear about the importance of reflection through the portfolio.

With the permission of students and graduates, all interviews were recorded and transcribed. The final transcription report was provided to the participants for verification and signature. By signing these reports, they have confirmed that

http://ijlter.org/index.php/ijlter

29

their answers were correct and the meaning they wanted to give has not changed through these statements.

The research was conducted to answer the following research questions: 1. ‘How will a portfolio help pre-service teachers in their professional development’?

2. ‘How do pre-service teachers understand the importance of a portfolio at different stages of studies’?

3. ‘What impact does pre-service teacher alumni portfolio have on instructors’ employment’?

4. ‘In what way does self-reflection orient pre-service teachers towards motivation for professional development’?

5. ‘How does self-reflection affect in-service teachers for continuous professional development’?

6. ‘How appropriate is the portfolio as a tool to develop self-reflection in preservice teachers’?

The questions asked in the interview were semi-structured and intended to obtain the students’ opinions on the following: • The importance of portfolio preparation and reflection for pre-service preschool teachers; • The benefits of portfolio preparation; • The portfolio’s significance for professional development; and • The portfolio’s importance in employment.

4. Research outcomes 4.1 The importance of portfolio preparation and reflection for pre-service preschool teachers Based on interviews regarding students’ opinions on the preparation of portfolios, it is evident that learners had very different perceptions. Since the concept of portfolio creation is perceived differently by students, varying in various levels of study, second-year students are found to be more inclined to viewing it as a chore that should be checked off of their list of obligations; “I spend a lot of time preparing my portfolio…it takes hard work.”

Nevertheless, third-year students seem to understand its importance in their professional upgrading. They begin to consider portfolios as a useful tool which they will continue to use at their jobs even after graduation. A third-year student stated that: “In the beginning, I thought it was not important but now it seems to be very helpful.”

By contrast, fourth-year students or alumni do not view portfolios as a task but instead as a tool for summarising and evidencing works and professional ideas.

Similar to the previous statement, a fourth-year student declared that: “By using the portfolio, I know what I’m doing…selection of materials is important for ongoing development.”

http://ijlter.org/index.php/ijlter

30

Meanwhile, the alumni expressed that: “The portfolio is a necessary tool as it helps to reflect on professional development.”

Based on these answers, it can be concluded that the portfolio is seen as an exceedingly important tool, necessary for the ongoing professional development of graduates who have entered the teaching profession.

4.2 The benefits of portfolio preparation There is a discrepancy between second-year students and other years students’ opinions regarding the benefits of preparing a portfolio. A second-year student stated that: “Portfolio helps me to get a good grade.”

Meanwhile, third-year students stated that: “The portfolio helps me recognise my weaknesses…”

This attitude is directly related to the reflection of the pre-service teacher related to his/her work. Through portfolios, students manage to self-evaluate their work and professional development. On the other hand, a fourth-year student notes the portfolio’s importance as an opportunity for the development of critical and creative thinking. In this regard, she declares that: “The portfolio helped me to be a critical thinker in my work and professional development. By using the portfolio, I can easily identify my strengths and my weaknesses.”

Moreover, an alumni student considers portfolios as a tool that helps her to focus on completing her objectives. In this regard, she stated: “The portfolio has helped me become more efficient, improve my quality of work in planning and my cooperation with colleagues.”

Based on the statements provided by the students, it has been concluded that second-year students consider the portfolio as a means to achieve the highest grades, while third-year, fourth-year and alumni students view it as an essential implement for professional development. Their opinions have developed precisely from their experiences with portfolio preparation. Self-reflection on the professional development of pre-service teachers is more witnessed among the fourth-year students than the students of previous years. They have already realised the importance of reflecting on their work, and through this, they begin to make ongoing plans for their professional development. The reflection itself is even more emphasized among the graduates. Through statements, they have emphasized that through this form, they become more efficient and focus to a greater extent on the quality of their work. Reflections has allowed them to improve their professional practices and knowledge, thus making them better prepared teachers.

4.3 The portfolio’s significance for professional development Students have expressed the significant contribution of portfolios to their professional development as well as their reasoning behind it. Even in the case of

http://ijlter.org/index.php/ijlter

31

second-year students, it is evident that their lack of experience makes them more inclined to consider portfolio as a task through which they can learn. At this level, they have not yet fully established the importance of portfolios in their personal and professional development. Therefore, by continuing to think that the employment of portfolios enables them to learn new things and compare their work with others, the portfolio becomes more appealing. A second-year student declared: “I do research so that I can have a good portfolio, which in return will help me improve my work. The portfolio also helps me to compare my work with others.”

Meanwhile, the opinions of third-year students begin to present a higher level of advancement regarding the importance of portfolios in their professional development. Hence, third-year students are more inclined to consider it as an encouraging tool for cooperation with others and as something that helps them feel secure. In this regard, a third-year student stated: “The portfolio has helped me to cooperate with others. Cooperation is a job-requested skill…”

In contrast to the second and third-year students, the opinions of the fourth-year students hold the portfolio as a source of new ideas, an instrument that provides them with security, and more importantly, an opportunity for self-evaluation. A fourth-year student declared: “The preparation of portfolios has given me more confidence and I do better quality work. I am more confident in what I do” This allows for a better understanding of the fourth-year students’ view because through their experience, whether they have been taught on campus or internships, they have found the portfolio to have a significant impact on their preparation for the teaching profession. Therefore, their opinions at this stage are also more constructive and based on the achieved results. Meanwhile, graduates see the importance of the portfolio as an evidence of their qualitative work with which they can secure their job more easily, which encourages them to apply this kind of assessment with the children involved in their work. In this regard, an alumni declared: “The portfolio is the summary of my work. Additionally, the portfolio has enabled me to get a job at a private preschool.”

Based on the students and alumni’s answers, we can conclude that portfolios are a critical tool in the professional development of pre-service teachers. By using portfolios, pre-service teachers understand goals and ways to achieve those goals.

The portfolio is a testimony of the acquired knowledge and skills of pre-service teachers, which consequently enables them to focus on their professional development.

4.4 The portfolio’s importance in employment Due to the portfolio being one of the most important tools for their professional development, students have shared their opinions on the importance of the portfolio in their employment phase. This question has been answered by third-

http://ijlter.org/index.php/ijlter

32

year students, fourth-year students and alumni since their chances of obtaining a job are higher than those of second-year students. The students have described their opinions about the professional preparation that they have achieved through the development of portfolios. They explained how much the preparation of portfolios has helped or will help them to find employment in their professional field. Third-year students have said that the portfolio has helped them to love their future profession and at the same time has helped them to develop presentational skills. As a result, they can better present their work, and according to them, these are the benefits that will give them advantage over others. One of the opinions of the third-year student in this regard was: “The portfolio has helped me to further develop my presentational skills as I had to present my work. It has helped me to love my future profession more…”

A fourth-year student stated that throughout the portfolio preparation process, they are taught to organise and document their work, which is of great advantage to all those who work. The student also said that developing portfolios requires creativity and the use of new ideas and different materials, especially those that are recycled so that the portfolio is more meaningfully presented to others.

“The portfolio helped me understand that it is necessary to be organised at work… Portfolio preparation pushed me to be more creative and to use recycled and reusable materials.”

Given the above-mentioned characteristics, students are convinced that they will find jobs more easily since those aspects are required in the first interview.

Graduates have stated that during their studies, in certain cases, they did not consider portfolio as useful work. However, when they applied for work, they realised that a portfolio is an extraordinary tool used to present their professional preparation: “In the beginning, I thought I am doing work that nobody appreciates, but now I see that the portfolio is an amazing testimony. Each time I look at my portfolio, I feel like a teacher, and I will create a portfolio for my preschool students.”

Based on the interviewers’ opinions, we can conclude that the portfolio also plays an important role in pre-service teacher employment. Through portfolios, preservice teachers can easily prove their professional development and present their work to children. It is of utmost importance that they apply the same kind of assessment to pre-school children; the application of formative assessments and the documenting of the level of childhood achievement. Furthermore, they are able to document the ideas, approaches and goals related to their teaching profession.

5. Conclusions According to the research data, portfolio is a siginificant factor in educators’ professional development. This instrument provides future teachers with the opportunity to learn to self-reflect and to simultaneously organise their work better. To create skills in view of collaborating with others directly and to reflect

http://ijlter.org/index.php/ijlter

33

by analysing and comparing the work of each-other affect the development of professional competencies for teachers.

Furthermore, through the literature review and the results of the research, it is noted that the preparation of portfolios enables students to self-evaluate their knowledge, their skills and the need for further professional development and to assess others’ work. Through self-evaluation, learners can advance their approach and achieve a high level of professional performance. Using feedback and reflection to modify teaching practice in the early stages of one’s career as a teacher educator is a fascinating and fruitful process (Hamilton, 2018). We also note that portfolios increase pre-service teachers’ self-confidence, which improves their quality of work.

Portfolios develop creative and critical thinking, decision-making skills and communication and presentation skills since one must present one’s work to others and receive and give feedback. As a result, teachers become prepared and competent for their future assignments. They would have benefited from a group discussion about the school’s strategy work as well as a brief reflection on the lessons learned (Körkkö et al., 2020).

Besides, a teacher who has prepared his/her portfolio is interested in doing the same thing with children. Documenting children’s development is important and highly appreciated by parents. Also, encouraging children at a young age to reflect on themselves and their work instructs them to think critically and creatively. Consequently, they will employ a more positive approach to themselves and others.

6. References Baumgartner, P. (2009). Developing a taxonomy for electronic portfolios. Studien Verlag.

Beka, A. (2015). Sustainability in higher education: Linking teacher education with labor market in Kosovo. In J. P. Filho (Ed.), Challenges in higher education for sustainability (p. 189-197). Springer. https://doi.org/10.1007/978-3-319-23705-3_8 Beka, A., & Gllareva, D. (2016). The importance of using electronic portfolios in teachers work.

Applied Technologies and Innovations, 12(1), 32–42. http://doi.org/10.15208/ati.2016.03 Belgrad, S. F., Burke, K. B., & Fogarty, R. J. (2008). The Portfolio Connection: Student Work Linked to Standards (3rd ed.). Corwin.

Ciesielkiewicz, M. (2019). The use of e-portfolios in higher education: From the students’ perspective.

Issues in Educational Research, 29(3), 649–667. http://www.iier.org.au/iier29/ciesielkiewicz.pdf Cimermanová, I. (2019). Teaching portfolio as a source of pre-service teacher training programme needs analysis.

Pedagogika, 131(3), 201–216. https://doi.org/10.15823/p.2018.42 Dilger, B. (2007). Der selbstreflektierende lerner [The self-reflective learner] (1st edition). EuslVerlag, Paderborn, Germany.

Driessen, E. (2017). Do portfolios have a future? Advances in Health Sciences Education: Theory and Practice, 22(1), 221–228. https://doi.org/10.1007/s10459-016-9679-4 Faculty of Education. (2013). [Fakulteti i Edukimit]. https://edukimi.uni-pr.edu/

http://ijlter.org/index.php/ijlter

34

Fernández, D. M. (2014). "Integrating the European portfolio in a competency-based teacher education approach". In English as a foreign language teacher education, (Vol. 27, pp. 35-38). Brill | Rodopi. https://doi.org/10.1163/9789401210485_005 Häcker, T. (2009). Wurzeln der Portfolioarbeit [Roots of portfolio work]. In T. H. I. Brunner (Ed.), Das handbuch portfolioarbeit [Portfolio handbook] (pp. 27–32). FriedrichVerlag Hamilton, M. (2018). Bridging the gap from teacher to teacher educator: The role of a teaching portfolio.

Studying Teacher Education, 14(1), 88–102. https://doi.org/10.1080/17425964.2017.1414041 Henderson, J. L., Davis, R. R., & Day, T. M. (2015). Developing Portfolios to Document Student Performance and Accomplishments. NACTA Journal, 59(4), 350-353.

Jones, M., & Shelton, M. (2011). Developing your portfolio enhancing your learning and showing your stuff. Routledge.

Kayler, M. A. (2004). Portfolio assessment and teacher development. Academic Exchange Quarterly, 8(3), 265-274.

Kerr, R. (2007). Portfoliomappe Selbstdisziplin [Portfolio self-discipline]. Beltz Verlag.

Körkkö, M., Kotilainen, M.-R., Toljamo, S., & Turunen, T. (2020). Developing teacher inservice education through a professional development plan: modelling the process.

European Journal of Teacher Education, 1–18. https://doi.org/10.1080/02619768.2020.1827393 Lynch, M. (2000). Against reflexivity as an academic virtue and source of privileged knowledge.

Theory, Culture and Society, 17(3), 26–54. https://doi.org/10.1177/02632760022051202 Martin-Kniep, G. O. (1999). Capturing the wisdom of practice: Professional portfolios for educators. Alexandria, VA: Association for Supervision and Curriculum Development.

McKimm, J. (2014). Assuring quality and standards in teaching. In H. Fry (Ed.) A Handbook for Teaching and Learning in Higher Education: Enhancing academic practice (4th ed.).

Routledge.

MEST (2011). Ministry of Education Science and Technology. Law on higher education. https://masht.rks-gov.net/en/legjislacioni MEST (2015). Ministry of Education Science and Technology. http://masht.rks-gov.net/ MEST (2016). Ministry of Education Science and Technology. http://masht.rks-gov.net/ Paris, S. G., & Ayres, L. R. (1994). Psychology in the classroom: A series on applied educational psychology. Becoming reflective students and teachers with portfolios and authentic assessment.

American Psychological Association. https://doi.org/10.1037/10158-000 Riebenbauer, E., Dreisiebner, G., & Stock, M. (2017). Providing feedback, orientation and opportunities for reflection as key elements for successful mentoring programs: Reviewing a program for future business education teachers. Global Education Review, 4(4), 54–69. https://ger.mercy.edu/index.php/ger/article/view/255 Slepcevic-Zach, P., & Stock, M. (2018). ePortfolio as a tool for reflection and self-reflection.

Reflective Practice, 19(3), 291–307. https://doi.org/10.1080/14623943.2018.1437399 Stock, M., & Winkelbauer, A. (2012). ePortfolio-implementation in the master’s program of business education and development. International Journal of Business Education, 152, 48–55. https://www.ijbe.online/2012-eportfolio-implementation-in-themasters-program-of-business-education-and-development.html Trávníčková, P., & Puhrová, B. P. ů. (2019). Professional Portfolio of the Preschool Teacher: What Is the Real Content? Advances in Intelligent Systems and Computing, 156–162. https://doi.org/10.1007/978-3-030-20135-7_15

http://ijlter.org/index.php/ijlter

35

University of Prishtina. (2004, 10, 21). [Universiteti i Prishtines]. https://edukimi.unipr.edu/ Webster, R. S., & Whelen, J. D. (2019). The Importance of Rethinking Reflection and Ethics for Education. Rethinking Reflection and Ethics for Teachers, 1–12. https://doi.org/10.1007/978-981-32-9401-1_1

http://ijlter.org/index.php/ijlter

36

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 36-60, February 2021 https://doi.org/10.26803/ijlter.20.2.3

The Efficacy of Habits of Mind in the Inculcation of Self-Directed Learning Skills in Pre-Service Teachers Gideon Petrus Van Tonder*, Byron John Bunt, Alvine Petzer, Hermann Davin Bosch, Nicolaas Van Deventer, Amanda Gerber and Lizelle Van Schadewijk North-West University, Hendrik van Eck Blvd, Vanderbijlpark, South Africa https://orcid.org/0000-0001-5486-1242 https://orcid.org/0000-0002-2102-4381 https://orcid.org/0000-0002-4679-0189 https://orcid.org/0000-0002-2486-7624 https://orcid.org/0000-0003-4900-7994 https://orcid.org/0000-0002-7532-2448 https://orcid.org/0000-0002-1942-1433 Abstract. The purpose of this research was to determine the extent to which pre-service teachers were self-directed before implementing habits of mind as a teaching strategy. The researchers purposively focused on first to fourth-year BEd students. The sample consisted of 277 students who volunteered. A concurrent mixed-method triangulation approach was applied. Even though the students overestimated themselves in Williamson's self-rating scale of self-directed learning, the habits of mind positively affected developing various aspects of the self-directed learning process. Pre-service teachers recognised the importance of applying habits of mind, which will help them use their new knowledge in classroom practices and promote self-directed learning. Therefore, it is recommended that the future training of pre-service teachers be scaffolded around the habits of mind that will contribute to developing the teachers' ways of thinking, self-directed learning, and competencies. In addition, several habits of mind will support teachers in utilising and/or adjusting their prior learning to be open to or contribute to the construction of new knowledge. It is also recommended that habits of mind be infused into the current BEd curriculum in order to foster self-directed learning.

Keywords: self-directed learning; digital online learning; habits of mind

*

Corresponding author: Gideon Petrus Van Tonder; Email: deon.vantonder@nwu.ac.za

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

37

1. Introduction The changeable and demanding twenty-first century world requires not only the teaching of knowledge, but also the nurturing of thinking skills and particularly welldeveloped dispositions – in short, behavioural and thought competencies (Costa & Kallick, 2009; Eyre, 2016; Pink, 2006). The world faces interconnected global challenges that require global solutions. It is, therefore, essential that teaching moves beyond the development of individuals who can read, write, and count. Teaching has to be innovative, create universal ideals, foster an active concern, and be accessible to everyone (Barell, 2010; Costa & Kallick, 2009; Eyre, 2016; Gardner, 2006; Serdyukov, 2017).

Teacher-training institutions face a challenge to turn the tide and make teacher training more effective so that teachers will be able to provide cognitive education to learners at schools in conjunction with what the twenty-first century expects of learners. These school leavers ought to be creative thinkers and problem-solvers equipped with tools that can be utilised whenever they learn, think, and make decisions. South African teachers should acquire practical tools and strategies suitable for the South African teaching-and-learning context in order to transform classroom practices. This means pre-service teachers (students) can be guided and supported to actively and specifically initiate the thinking skills necessary to achieve the goals of the Curriculum and Assessment Policy Statement (CAPS). CAPS, which guides the implementation of education in South Africa, requires learners to “organise and manage themselves and their activities responsibly and effectively” as part of the critical cross-field outcomes (Department of Basic Education, 2011, p.5).

These outcomes require that teachers' classroom practices should promote selfdirected learning in all subject fields.

According to the researchers, self-directed learning is a deliberate learning process that is designed and evaluated by the learner. The researchers’ view self-directed learning as imperative for tertiary education, and thus the focus of this article is to take the initiative to improve self-directed learning using the 16 habits of mind and digital online learning in tertiary education. Brookfield (2009) describes self-directed learning as the learning method with which individuals take the initiative to identify their learning needs, define learning objectives, determine human and material resources for learning, select and adopt appropriate learning strategies, and analyse learning outcomes, with or without the guidance of others.

Habits of mind are groups of intellectual resources that must be developed purposefully and intentionally. When individuals are faced with challenging tasks, these resources ensure that the tasks are completed successfully and effectively and that high-quality results are reached. These habits are not learned and then forgotten – a habit develops from regular practice. Effective habits of mind become behaviours that are activated automatically and predictably without considerable effort and describe what thoughtful people do when faced with complex challenges (Costa & Kallick, 2014).

http://ijlter.org/index.php/ijlter

38

This article explains why the researchers undertook this study by elaborating on the theoretical and conceptual framework that underpin this study. Furthermore, the research methodology, data collection methods and analysis and the outcomes of the mixed-method approach are discussed. Thereafter, recommendations for future research, the limitations of the study and the conclusion are discussed.

2. Problem statement A school-wide community-of-practice project was launched by focusing on teaching various strategies to develop self-directed learning among first to fourth-year BEd pre-service teachers in different subjects within the School of Commerce and Social Studies in Education. The habits of mind teaching strategy is not a widely known or implemented approach to developing teacher education programmes. This community-of-practice project was launched in 2020 by the corresponding author, Van Tonder, within the School of Commerce and Social Studies in Education at the North-West University.

The COVID-19 pandemic affected the teaching and learning mode of delivery, which changed to remote online learning across the globe. During the COVID-19 pandemic, large-scale national attempts to use technology in support of remote learning (asynchronous), distance education (asynchronous and synchronous) and online learning (synchronous) were emerging and developing rapidly. Romero-Ivanova et al. (2020, p. 81) explain that due to COVID-19 new expectations developed during the first semester and transformed the lives of individuals towards a "new normal".

Higher education institutions with lecturers and students transitioning to online synchronous and asynchronous teaching and multimedia activities were impacted by these changes (Romero-Ivanova et al., 2020). Self-directed learning is a significant field of study for twenty-first century educationalists, and according to Holtz (2017), the self-direction for which teachers are preparing our learners is an essential skill.

However, the inherited classroom culture is not planned around self-direction and tends more towards compliant consumption, an “un-flipped paradigm” (Holtz, 2017), in which learners record what teachers say and redefine it through tests, assignments and essays to indicate understanding. The problem occurred when the school-wide community-of-practice project was launched during this “new normal” to prepare pre-service teachers with the essential skill of self-direction.

With the above in mind, the primary research question for this study is the following: What are the consequences of a school-wide community-of-practice project focusing on various teaching strategies on the development of self-directed learning among first to fourth-year BEd pre-service teachers in different subjects within the School of Commerce and Social Studies in Education?

The secondary research questions of this investigation were as follows: • How self-directed was the first to fourth-year pre-service teachers before the intervention?

• What role did the habits of mind play in developing self-directed learning?

http://ijlter.org/index.php/ijlter

39

•

What are the participants' perceptions regarding the development of self-directed learning through the use of habits of mind?

The primary aim of this article focuses on habits of mind as one of the various teaching strategies on the development of self-directed learning among first to fourth-year BEd pre-service teachers in different subjects within the School of Commerce and Social Studies in Education.

The secondary objectives of this investigation were as follows: • To determine the self-directed learning levels of the first to fourth-year preservice teachers before the intervention.

• To established what role the habits of mind played in developing self-directed learning.

• To explore the participants' perceptions regarding the development of selfdirected learning through the use of habits of mind.

The article subsequently focuses on the theoretical and conceptual framework adopted in the research.

3. Theoretical-conceptual framework 3.1 Theoretical framework Vygotsky's (1980) approach of the social constructivist theory was used as a lens to explore the impact of the Zone of Proximal Development (ZPD) and scaffolding of learning with students with the focus on various teaching strategies on the development of self-directed learning. According to Shabani, Khatib and Ebadi (2010), the primary purpose of scaffolding (techniques used to move students progressively toward more robust understanding and, ultimately, greater independence in the learning process) in teaching and learning is to assign responsibility for the assignment to the student. Self-directedness, however, does not mean learning is individualised, and an adult learner should work in isolation (Brookfield, 2009). The project focused on social constructivist theory, which relates to collaborative learning and social interaction to intensify self-directedness (Geduld, 2014). Individuals develop knowledge through social experiences and mutual learning, which increase cognitive levels (Bozkurt, 2017).

This research concentrates on the following key concepts: self-directed learning, digital online learning and habits of mind.

3.2 Conceptual background 3.2.1 Self-directed learning Self-directed learning has been researched, theorised, and practised for over 45 years (Knowles, 1975). Brockett and Hiemstra (2012), as well as Tough (1978), have classified self-directed learning as a personal attribute that allows individuals to be independent and highly self-directed in their learning, or as a process with which to organise instruction. As a personal attribute, self-directed learning refers to individual predispositions towards this type of learning and autonomous

http://ijlter.org/index.php/ijlter

40

engagement in the learning process. Self-directed learning is a learning approach that the learner controls. Knowles (1975, p. 18) defines self-directed learning as “a process in which individuals take the initiative, with or without the support of others, to identify their learning needs, formulate learning goals, identify human and material resources for learning, select and implement appropriate learning strategies, and assess learning outcomes”.

Self-direction is an ongoing exercise of reliable control by the student over all learning-related choices, as well as the skill to gain admission to, and choose from, a full range of available and appropriate resources (Knowles, 1975).

Self-directed students exhibit several specific and observable characteristics. Intrinsic motivation, the capacity to choose personal goals, self-discipline, self-assessment ability, and metacognitive skills are key features of self-directed students (King, 2011). Intrinsic motivation is the force that inspires students to pursue self-directed, independent learning. One of the most relevant basic educational objectives could be to build circumstances that contribute to intrinsic motivation and a mindset of selfdirected learning. Rogers (2004) recommends that self-directed students pursue their own learning by defining what they need to know and how to do so by planning and monitoring their learning through a variety of tools and strategies and by working with peers and teachers to assist in their learning.

Self-directed learning has been examined from distinct perspectives and with distinct techniques and has delivered some promising results. It has been found that selfdirected learning has several benefits: it enhances students' choices, self-confidence, freedom, motivation, and also the development of numerous lifelong learning skills (O'Shea, 2003). In order to encourage self-directed learning, it seems like several diverse teaching solutions can be used (Aleman, de Gea & Mondéjar, 2011; Horne et al., 2007). There is also sufficient evidence of the quality that shows a modest improvement in knowledge, but there is little difference in student skills or attitudes when comparing self-directed learning and conventional methods of learning (Murad et al., 2010). The purpose of this paper is to review the implication of habits of mind as a teaching strategy on the self-directed learning of the pre-service teachers at North-West University (NWU).

3.2.1 Digital online learning Anyone who has ever worked in a conventional classroom setting as a teacher or facilitator knows first-hand that with different classes or individual learners, the same content will never yield the same results (Shahabadi & Uplane, 2015). In addition, information may be relevant to the learning style of an individual, while the same information may be worthless in fulfilling the learning goals in the case of another individual (Masie, 2002; Zenger & Uehlein, 2001). The researchers can suggest the following argument from this empirical reality and from considering its ramifications for any means of delivering teaching materials through an online platform: in the end, it is the behavioural indicators of students that need to be considered when creating

http://ijlter.org/index.php/ijlter

41

and implementing e-learning programmes to develop self-directed learning (Shahabadi & Uplane, 2015). Consequently, the researchers agree with Codreanu and Vasilescu (2013) that the emphasis is on the student and their needs and requirements, and given the point of this study which is to focus on developing selfdirected learning through habits of mind, it is crucial to evaluate the effect on any programme developed and delivered through digital online learning platforms.

From this point on, the researchers will use the broad term of e-learning.

Today, the bulk of e-learning is asynchronous in nature. Shahabadi and Uplane (2015, p. 132) describe “asynchronous e-learning as a learner-centred process, which uses online learning resources to facilitate information sharing regardless of the constraints of time and place among a network of people”. Asynchronous e-learning has the benefits of computer-mediated communication (CMC) “to achieve the promises of learning anytime and anywhere through asynchronous online discussions, which is based on the constructivist theory, a learner-centred approach that emphasises the importance of peer-to-peer interactions” (Shahabadi & Uplane, 2015, p. 132). The researchers argue that in order to develop self-directed learning in an online environment, the system needs to cater for learner-centeredness, which is embedded in constructivist theory, as alluded to above. The researchers utilised this asynchronous method in this project, by using screencasting or interactive PDFs and PowerPoints of study units, which has been pre-recorded for students. This learner-centred method was difficult to follow due to a lack of infrastructure and experience at the beginning of the semester. However, after gaining experience with these approaches, it became easier to implement.

Comer and Lenaghan (2013) argued that asynchronous online learning offers an excellent probability of building a learning-centred surrounding that stimulates rich interactions between lecturers and students.

3.2.3 Habits of mind Habits of mind are important cognitive resources for effectively completing tasks, coping with challenges, and solving problems. Habits of mind are based on a process that involves mindful thinking (Costa & Kallick, 2008) that increases learners' eagerness to act with intellect when faced with problems, and it may lead to answers that are not immediately apparent.

The term 'habits of mind' was first introduced by Costa (1985) in his article on hierarchy of thinking and further established by Marzano (1992) and Marzano, Pickering and McTighe (1993). Habits are characterised as almost instinctive behaviours that create a mindset for problem solving while not requiring a particular form of thinking to be applied. Habits have also been described as a collection of dispositions learned and practised before a person can almost effortlessly accomplish a specific task (Costa & Kallick, 2000). Costa and Kallick (2000) proposed 16 habits that could empower students to make intelligent decisions both inside and outside the classroom, while Marzano divided habits of mind into three categories. These three categories are (i) self-organisation, (ii) creative thinking, and (iii) critical thinking (Marzano, 1992). According to Costa and Kallick (2008), mental patterns are

http://ijlter.org/index.php/ijlter

42

a combination of many desired abilities, habits, signals, past experiences, and ambitions. This means that there exists an empirical behavioural pattern that an individual uses at a particular time. Being able to think intelligently was important, as it allowed students to find ways to address specific problems. Students should therefore be conscious of their behaviour and should have habits of mind in order to solve problems (Gloria & Indriyanti, 2017).

Several studies related to habits of mind have been conducted. These studies developed different developmental lessons that could shape or strengthen habits of mind. Yandari’s study (2019) emphasised that the habit of mind's strength is students' ability to practice the skill of determining solutions to a problem, so habits of mind can be interpreted as having an impact on problem-solving skills. Research by Sriyati, Rustaman and Zainul (2010) indicated that formative assessment led to the development of habits of mind. Additionally, it also clarified that habits of mind could be learnt. Hew and Cheung’s research (2011) found that online learning could explore self-awareness and open-mindedness as indicators of habits of mind. Various researchers provide their views of the strengths of the 16 habits of mind with regards to improving cognitive development and self-directed learning. A short review of the 16 habits of mind follows.

1. Persisting: never give up.

Persistence means trying something else when one encounters an obstacle instead of simply giving up. In a continuously changing education environment, teachers have to lay a foundation for persistence that encourages learners to focus on work at hand and to persevere in spite of distraction to ensure that the work gets done (Adeyemo, 2003; Seif et al., 2011). Feuerstein and Hoffman (1995, p. 33) developed the elements of a plan as a tool for persistence, which supports learners who find it difficult to use this tool and following the next steps in order to continue: • Step 1: Define my goal.

• Step 2: Look at what I have/know.

• Step 3: What strategy/skill do I need to apply?

• Step 4: Where will I start?

• Step 5: What are the rules/criteria?

• Step 6: Check my work 2. Managing impulsivity: taking time to deliberate before acting.

According to Costa and Kallick (2008, p. 274), managing impulsivity means that you “[t]ake your time. Think before you act. Remain calm, thoughtful, and deliberate”. It is therefore not a reference to working faster, but rather more determinedly and thoughtfully. Al-Assaf (2017) maintains that managing impulsivity is the ability to contemplate and listen to alternative points of view and instructions, to consider and think before creating a vision or establishing a work plan for a task. Further, it includes the ability to reflect and focus on options and results from various perspectives, and then constructing plans to comply with the task and to delay making decisions. This method often implies the deliberation and analysis of

http://ijlter.org/index.php/ijlter

43

problem-solving before making decisions on a particular idea. People must also consider multiple alternatives and findings before they realise that they understand the dimensions of such problems entirely.

3. Listening with understanding and empathy: understanding the viewpoints of others.

Human beings spend about 55% of their lives listening, but listening is one of the least taught skills in schools (Costa & Kallick, 2009). Beall et al. (2008, p. 130) suggest that teachers should be “aware of listening and trained to be better listeners to begin making a difference in learners' listening competence”. Possible strategies the researchers envisage to improve students' listening skills are to paraphrase what another student has said before they add to what was said or offer their own comments, or to ask students to repeat what was said while they were listening.

4. Thinking flexibly: being mindful of numerous viewpoints and perspectives.

For teachers to think flexibly, they need to look at things in another way, and they should be able to change perspectives and generate alternatives (Seif et al., 2011).

When confronting a situation or addressing a specific issue, the person who thinks flexibly has the opportunity to alter their mental state. This practice often involves looking at old ideas with a fresh vision and creative imagination and proposing different approaches while solving a problem (Al-Assaf, 2017).

5. Thinking about own thinking (metacognition): understanding your own thoughts.

Metacognition is an essential skill that all teachers have to learn. They have to become self-aware of their ways of thinking (Costa & Kallick, 2009). According to Grosser and Brettenny (2014), different techniques can be implemented to improve students' metacognition: let students think aloud not only by presenting responses or solutions to problems, but also by exploring the procedures, steps, and methods used to get a response or solve a problem. Keeping a running record of their thought processes, plans, and actions will help them with metacognition. It allows students to recognise errors and self-correct these same errors by focusing on processes, measures, and strategies. Let students describe what they have accomplished and what has challenged them. Encourage students to get their peers' input.

A variety of questions can promote metacognition: • How do you know if you are correct?

• How do you prove you are right?

• How did you know where to start?

Do not provide students with responses, but instead say: • I assume that you did not read the question well, so your argument is incorrect.

• Read the question once again, please.

• Your addition has to be reviewed.

6. Striving for accuracy: acting according to higher norms.

Learners who strive for accuracy can figure things out, clarify problems, gather data,

http://ijlter.org/index.php/ijlter

44

and recheck information (Adeyemo, 2003). Accuracy is valuable in the classroom and real life. Do not correct students' errors. Indicate 'You have five spelling errors, please correct them.' No task can be submitted unchecked to a lecturer – self-assessment or peer assessment should take place before submission. This can be achieved by instructing students to complete checklists that show how the work was reviewed.

Uncontrolled work should be graded by deducting marks from the final mark. By applying this method, students will learn the skill of accuracy and fewer mistakes might occur (Grosser & Brettenny, 2014).

7. Questioning and problem posing: finding problems to resolve.

According to Costa and Kallick (2009), one of the attributes that defines humans is our tendency for and capacity to resolve issues. Successful problem solvers understand how to pose questions to bridge the gap between what they know and what they do not really know. Kowalski (2009, p. 344) states that “when teaching, the most powerful tool teachers have is the ability to engage the learners by asking questions”.

Learners prefer to speak instead of listening, and asking questions makes it easier to communicate. Questioning enables both learners and teachers to clarify their thinking, and it can challenge learners and teachers to solve complex problems that are in line with the core competencies for the future (European Union, 2019).

8. Applying past knowledge to new situations According to Costa and Kallick (2009), people who are intellectual learn from their mistakes. They can rely on prior experiences when faced with a new and perplexing problem. Ștefӑnicӑ et al. (2017) indicate that subject-specific prior knowledge is the primary predictor for the development of professional competence.

9. Thinking and communicating with clarity and precision: attempts to transmit information correctly.

Clear and precise communication ensures that information is not lost between parties and contributes to a better understanding among different individuals. Without effective communication among stakeholders, learning becomes less structured and effective (Seif et al., 2011). Therefore, effective communication is a crucial skill every teacher should practice to enhance effective communication and collaboration with all stakeholders.

10. Gather data through all senses Most learning relies on learners' auditory and visual senses. Hearing and sight are often viewed as the gateway to learning. However, this narrow view excludes several other senses that can contribute to learners' learning and understanding, such as olfactory (smell), tactile (touch), or kinaesthetic (movement) senses (Costa & Kallick 2009). These other senses may be used when learners work with three dimensional models relating to the subject content, where group projects require building or constructing models (tactile and kinaesthetic) that will contribute to a much richer learning experience for learners.

http://ijlter.org/index.php/ijlter

45

11. Creating, imagining, and innovating: generating modern and ground breaking strategies.

Generating new and innovative methods and solutions are key ways of thinking to support teachers to create positive mindsets. Creativity is a skill that is included in several of the key competencies that learners will need in the future (EU, 2019).

12. Responding with wonderment and awe: being intrigued by the universe.

According to Costa and Kallick (2009), teachers want their learners to be curious, to interact with the environment around them, to consider the transforming of a cloud, to be enchanted by the opening of a flower, and to value the rational of mathematical logic. According to Eisner (1991, p. 11), “[o]ne important aim of schooling should be to create a climate that evokes learners' sense of wonder and inspires their imaginations to soar”.

Further, Eisner (199, p. 115) emphasises that “in classrooms, it also counts to teach the learners the importance of wonder”.

13. Taking responsible risks: take chances without knowing the outcomes.

Costa and Kallick (2009) assert that flexible people seem to have an almost uncontrollable desire to push boundaries. They are uncomfortable with their level of comfort; they live on the edge of their ability. Teachers are expected to take chances and explore new teaching methods in their classrooms (Adeyemo, 2003), but they need to be trained how to do it effectively. High-risk experiences teach individuals that they are far more capable of taking action than they previously believed. Risktaking is often a convergence of intuition, experience, and a sense of taking on new challenges (Vazquez, 2020).

14. Finding humour: create laughing and fun.

Humour has been found to have mental health benefits, according to Costa and Kallick (2009). It encourages higher-level cognitive skills including analysing, discovering innovative relationships, visual imaging, and creating analogies. Lei, Cohen and Russler, (2010, p. 331) found that “the use of humour can increase learners' interest, attention, motivation and comprehension of the course material”.

15. Thinking interdependently: working in teams.

Learners should be able to communicate constructively in different environments, collaborate in teams, and negotiate as part of personal, social, and learning-to-learn competencies that are required for modern educational contexts (EU, 2019).

16. Remaining open to continuous learning: continually learning new things and striving for growth.

According to Costa and Kallick (2009), continuous learning is a constant mode for intelligent people. The pursuit of lifelong learning energises them. Their trust, combined with their intellectual curiosity, enables them to continually explore new and better ways. Teachers should become lifelong learners and be receptive for continuous learning (Hayat et al., 2019).

According to Anderson (2010) and Costa (2009), the only way to get better at applying

http://ijlter.org/index.php/ijlter

46

habits of mind is to become more attentive to situations that call for the application of these habits. Students need to apply these habits more often without the lecturers' direction. The researchers believe that if students commit to practising these habits daily, they can effectively self-assess, self-direct, and self-manage their development of habits of mind without being externally directed.

4. Methodology 4.1 Participants In this study, the researchers used non-probability sampling methods, which means that not everybody in the population has a fair chance of taking part in the research (Maree & Pietersen, 2020). The researchers purposively focused on first to fourth-year BEd pre-service teachers in Business Studies, Economics, Accounting and History from the NWU on the Vanderbijlpark campus within the School of Commerce and Social Studies in Education. BEd pre-service teachers have been chosen because possible deficiencies and weaknesses in relation to the development of self-directed learning could be identified during their studies at higher-education level, and action plans to address these deficiencies and weaknesses during their four years of study can be implemented before they complete their studies and enter their teaching careers. The sample could also be considered convenient, as the participants were located on the same site where the researchers work. An independent person added an announcement on the researchers’ communication sites to recruit participants and informed them about the purpose of the study to ensure that the participants were not confused or misunderstand their involvement in the research. The online questionnaire contained an embedded consent form where the participants were informed that participation was voluntary and that they could discontinue the survey at any time by simply closing their browser. Anonymity and confidentiality were ensured without penalty at all stages. The participants completed the reflections anonymously and were identified by means of numbers. Although 407 students enrolled for the school-based project only 277 of them completed Williamson's questionnaire (the school-based project was compulsory as part of the course; however, it was not compulsory to complete the questionnaire and the reflections on the different teaching strategies). The purpose of this research was to focus on effective teaching strategies which may support the development of self-directed learning.

4.2 Research design The researchers followed a concurrent mixed-method triangulation approach, where both quantitative–descriptive survey-research strategies and qualitative– phenomenological research strategies were employed to collect data to determine whether habits of mind may support the development of the participants' selfdirected learning (Creswell & Plano Clark, 2018). During the implementation of the quantitative method, Williamson's (2007) self-rating scale of self-directed learning questionnaire was distributed to test the participants' self-directed learning. After the implementation of the strategy, qualitative data were collected in the form of the participants' written reflections on the benefits that the habits of mind strategy held

http://ijlter.org/index.php/ijlter

47

to support their development of self-directed learning. An independent person, who had signed a confidentially clause and had no conflict of interest in the project, administered the collection of data.

4.3 Quantitative methods and instruments As mentioned in the previous section, participants were required to complete Williamson's (2007) self-rating scale of self-directed learning (SRSSDL) in higher education questionnaire. The questionnaire comprised 60 items categorised under five distinct areas of self-directed learning, namely awareness, learning strategies, learning activities, evaluation, and interpersonal skills. Statistical analysis included the use of both descriptive (frequency and percentages) and inferential statistics (confidence intervals). Responses for each item were rated by using a four-point scale instead of a five-point Likert scale. The researchers have modified the Likert scale to prevent students from choosing option three in the five-point scale, namely 'sometimes', in order to get clear and precise answers from participants. The modified Likert scale was statistically approved by a qualified statistician from the statistical consultation services of the NWU. 4.4 Qualitative method and instruments Participating students were requested to write reflections on their experiences of the new teaching approach that was incorporated with the purpose to support their development of self-directed learning. They also had to state whether they thought they would be able to apply this approach in their own learning and future teaching.

5. Results Both quantitative and qualitative results are outlined in this section.

5.1 Quantitative results Table 1 indicates the number of participants and the variation of subject areas the participants were exposed to in the different teaching strategies.

Table 1: Subject area Subject area

Frequency

Percent

Accounting

41

14.8

Business Studies

109

39.3

Economics

60

21.7

History

67

24.2

Total

277

100.0

http://ijlter.org/index.php/ijlter

48

The largest group of participants represent the Business Studies subject area (39. 3%).

History is the second largest group (24.2%) with Economics (21.7%) as the third largest, while Accounting had the lowest percentage of participants (14.8%).

According to participants' answers on the Likert-type scale of one to four for the 60 questions, the total scores were determined based on Williamson's questionnaire. The participants were divided into three distinct groups regarding their self-directed learning as perceived by themselves, namely: low (48–112), moderate (113–176) and high (177–240). This grouping was categorised using a standardised statistical procedure. Table 2 shows the number of student participants divided into the three specific groups of self-directed learning according to the questionnaire. This research focused on the results obtained for the moderate and high groups, as there were no participants in the low group.

Table 2: Number of participants grouped according to their level of self-directed learning Frequency

Percent

Moderate (113–176)

36

13

High (177–240)

241

87

Total

277

100

The majority of the participants, according to Table 2, fall into the group that indicated a high rate of self-directing skills before the application of the habits of mind teaching strategy. This was interpreted that the students overrated or overestimated their own self-directed learning abilities.

Table 3: Construct reliability for each section Construct

Questions

Cronbach's alpha

Mean

Standard deviation

Awareness

Q1.1–Q1.12

0.755

40.15

4.24

Q2.1–Q2.12

0.714

38.60

4.33

Q3.1–Q3.12

0.800

38.57

4.73

Evaluation

Q4.1–Q4.12

0.749

39.85

4.44

Interpersonal skills

Q5.1–Q5.12

0.806

39.71

4.80

SRSSDL

Q1.1–Q5.12

0.867

196.87

18.18

Learning strategies Learning activities

According to Taber (2018), Cronbach alpha values of 0.7 or higher indicate an acceptable internal reliability. Table 3 indicates that all the reported Cronbach alpha values were above the guideline value of 0.7, which indicate that they are reliable. As part of a bigger research project, the self-rating scale of self-directed learning

http://ijlter.org/index.php/ijlter

49

(SRSSDL) has already been used at the NWU (N=403), and they obtained Cronbach's alpha coefficients between 0.76 and 0.88 for the SRSSDL for the five categories of the questionnaire indicating the SRSSDL was reliable in the South African context (Petersen & Mentz, 2016). The means of the resulting factors ranged between 38.57 (SD = 4.73) and 40.15 (SD = 4.24) and the reported means of the overall SRSSDL score is 196.87 (SD = 18.18) indicating that participants' self-directed learning is high.

5.2 Qualitative results Only 115 students who enrolled for the school-based project completed the reflections. The number of students per year group and subject are indicated in Figure 1.

3. 3% 1. 1%

3. 3%

1. 1% 1. 1%

7. 6%

4. 3% 4. 3%

34. 30%

7. 6% 50. 43%

1st Year History 2nd Year History 3rd Year History 3rd Year Busniness Studies

1st Year Accounting 2nd Year Business Studies 3rd Year Accounting 3rd Year Economics

Figure 1: Reflections completed

The student reflections on habits of mind were analysed by means of inductive thematic analysis. From the data, meaningful parts were grouped under initial codes followed by grouping the initial codes under categories. From the categories, the themes emerged (Nieuwenhuis, 2020). The categories under each theme (Table 4) will be supported by verbatim quotes from the various year groups.

http://ijlter.org/index.php/ijlter

50

Table 4: themes and categories Themes

Categories •

The effect that habits of mind have on self-directed learning skills

• • •

Subject content knowledge and skills

•

Pedagogical content knowledge and skills

•

Motivation

Usefulness of habits of mind

Willingness to incorporate habits of mind

Individual responsibility and ability Planning, implementing, and monitoring own learning Variation in own learning

5.2.1 The effect that habits of mind have on self-directed learning skills The habits of mind had a positive impact on self-directed learning skills. In general, by applying the habits, the participants realised that their own learning is not the sole responsibility of the lecturer. By identifying their personal responsibility in their own learning, the participants identified that the habits enabled them to be independent in various aspects of the learning process.

• Individual responsibility and ability Participants identified the responsibility that they had in their own learning, and they indicated that their own ability to be independent in various aspects of the learning process also improved.

“It teaches one to be self-independent, being able to think out of the box without any assistance”. (3rd Year, Economics) “They help me to be a responsible student, take charge for everything I do and also come up with ideas that will help me in this difficult time”. (4th Year, Business Studies) “Made me take charge of my own learning”. (4th Year, Accounting) • Planning, implementing, and monitoring own learning Participants indicated specific functions of the learning process that they can implement on their own with their improved abilities.

“It guides me as a first year through varsity studies as I'm not familiar with it and I can positively reflect on my work and strategies on how to do my academic work effectively”. (1st Year, Accounting) “I identify learning goals, question the significance of the theme, monitor my self-learning process, start the background on the theme”. (3rd Year, Economics)

http://ijlter.org/index.php/ijlter

51

“It develops self-directed learning in terms of highlighting what outcome I can get if I apply certain habits of mind and it allows me to be patient enough in applying those strategies and remembering that to each work there's a certain strategy that I can apply and that with the work dynamic I may face either it be I'm working with a group or as an individual there's strategies that I can apply”. (4th Year, Business Studies) • Variation in own learning Participants specified that they could adapt specific functions of the learning process that they can implement on their own.

“The habits of minds provide new ways of handling problems based on the content of that particular module. they also helped in a way that they opened up my mind, by so saying I mean that i have adapted to finding more than one way to deal with problem, this helped me understand what i was doing much more efficiently”. (1st Year, Accounting) “It also develops my self-directed learning in a way that each and everything that i do, i must first reflect on how I will approach it and what are the outcomes going to be. This helps to monitor each and every activity that I do that I must do reflection and also I must be persistent enough to myself then I will get everything in order”. (3rd Year, Economics) “Habits of mind increases the ability for one to solve problems, not only activities or assignments but also real life situations”. (4th Year, Business Studies) 5.2.2 Usefulness of habits of mind Participants recognised the utility of the habits of mind, and they focused most on its unlimited application for learning and teaching the habits of mind themselves.

• Subject content knowledge and skills Most of the participants indicated that the habits of mind improved their general and specific thinking skills and understanding their own thinking, which resulted in improved learning in the area of subject-content knowledge and skills.

“It helps me to be able to make my work much easier and more simpler than before. The strategy is abroad and develops a persons thinking skills”. (2nd Year, Business Studies) “It has enhanced my critical thinking as I no longer think narrow it made me to think abroad and to ask or question other things”. (3rd Year, Accounting) “It has helped me to really study the content with understanding and draw my own aims for the content that I'll be working on.” (4th Year, Business Studies)

http://ijlter.org/index.php/ijlter

52

• Pedagogical content knowledge and skills The majority of the participants were willing to apply the habits of mind as a teaching-and-learning strategy; their reflections indicated that they are able to initiate the planning and development of such teaching-and-learning strategies.

“As a teacher you have to have different learning styles, habits of mind and apply them on different activities in order to see, which one is the best to use for your learners”. (3rd Year, Business Studies) “By being patient and willing to learn from learners. By also encouraging learners to always be positive and not be impulsive. Applying pre knowledge to current knowledge.” (3rd Year, History) “Giving learners case studies to work in groups to solve the problems in the case study, were they will apply their thinking and different viewpoints so that other can understand or learn from certain perspective.” (3rd Year.

Accounting) 5.2.3 Willingness to incorporate habits of mind Most of the participants indicated that the habits of mind had a positive influence on their overall learning, which resulted in a positive perception about the habits of mind, which in turn resulted in the participants being more willing to apply it in their own learning and teaching in the future.

•

Motivation “That is good development that could be implemented in classroom situation and it has good advantages to learners as well to teachers”. (3rd Year, Business Studies) “I feel very positive about habits of mind they help an individual to have confidence in themselves and work, think more about a situation and solve problems. I would use them both for myself and my learners”. (4th Year Business Studies)

The small minority of participants that reflected negatively about the habits of mind were unwilling to incorporate them. These participants were mostly overwhelmed by the extra work it took to incorporate the habits of mind.

“It is time consuming and adding extra work on the work that we have to do.’

Its interesting but its too much”. (4th Year, Business Studies)

6. Discussion The School of Commerce and Social Studies in Education at the Vanderbijlpark campus of the NWU follows a project-based approach with an online teaching-andlearning strategy followed by most educational institutions due to the COVID-19 pandemic. The school-based project aimed to implement learning strategies to enhance self-directed learning to support pre-service teachers to think and work independently without direct instructions from the lecturers. Although the project-

http://ijlter.org/index.php/ijlter

53

based approach was a compulsory part of the course, it was not mandatory to complete Williamson’s questionnaire and the reflections on the different teaching strategies, and 277 participants out of 407 (68%) took part in this study. According to Table 1, Business Economics generally attracts more students than any other subject area because it has no admission requirements. At the same time, the low percentage of participants from Accounting (14.8%) was a reflection of the scarcity of education students who met the admission requirements for the subject.

With reference to the quantitative results based on Williamson’s questionnaire, the majority of the participants (Table 2) indicated a high level of self-directedness when starting with the project. The findings showed that none of the participants fell in the low self-directed learning bracket.

The literature predicted that the students would overestimate their own selfdirectedness, as students who are not fully aware of what self-directed learning entails tend to make assumptions regarding their own levels of self-directed learning (Petersen & Mentz, 2016). The prediction came to fruition when the qualitative and quantitative findings were compared, as the qualitative results were often in complete contradiction to what was found in the quantitative results. In this study, when comparing the qualitative data analysis results to the quantitative results, the qualitative results showed that the participants might have overestimated their own self-directedness; however, the qualitative findings also show potential in terms of self-directed learning abilities. This could indicate that the Williamson self-rating scale only captures self-directed learning at a specific moment in time and is subjective. However, these results also indicated that they recognised the benefits of applying the habits of mind when it comes to their own learning. These 115 participants could see the value of working with habits of mind as their learning improved. This resulted in the students improving in multiple aspects of the selfdirected learning process, as indicated by themselves. A student with a high level of self-directedness should, for example, be independent in their own learning from the start, an aspect that clearly only improved after the habits of mind were applied. Even though the students overestimated themselves (according to Williamson’s questionnaire), the habits of mind had a clear and positive effect on developing various aspects of the self-directed learning process. In their reflections, the participants reported the following on the 16 habits: 1. Persisting The elements of a plan was an excellent tool for the participants to utilise when they could not continue with a task. According to Feuerstein and Hoffman (1995), this plan supports students in following certain steps instead of giving up. The students reflected that they become more persistent by applying the elements of a plan. One participant said, “help to follow a plan to persist in that way i wont give up when completing tasks”. The participants explored multiple ways of solving problems when applying the habit of persistence during practical teaching by allowing learners to work together to explore solutions with their peers.

http://ijlter.org/index.php/ijlter

54

2. Managing impulsivity To schedule and plan, participants need to think before acting, realising that they had to take responsibility of the outcome of their thinking process, staying calm if things do not work out as planned, being cautious, and taking calculated risks when necessary. The deliberation and analysis of problem solving before making decisions was one of the skills the students mentioned that did improve: “It develop problem solving skills, to interact with others, understand other peoples view points”.

3. Listening with understanding and empathy In the case of listening with understanding and empathy, the participants experienced that the suggestions of the lecturers and their peers helped them to develop their comprehension and appreciation skills, and understanding the viewpoints of others. Group work was a favourite activity where students collaborated through technology. In this instance, one of the participants mentioned that “It helps a person to solve problems critically and be able to listen to other people’s ideas”.

4. Thinking flexibly Through heterogeneous teams the participants assessing several other senses that can contribute to learners' learning and understanding, such as olfactory (smell), tactile (touch), or kinaesthetic (movement) senses they were not familiar with, and succeeded to think out of the box. Seif et al. (2011) and findings emphasised flexibility, as reflected in the words of one participant: “I learnt that there are many ways to teach learners since they are all different”.

5. Thinking about own thinking (metacognition) By practising metacognition, the mindset of the students evolved, which allowed them to select a teaching method that best suited a particular topic. They also reflected on their teaching during and after a presentation, which enabled them to improve for their next lessons. From the findings, it is evident that students also discovered that the way they think would transform their minds and steer them in the right direction to reflect on their thoughts. One of the participants mentioned the following: “… it leads individuals to reflect on, evaluate, modify and carry forth my learnings to future applications”. In this way, students developed the habit of reflecting on their own work and seeing where they could improve, which improved their metacognition.

6. Striving for accuracy Students who strived for accuracy could figure things out, clarify problems, gather data, and recheck information. The participants noticed that accuracy is extremely valuable in class and real life.

7. Questioning and problem posing Participants could engage the skills to raise relevant questions to get answers for problems when working with peers. From the findings, participants reported that their critical thinking increased as a result of posing questions, because now, before they ask a question, they have to consider whether the question is appropriate. Others remarked that their thinking broadened when someone else asked them questions: “I

http://ijlter.org/index.php/ijlter

55

no longer think narrow . . .”.

8. Applying past knowledge to new situations This habit contributes to participants’ ability to construct new knowledge effectively, based on prior knowledge to link to real-life situations. Participants could apply this habit to contribute to a broader learning experience. The participants indicated that the application of this habit could be used effectively in class, especially when doing assignments that required examples from real-life experiences: “It will help learners channel their minds in the right direction in order to give out the relevant examples. It will make it easy for them to incorporate real life experiences with classroom based content”.

9. Thinking and communicating with clarity and precision Communication is embedded in and threaded through most of the key competencies for the future, making it vital for the participants to communicate with society: “Learners will work in groups to complete some activities which will enhance the communication skills, learners will be open minded and actually learn from other learners”.

10. Gather data through all senses Students also fall into the habit of using only visual data copied from the internet or textbooks. Talking and listening to people became a vital data gathering source that contributed to a broader and richer learning experience. The participants noticed that they needed to use all their senses to obtain information for their assignments.

11. Creating, imagining, and innovating Creativity, imagination, and innovation are foundational skills that enabled participants to facilitate similar learning in their learners in order to expand their thinking: “I developed a skill of creativity and made me to the right thing”.

12. Responding with wonderment and awe Participants experienced that changing to teaching methods that incorporated play activities engaged learners better, and made them feel satisfied and eager to learn more.

13. Taking responsible risks The responses of the participants to this habit were to try new and innovative methods for which there existed a risk of failure. It was essential to challenge learners and to make the new learning process inspiring: “Fostering optimism in students’ ability to take responsible risks”.

14. Finding humour By implementing humour in lessons, participants found that learners felt more comfortable and free to ask questions: “They were able to help me to be more creative in my work and be able to think outside of the box by finding humour and being innovative in my work, it

http://ijlter.org/index.php/ijlter

56

allows me to go beyond the knowledge I’ve already accumulated”.

15. Thinking interdependently Participants found value in the reciprocal approach to interdependent thinking, which benefitted everyone when working in groups. The participants agreed with the consequent co-construction of knowledge benefits that all learners were involved and it was therefore not one-sided, which motivated learners.

16. Remaining open to continuous learning Participants realised that they need to be lifelong learners and strive to improve their own learning, in which case learners will benefit from their updated knowledge: “Using the habits of mind strategies helps one to learn easier as they understand the importance of self learning and become involved in long term understanding of the content”.

7. Recommendations It is recommended that future training of pre-service teachers be scaffolded around the habits of mind that will contribute to developing the teachers' ways of thinking, self-directed learning, and competencies. In addition, several habits of mind will support teachers in utilising and/or adjusting their prior learning to be open to or contribute to the construction of new knowledge. It is also recommended that habits of mind be infused into the current BEd curriculum in order to foster self-directed learning.

8. Limitations It is essential to note that this study was by no means without limitations. One of these is that the study was confined to only one school in the Faculty of Education and only at one of the university’s three campuses. As a result, the findings based on this research study might be construed by some critics to be one-sided and not representative of the views of the majority of pre-service teachers in South Africa.

However, the data support and add to the findings of large-scale studies conducted in South Africa and abroad.

9. Conclusion The participants frequently experienced various difficulties in learning practices, both in class and in practice activities. They had problems with varying degrees of complexity, but they were usually expected to have reasoning ability and high-level thinking skills. Pre-service teachers who had these habits of mind revealed behaviours such as persistently searching for unique and innovative ways to improve, evolve, learn, change, and enhance themselves. It can be concluded that these new and innovative learning methods have the potential to lead to self-directed learning, an ongoing learning improvement process. Another positive result for preservice teachers was that they recognised the importance of applying these habits of mind that will support them in utilising their new knowledge in classroom practices, which will in turn promote self-directed learning that will equip learners with the

http://ijlter.org/index.php/ijlter

57

tools necessary to become self-directed critical thinkers and problem solvers.

10. Declarations Funding: This study is funded by the Research Unit: Self-Directed Learning at NorthWest University.

Conflicts of interest/Competing interests: In the study, all participants agreed to the use of data collected from them. We have no conflict of interest.

Availability of data and materials: We are sure that all data support our published claims and comply with field standards.

Authors' contributions: All authors contributed to the study conception and design.

All authors performed the literature search and review, material preparation, data collection and analysis. The corresponding author wrote the manuscript and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

11. References Adeyemo, S. A. (2003). Studies of the effects of aptitude, instructional leadership styles and learning environment on student’s achievement in physics [Unpublished doctoral thesis].

University of Lagos.

Al-Assaf, J. A. A. F. (2017). The teaching habits of mind, their relationship to positive behavior of social studies teachers in lower basic stage in university district–the capital (Amman). Journal of Curriculum and Teaching, 6(2), 30-51. doi:10.5430/jct.v6n2p30 Aleman, J. L. F., de Gea, J. M. C., & Mondéjar, J. J. R. (2011). Effects of competitive computerassisted learning versus conventional teaching methods on the acquisition and retention of knowledge in medical surgical nursing students. Nurse Education Today, 31(8), 866-871. doi:10.1016/j.nedt.2010.12.026 Anderson, J. (2010). Succeeding with habits of mind. Hawker Brownlow Education.

Barell, J. (2010). Problem-based learning: The foundation for twenty-first century skills. In J. Bellanca & R. Brandt (Eds.), Twenty-first century skills. Rethinking how students learn (pp. 175-199). Solution Tree Press.

Beall, M. L., Gill-Rosier, J., Tate, J., & Matten, A. (2008). State of the context: Listening in education.

International Journal of Listening, 22(2), 123–132. doi:10.1080/10904010802174826 Bozkurt, G. 2017. Social constructivism: Does it succeed in reconciling individual cognition with social teaching and learning practices in mathematics?.. Journal of Education and Practice, 8(3), 210-216.

Brockett, R. G., & Hiemstra, R. (2012). Reframing the meaning of self-directed learning: An updated model. Proceedings of the 54th Annual Adult Education Research Conference, Saratoga Springs, 45, 155-161.

Brookfield, S. D. (2009). The power of critical theory: Liberating adult learning and teaching. JosseyBass.

Codreanu, A., & Vasilescu, C. (2013). E-learning behaviors and their impact on andragogy. eLearning & Software for Education, 1, 126-137. doi:0.13140/RG.2.1.4223.7289 Comer, D. R., & Lenaghan, J. A. (2013). Enhancing discussions in the asynchronous online classroom: The lack of face-to-face interaction does not lessen the lesson. Journal of Management Education, 37(2), 261-294. doi:10.1177/1052562912442384

http://ijlter.org/index.php/ijlter

58

Costa, A. L. (2009). Describing the habits of mind. In A. L. Costa & B. Kallick (Eds.), Learning and leading with habits of mind. 16 Essential characteristics for success (pp. 15-41). Hawker Brownlow Education.

Costa, A. L., & Kallick, B. (Eds.). (2000). Habits of mind – A developmental series. Association for Supervision and Curriculum Development.

Costa, A. L., & Kallick, B. (2008). Creating a culture of mindfulness. In A. L. Costa & B. Kallick (Eds.), Learning and leading with habits of mind: 16 Essential characteristics for success (pp.

271-290). Association for Supervision and Curriculum Development.

Costa, A. L., & Kallick, B. (2009). Preface. In A. L. Costa & B. Kallick (Eds.), Learning and leading with habits of mind. 16 Essential characteristics for success (pp. xiv–xxii). Association for Supervision and Curriculum Development.

Costa, A. L., & Kallick, B. (2014). Habits of mind. In L. Green (Ed.), Schools as thinking communities (pp. 83-98). Van Schaik.

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Department of Basic Education. (2011). Curriculum and Assessment Policy for Grades 10–12.

Pretoria.

Eisner, E. W. (1991). What really counts in schools. Educational Leadership, 48(5), 10-17.

European Union. (2019). Key competences for lifelong learning. European Commission for Education and Training.

Eyre, D. (2016). High performance learning: How to become a world class school. Routledge.

Feuerstein, R., & Hoffman, M. B. (1995). Teacher’s guide to organisation of dots. IRI/Skylight Training and Publishing.

Gardner, H. (2006). Multiple intelligences: New horizon in theory and practice. Basic books.

Geduld, B. (2014). Re-thinking the value of learning theories to develop self-directedness in open-distance students. Journal of Educational and Social Research, 4(6), 11. doi:10.5901/jesr.2014.v4n6p11 Gloria, R. Y., & Indriyanti, D. R. (2017). Formative assessment with stages of understanding

by Disgn (UbD) in improving habits of mind. International Journal of Environmental and Science Education, 11(10), 2233-2242.

Grosser, M. M., & Brettenny, E. (2014). Habits of mind. Short learning programme in cognitive education. North-West University.

Hayat, M. S., Rustaman, N. Y., Rahmat, A., & Redjeki, S. (2019). The improvement of prospective teachers’ habits of mind during the 5E+ e inquiry learning program in horticulture course. International Journal of Environmental and Science Education, 14(9), 535-545.

Hew, K. F., & Cheung, W. S. (2011). Student facilitators’ habits of mind and their influences on higher-level knowledge construction occurrences in online discussions: A case study. Innovations in Education and Teaching International, 48(3), 275-285. doi:10.1080/14703297.2011.593704 Holtz, S. (2017, 15 June). Why it’s important to support self-directed learning in the classroom.

NEO Blog. https://blog.neolms.com/supporting-self-directed-learning-classroom Horne, M., Woodhead, K., Morgan, L., Smithies, L., Megson, D., & Lyte, G. (2007). Using enquiry in learning: From vision to reality in higher education. Nurse Education Today, 27(2), 103-112. doi:10.1016/j.nedt.2006.03.004

http://ijlter.org/index.php/ijlter

59

King, C. (2011). Fostering self-directed learning through guided tasks and learner reflection.

Studies in Self-Access Learning Journal, 2(4), 257-267.

Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Follett Publishing Company.

Kowalski, K. (2009). Situations in which questions are valuable. The Journal of Continuing Education in Nursing, 40(8), 344-345. doi:10.3928/00220124-20090723-10 Lei, S. A., Cohen, J. L., & Russler, K. M. (2010). Humor on learning in the college classroom: Evaluating benefits and drawbacks from instructors’ perspectives. Journal of Instructional Psychology, 37(4), 326-331.

Maree, K., & Pietersen, J. (2020). Sampling. In K. Maree (Ed.) First steps in research (2n ed., pp.

191-202). Van Schaik Publishers.

Marzano, R. J. (1992). A different kind of classroom: Teaching with dimensions of learning.

Association for Supervision and Curriculum Development.

Marzano, R. J., Pickering, D., & McTighe, J. (1993). Assessing student outcomes: Performance assessment using the dimensions of learning model. Association for Supervision and Curriculum Development.

Masie, E. (2002). Blended learning: The magic is in the mix. The ASTD e-learning handbook, 58, 63.

Murad, M. H., Coto‐Yglesias, F., Varkey, P., Prokop, L. J., & Murad, A. L. (2010). The effectiveness of self‐directed learning in health professions education: A systematic review. Medical Education, 44(11), 1057-1068. doi:10.1111/j.1365-2923.2010.03750.x Nieuwenhuis, J. (2020). Analysing qualitative data. In: K. Maree, (Ed.), First steps in research (pp. 117-151). Van Schaik.

O’Shea, E. (2003). Self‐directed learning in nurse education: A review of the literature. Journal of Advanced Nursing, 43(1), 62-70. doi:10.1046.j.1365-2648.2003.02673.x Petersen, N. T., & Mentz, E. (2016). The influence of cooperative learning methods on second year tertiary student-teachers’ levels of self-directedness in learning. In M. A. Mokoena & I. Oosthuizen (Eds.), A kaleidoscope of advances in modern day education (pp.

41-63). Ivyline Academic Publishers.

Pink, D. H. (2006). A whole new mind. Why right-brainers will rule the future. Riverhead Books.

Rogers, A. (2004). EFA and adult learning. Convergence, 37(3), 3.

Romero-Ivanova, C., Shaughnessy, M., Otto, L., Taylor, E., & Watson, E. (2020). digital practices & applications in a COVID-19 culture. Higher Education Studies, 10(3), 80-87.

Seif, E., Kallick, B., & Costa. A. (2011). Effective teachers are made, not born: Habits of mind make the difference.

Era 3 learning. era3learning.org/resources/effective_tchrs_are_made.pdf Serdyukov, P. (2017). Innovation in education: what works, what doesn’t, and what to do about it? Journal of Research in Innovative Teaching & Learning, 10(1), 4-33. doi: 10.1108/JRIT-10-2016-0007 Shabani, K., Khatib, M. & Ebadi, S. (2010). Vygotsky's zone of proximal development: Instructional implications and teachers' professional development. English Language Teaching, 3(4), 237-248.

Shahabadi, M. M., & Uplane, M. (2015). Synchronous and asynchronous e-learning styles and academic performance of e-learners. Procedia-Social and Behavioral Sciences, 176, 129138. doi:10.1016/j.sbspro.2015.01.453

http://ijlter.org/index.php/ijlter

60

Sriyati, S., Rustaman, N. Y., & Zainul, A. (2010). Kontribusi asesmen formatif terhadap habits of mind mahasiswa biologi. Jurnal Pengaajaran MIPA, 15(2), 77-86. Ștefӑnicӑ, F., Abele, S., Walker, F., & Nickolaus, R. (2017). The subject-specific prior knowledge proves to be the main predictor for the development of professional competence. In M. Mulder (Ed.), Competence-based vocational and professional education. Bridging the worlds of work and education (pp. 834–858). Springer.

Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296. doi:10.1007/s11165-016-9602-2 Tough, A. (1978). Major learning efforts: Recent research and future directions. Adult Education, 28(4), 250-263. doi:10.1177/074171367802800403 Vazquez, J. C. (2020). The impact of habits of mind: an exploratory study [Doctoral dissertation, Western Connecticut State University].

WCSU Repository. https://repository.wcsu.edu/educationdis/96/ Williamson, S. N. (2007). Development of a self-rating scale of self-directed learning. Nurse Researcher, 14(2), 66-83. doi:10.7748/nr2007.01.14.2.66.c6022 Yandari, I. A. V., Supartini, S., Pamungkas, A. S., & Khaerunnisa, E. (2019). The role of habits of mind (HOM) on student's mathematical problem solving skills of primary school.

Al-Jabar: Jurnal Pendidikan Matematika, 10(1), 47-57. doi:10.24042/ajpm.v10i1.4018 Zenger, J., & Uehlein, C. (2001). Why blended will win. Training & Development, 55(8), 54-62.

http://ijlter.org/index.php/ijlter

61

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 61-80, February 2021 https://doi.org/10.26803/ijlter.20.2.4

The Impact of Professional Development, Modern Technologies on Lecturers’ Self-Efficacy: Implication for Sustainable Science Education in Developing Nations David Agwu Udu*, Benson Ikechukwu Igboanugo, John Nmadu, Chidebe Chijioke Uwaleke, Benjamin Chukwunonso Okechineke, Adaora Phina Anudu, Precious Chisom Attamah, David Onyemaechi Ekeh and Mercy Ifunanya Ani Alex Ekwueme Federal University, Ndufu-Alike, Ebonyi State, Nigeria https://orcid.org/0000-0003-3632-8190 https://orcid.org/0000-0002-5913-317X https://orcid.org/0000-0002-1787-9491 https://orcid.org/0000-0003-4504-6064 https://orcid.org/0000-0002-2806-6146 https://orcid.org/0000-0002-8704-5716 https://orcid.org/0000-0002-4670-7576 https://orcid.org/0000-0002-8247-2460 https://orcid.org/0000-0002-0776-3882

Abstract. This study evaluated the impact of a professional development program (PDP) on science lecturers' level of knowledge/understanding and the extent of use of modern technologies for improving their self-efficacy in teaching in the areas of students' engagement, instructional strategies, and classroom management, in some selected universities in Nigeria. The study adopted a descriptive survey design. The sample was 269 lecturers. The researchers adapted the teacher sense of efficacy scale (TSES) developed

by Tschannen-Moran and Hoy (2001) for the study. The TSES has two parts (1) and (2) that contained 12 questions each. It was a validated instrument, with overall reliability indices of 0.92 and 0.95 respectively determined using Cronbach’s Alpha. The data obtained were analyzed with mean, standard deviation, and paired sample t-test. The study found that the PDP improved the lecturers' understanding of modern technologies and self-efficacy in teaching (t (268) = 30.959, p = 0.000 ˂ 0.05). Also, the PDP improved the lecturers’ extent of utilisation of modern technologies and self-efficacy in teaching (t (268) = 28.510, p = .000 ˂ .05). The study also found that the lecturers’ understanding of modern technologies after the PDP was high, but their use of modern technologies in teaching was still low. The researchers recommended that science lecturers' participation in PDPs should be regular to ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

62

improve their understanding and use of modern technologies in teaching for enhanced self-efficacy and sustainable science education in developing nations.

Keywords: professional development; modern technologies; selfefficacy; science lecturers; sustainable science education

1. Introduction Speck and Knipe (2005) see professional development as people development and engagement activities culminating into learning to earn or maintain professional credentials in general.

Speck and Knipe note that these activities are acquiring academic degrees through formal coursework, attendance at conferences, workshops, seminars, and other informal learning opportunities situated in practice in professional development programs. In the educational context, Liu (2012) defines professional development as a wide variety of specialized training, formal education, or advanced professional learning intended to help and support school administrators, teachers, and other educators to improve their professional competence, skill, and effectiveness.

Similarly, the Organisation for Economic Cooperation and Development, OECD, (2009) sees professional development programs as activities that develop an individual’s skills, expertise, and other characteristics.

On the other hand, Djigic, Stojiljkovic and Doskovic (2014) and Caprara et al.

(2006) describe self-efficacy as an educator's belief in his ability to successfully cope with tasks, obligations, and challenges related to his/her professional roles in the educational environment. Klassen and Tze (2014) note that an understanding of educators' self-efficacy is vital to teaching effectiveness, instructional practices, and improving students' academic achievement. Some studies have proven that using modern technologies in the teaching and learning processes enhances the educator’s sense of self-efficacy. Moreover, professional development programs are vital tools for assisting educators in knowledge acquisition and utilizing modern technologies in teaching and learning processes (Caprara et al., 2006, Barni et al., 2019, Udu, 2020). Tertiary institutions lecturers should be exposed to professional development programs (PDPs) regularly to enhance their knowledge and use of modern technologies in teaching. It is very vital to the enhancement of their self-efficacy and for sustainable science education programs in the developing nations, including Nigeria.

Most developing nations saw the importance of modern technologies in teaching in their tertiary institutions and have committed some funds to acquire some modern technologies. For instance, in Nigeria, many Universities have purchased some modern technological gadgets for teaching. The items were provided in the universities by the Nigerian Tertiary Education Trust Fund (TETfund) in conjunction with the Nigerian Communications Commission (NCC). Previous researchers found that many modern technological gadgets have been acquired and installed in the universities for teaching purposes, however, many lecturers do not use the gadgets in their classroom teaching and

http://ijlter.org/index.php/ijlter

63

learning processes. The lecturers have not been successfully integrating these modern technological gadgets into their courses for effective teaching and learning processes (Hennessy, Ruthven & Brindley, 2005, Goktaş, Yıldırım & Yıldırım, 2008, Udu, 2020). Besides, McCarney, (2004) notes that professional development programs help lecturers to learn effective ways of utilizing and integrating modern technologies into teaching, thereby achieve their self-efficacy in teaching and learning processes. Therefore, this study that evaluates the impact of professional development programs on science lecturers' knowledge and utilization of modern technologies offers a route map for enhancing their effectiveness in teaching and learning processes, thereby improving their selfefficacy for sustainable science education.

2. Modern Technologies in Education Modern technology refers to the electronic tool, piece of equipment, or device that helps students to accomplish specified learning goals (Davies, Sprague & New, 2008). Modern technologies play crucial roles in human lives and are specifically vital in the teaching and learning processes. Marshall (2010) observes that the impacts of modern technologies are felt in all fields of study, including education. Marshall notes further that effective use of technologies in education will change the face of education and create more educational opportunities. For instance, the lecturers are beginning to learn how to integrate technologies in their classrooms, while the students are getting more interested in learning with technologies (Karehka, 2013). Besides, Chen (2008) opines that teaching and learning facilities should be made available to the lecturers for effective teaching and learning, such as modern technological gadgets, ranging from low-tech equipment, like the chalkboard, to more sophisticated presentation software.

These may include online collaboration and conferencing materials for effective teaching and learning processes in the universities. However, Lynch (2017) observes that modern technology may not be applied effectively in the classrooms without knowledgeable educators with the basic knowledge of the technology and its implementation to meet educational goals. Continuing, Lynch analyzed the modern technology utilization in the classrooms and likened it to a foray into modern invention in which the educator gets to be the expedition leader. Concluding Lynch infers that rather than viewing modern technologies (such as digital devices and internet spaces) as a “threat” to their duties, the educators (lecturers and teachers) should view them as unexplored areas of growth for both themselves and their learners. Besides, the United States Department of Education (2011) contributed by observing that modern technologies are potential transformers of teaching by ushering in a new model of connected wholes in teaching. This new model will be equipped and link the lecturers to their students, professional content, resources, and other systems that improve the instructional strategies and sense of efficacy. Therefore, the importance of modern technology integration in science education for effective teaching and learning processes in the universities cannot be over-emphasized.

3. Professional Development Programs and Self-efficacy Padwad and Dixit (2011) see a professional development program as a planned, continuous, and lifelong process in which lecturers and other educators develop

http://ijlter.org/index.php/ijlter

64

their personal and professional qualities, to improve their knowledge, skills, and practice0 leading to the achievement of their self-efficacy. Similarly, Kennedy (2005) and Whitcomb, Borko, and Liston (2009) reiterated that engaging the lecturers in regular professional development programs will improve their teaching quality, enhance students' achievement and improve the self-efficacy of the educators. Furthermore, engagement in professional development programs positively impacts the lecturers’ ability to critically develop the knowledge and skills they require for good professional practices with their students and colleagues (Buczynski & Hansen, 2010; Gabriel, Day & Allington, 2011, Tareq, 2020). Besides, Fullan (2015) and Yoo (2016) emphasized that professional development programs contribute to the educators' effectiveness by providing the avenue for a continuous individual and collective enhancement of their teaching strategies, which are necessary for the overall improvement of the students’ learning outcomes. Besides, Mohammad and Khaled (2017) averred that lecturers need regular opportunities to update their professional knowledge and skills. These are available in professional development programs.

Furthermore, according to Tschannen-Moran and Hoy (2001) self-efficacy is the judgment of the lecturer’s/teacher's capabilities to bring about desired outcomes of students’ engagement and learning. Tschannen-Moran and Hoy note further that self-efficacy can measure effectiveness in students' engagement, use of instructional strategies, and classroom management. Some research findings suggest that self-efficacy correlates positively with other constructs like selfregulated learning (Lau, Liem & Nie, 2008), motivation and academic performance work stress (Usher & Pajares, 2008), job satisfaction (Chong et al., 2010), and students' outcome (Moe, Pazzaglia & Ronconi, 2010). Lecturers with a high level of sense of self-efficacy are more likely to stay in their teaching professions, spend more time teaching, make great efforts in classroom management, and show great enthusiasm for teaching. They respond to their student's needs and are willing to spend more time working with problematic students (Tschannen-Moran & Hoy, 2001, Ho & Hau, 2004). Lecturers with a high level of sense of self-efficacy are very valuable in the teaching profession.

This study that evaluates the use of modern technologies to improve lecturers' self-efficacy through professional development programs is justified.

Meanwhile, the theoretical underpinning of lecturers' self-efficacy is directly related to the social cognitive theory of behavioral change developed by "Bandura" in 1977. Bandura's learning theory relates to the lecturer's belief in his ability to cope with tasks, obligations, and challenges in his professional role successfully (Barni et al. (2019). Caprara et al. (2006) identified the roles to include; didactical tasks, managing discipline problems in the classroom, and other issues. Based on bandura's socio-cognitive theory of behavioral change, researchers have opined that lecturers with a high level of students' engagement in the teaching and learning processes exhibit a high level of self-efficacy (Caprara et al., 2006). Similarly, when lecturers utilize several instructional strategies to drive home learning, the outcome is usually high academic achievement and a high level of self-efficacy (Djigic et al. 2014). Also, the exhibition of good managerial skills evidenced in the lecturer's effective

http://ijlter.org/index.php/ijlter

65

classroom management gives rise to the lecturers' high self-efficacy (Sousa et al.

2012). This study, therefore, investigates how the use of a professional development program on the knowledge and use of modern technologies can improve the lecturers’ self-efficacy in the teaching and learning processes in the universities.

It has been established from research studies that professional development programs are vital tools for the improvement of the lecturers’ self-efficacy in the teaching and learning processes in the universities. It is expedient that lecturers should avail themselves the opportunities for active engagement of modern technologies in teaching and learning processes provided by professional development programs. It will enhance their professional proficiency and selfefficacy in the teaching and learning processes.

Meanwhile, the Advanced Digital Appreciation Programme for Tertiary Institutions (ADAPTI) is a professional development program that provides training on modern technologies in education. The ADAPTI program is a unique platform used for training the academic and non-academic staff of Nigerian tertiary institutions. The focus of the program is on the proficient use of modern technologies such as basic information and communication technology (ICT) office productivity tools, including the internet, use of e-teaching and e-learning facilities as well as collaborations in course delivery for enhanced e-teaching and learning (Digital Bridge Institute, 2018). The ADAPTI equips participants with these essential modern technological skills through hands-on training and learning approaches to translate the acquired knowledge and skills to improve teaching and research (Udu, 2018).

Several studies have shown that professional development programs increase knowledge and use of modern technologies in teaching and learning (Voogt, Almekinders, Van den Akker, & Moonen, 2005; Lavonen et al., 2006; Giordano, 2008).

However, some studies found that the professional development program increases the educator's acquisition of modern technology skills, and the extent of usage of these modern technologies was still low (Brinkerhoff, 2006; Fragkouli & Hammond, 2007; Glazer et al., 2009). Nevertheless, Liu (2012) supports the fact that professional development programs increase the educator’s skills in modern technology usage in teaching and learning.

Based on the preceding facts, professional development programs have enhanced the lecturers' acquisition of modern technology skills for teaching, however, regular use of these technologies that enhance the lecturers' selfefficacy in teaching and learning processes were yet to be accomplished. Hence this study evaluates the relationship between professional development programs and the lecturers' knowledge and use of modern technologies for improving their self-efficacy in teaching in the areas of students' engagement, instructional strategies, and classroom management, as illustrated in figure 1.

Specifically, this study evaluates the impact of PDP on the science lecturers'

http://ijlter.org/index.php/ijlter

66

knowledge and use of modern technologies for improving their self-efficacy in teaching on the completion of the professional development program (ADAPTI).

Lecturers' Professional Development Programme

Lecturer’s Self-efficacy

Students Engagement Knowledge of Modern Technology

Instructional Strategies

Use of Modern Technology

Classroom Management Figure 1: Model of the variables relationships of the study (Source: own elaboration)

The knowledge and use of modern technologies in teaching and learning processes acquired through professional development programs are becoming a priority to educational institutions, lecturers, and other stakeholders in developing nations, particularly, in Nigeria (Udu, 2020). It is, therefore, essential to explore how professional development programs can enhance lecturers’ knowledge and use of modern technologies for improving their self-efficacy in teaching and learning processes in Nigerian universities. Specifically, this study addresses the following research questions, with corresponding hypotheses: R1 What is the impact of PDP on the science lecturers’ level of knowledge of modern technologies for improving their self-efficacy in teaching in the universities?

Ho1 There is no significant impact of PDP on the science lecturers' knowledge of modern technologies for improving their self-efficacy in teaching in the universities.

R2 What is the impact of PDP on the science lecturers’ use of modern technologies for improving their self-efficacy in teaching in the universities?

H o2 There is no significant impact of PDP on the science lecturers’ use of modern technologies for improving their self-efficacy in teaching in the universities.

http://ijlter.org/index.php/ijlter

67

R3

H o3

Is there any difference in the science lecturers' overall knowledge and use of modern technologies for improving their self-efficacy in teaching after completing the PDP? There is no significant difference in the science lecturers' overall knowledge and use of modern technologies for improving their selfefficacy in teaching after completing the PDP.

4. Method and Materials The study adopted a descriptive survey design. The sample was 269 science lecturers from three universities that participated in a professional development program (PDP) - the Advanced Digital Appreciation Program for Tertiary Institutions (ADAPTI) in 2018. The sample was composed of 175 (65%) males and 94 (35%) females drawn using a stratified random sampling technique. The participants indicated binary gender only. All the lecturers were full-time staff of the universities, with an age range of 33 to 55 years (M = 42, SD = 6.2). Years of experience range from 5 to 25 years (M = 13, SD = 5.6). The Assistant Lecturers were 94 (35%), Lecturer II, 81 (30%), Lecturer I, 62 (25%), and Senior Lecturers 27 (10%). They were biological sciences and biology education, 91 (34%), chemistry and chemical education 86 (32%), physics and physics education, 57 (21%), and mathematics and related courses, 35 (13%).

The “Teachers’ Sense of Efficacy Scale” (TSES) developed by Tschannen-Moran and Hoy (2001) to measure the sense of efficacy for teachers was the instrument adapted for this study (Appendix 1). The TSES has two versions: a long-form (24 items) and a short form (12 items). Tschannen-Moran and Hoy (2001) indicated that both forms of the TSES were considered almost identical in their effectiveness in measuring self-efficacy because they have very similar psychometric values. Specifically, the short form/version of the TSES was the instrument adapted for this study. The short version of the TSES is composed of 12 items with three subscales which are: efficacy for student engagement (ESE), with (4) items, efficacy for instructional strategies (EIS), with (4) items, and efficacy for classroom management (ECM), with (4) items. The adapted TSES was used in the two (2) parts of the instrument. Part 1 assessed the lecturers' selfefficacy in the knowledge of modern technologies in teaching while part 2 assessed the lecturers' self-efficacy in the use of modern technologies in teaching in the areas of efficacy for students' engagement (ESE), efficacy for instructional strategies (EIS), and efficacy for classroom management (ECM). The TSES was a validated instrument with factor loadings that ranged from 0.60 to 0.85 and accounted for 61% of the variance.

Also, the construct validity that assessed its correlation with other existing scales of teaching efficacy, shows that it successfully confirms the construct of efficacy (Tschannen-Moran & Hoy, 2001). In part 1 of the TSES, the original instrument had alpha (α) values of 0.81, 0.86, and 0.86 for the ESE, EIS, and ECM respectively, and an overall value of 0.91. In the adapted TSES the alpha (α) values obtained were 0.81, 0.86, and 0.88 for the ESE, EIS, and ECM respectively, and an overall value of 0.92. These values were consistent with the results of Tschannen-Moran and Hoy (2001) thereby confirm the reliability of the adapted

http://ijlter.org/index.php/ijlter

68

instrument. Some examples of items adapted in the questionnaire for the ESE, EIS, and ECM are as given. The item "How much can you do to motivate students who show low interest in schoolwork?” was adapted as “To what extent can your knowledge of modern technologies help you to motivate students who show low interest in schoolwork?" The item "To what extent can you craft good questions for your students?" was adapted as "To what extent can your knowledge of modern technologies assist you in crafting good questions for your students?" The item "How much can you do to control disruptive behavior in the classroom?" was adapted as "To what degree can your knowledge of modern technologies help you to control students' disruptive behavior in the classroom?" Similarly, in part 2 of the TSES, the original instrument had alpha (α) values of 0.81, 0.86, and 0.86 for the ESE, EIS, and ECM respectively, with an overall value of 0.91. In the adapted TSES the alpha values obtained are 0.79, 0.86, and 0.83 for the ESE, EIS, and ECM respectively with an overall value of 0.95. These results were also consistent with the results obtained

by Tschannen-Moran and Hoy (2001) which confirms the reliability of the adapted instrument. Some examples of items adapted in the questionnaire for the ESE, EIS, and ECM are given. The item "How much can you do to motivate students who show low interest in schoolwork?" was adapted as "To what extent do you use modern technologies to motivate students who show low interest in schoolwork?" The item "To what extent can you craft good questions for your students?" was adapted as "To what extent do you use modern technologies to craft good questions for your students?" The item "How much can you do to control disruptive behavior in the classroom?" was adapted as "To what extent do you use modern technologies to control disruptive behavior in your classroom?"

The researchers used the adapted TSES in the Pre- and Post-PDP surveys that assessed the Lecturers' knowledge and the use of modern technologies in teaching before and after completing the PDP. There were two sections, A and B in the TSES. Section A was for obtaining personal data of the respondents like age, gender, years of teaching experience, rank, department, etc., while section B contained 24 questionnaire items separated into two parts, 1 and 2, as described above. The TSES has a five-point rating scale instrument that ranges from 1 (very small extent) to 5 (very large extent).

Furthermore, the survey was conducted twice, before the PDP, and eight (8) months after the PDP. The participants were informed about the study and they consented to be involved in the study. The questionnaires were distributed to the lecturers on the first day of the ADAPTI program at their various universities where the researchers visited them. Their responses were collected on the spot and also their contact details. The contact details collected enabled the researchers to reach out to them after the program for the post-survey responses. The researchers received a total of 289 fully responded and completed questionnaires in the pre-program survey. Out of a total of 297 fully responded/completed questionnaires in the post-survey, only 269 questionnaires had "matching pairs" and were the sample of the study. The

http://ijlter.org/index.php/ijlter

69

researchers utilized the data obtained from the 269 participants who completed both the pre-surveys and post-surveys for the data analysis.

Data analyses using mean, standard deviation, and the paired sample t-test. The alpha level was set at p < 0.05 for the t-tests. The researchers used the Statistical Package for the Social Sciences (SPSS) software version 23 for data analysis.

5. Results The results are in tables according to the research questions and hypotheses. The first research question was on the impact of PDP on the science lecturers' knowledge of modern technologies for improving their self-efficacy in teaching in the universities.

Table 1: Summary of Mean and Standard deviation of Lecturers’ Pre- and PostSurveys on Knowledge of Modern Technologies Item Knowledge of modern technologies Teacher efficacy in students’ engagement

N

Group

269

Post-Survey Pre-Survey

Teacher efficacy in instructional strategies

269

Post-Survey Pre-Survey

Teacher efficacy in classroom management Total for knowledge of modern technologies Source: own elaboration.

269 269

Post-Survey Pre-Survey Post-Survey Pre-Survey

Mean

Std. Deviation

3.52 2.55 3.66

1.24 1.31 1.24

Std. Error Mean 0.08 0.08 0.08

2.52

1.30

0.08

3.34 2.46 3.51 2.51

1.22 1.31 1.22 1.30

0.07 0.08 0.07 0.08

Table 1 shows the summary of the lecturers' post- and pre-survey of the TSES that assessed their level of knowledge of modern technologies for improving their self-efficacy in teaching in the areas of ESE, EIS, and ECM. The results show a total mean and standard deviation of 3.51 and 1.22 of the lecturers' responses in the post-survey. And in the pre-survey, their mean and standard deviation score is 2.51 and 1.30 respectively. The table shows a mean difference of 1.00. However, the researchers could not ascertain from the table whether the observed mean difference is statistically significant or could be attributed to a sampling error. The result was subsequently subjected to a paired sample t-test as shown in table 2 to test hypothesis 1. The lower standard deviation of 1.22 for the lecturers' responses in the post-survey indicates that the lecturers’ responses were clustered around the mean in the post-survey more than their responses in the pre-survey with a standard deviation of 1.30.

http://ijlter.org/index.php/ijlter

70

Table 2: Data Process of Paired Sample t-test on Post- and Pre-survey of Lecturers’ Level of Knowledge of Modern Technologies Paired Differences 95% Confidence Interval of the Std. Difference Std. Error Mean Deviation Pair 1. Post-TPDP – Pre-TPDP

0.99

0.53

Mean

Lower

Upper

T

Df

Sig. (2tailed)

0.03

0.93

1.06

30.96

268

0.00

Source: own elaboration.

Table 2 is the data process result of a paired samples t-test that tested the null hypothesis1 (there is no significant impact of PDP on the science lecturers’ level of knowledge of modern technologies for improving self-efficacy in teaching in the universities). From the table, the t-value is 30.96 and the sig. or p-value is 0.00 which is less than 0.05 set for the study. Also, the confidence interval of the difference was from 0.93 (lower bound) to 1.06 (the upper bound). Based on these results, the null hypothesis 1 is, therefore, rejected. This implies that there is a significant impact of the PDP on the lecturers’ level of knowledge of modern technologies in teaching after completing the program (t (268) = 30.96, p = 0.00 ˂ 0.05). The results have shown that the observed difference in the mean responses of the lecturers presented in table 1 is statistically significant, and the PDP is effective in enhancing the lecturers' level of knowledge of modern technologies and improve their self-efficacy in teaching in the areas of SE, IS, and CM in the universities.

The second research question was on the impact of PDP on the science lecturers' extent of use of modern technologies for improving their self-efficacy in teaching in the universities.

Table 3: Summary of Mean and Standard deviation of Lecturers’ Pre- and PostSurveys of Extent of Use of Modern Technologies Item Utilization of modern Technologies Teacher efficacy in students’ engagement Teacher efficacy in instructional strategies Teacher efficacy in classroom management Total for utilization of modern technologies Source: own elaboration.

N

Group

Mean

Std. Deviation

269

Post-Survey Pre-Survey Post-Survey Pre-Survey Post-Survey Pre-Survey Post-Survey Pre-Survey

3.20 2.64 3.33 2.54 3.12 2.55 3.22 2.58

1.24 1.32 1.27 1.32 1.27 1.31 1.25 1.31

269 269 269

Std. Error Mean 0.08 0.08 0.08 0.08 0.08 0.08 0.07 0.08

Table 3 shows that the total mean and standard deviation of the lecturers in the post-survey are 3.22 and 1.25 respectively, while in the pre-survey the mean and standard deviation scores are 2.58 and 1.31 respectively. The table shows a mean difference of 0.64. However, the researchers could not deduce from the table

http://ijlter.org/index.php/ijlter

71

whether the observed mean difference is statistically significant or could be attributed to a sampling error. Hence, the corresponding hypothesis 2 was tested using a paired sample t-test shown in table 4. Meanwhile, the lower standard deviation of 1.25 obtained in the post-survey responses shows that the lecturers' responses were clustered around the mean, while the higher standard deviation of 1.31 in the pre-survey shows more diversity in their responses.

Table 4: Paired Sample t-test on Post- and Pre-survey of Lecturers’ Extent of Use of Modern Technologies Paired Differences 95% Confidence Interval of the Std. Difference Std. Error Mean Deviation Pair 1.

Post-TPDP – Pre-TPDP

0.64

0.37

Mean

Lower

Upper

T

Df

Sig. (2tailed)

0.02

0.60

0.69

28.51

268

0.00

Source: Authors’ elaboration.

Table 4 shows that the null hypothesis 3 (there is no significant impact of PDP on the science lecturers’ extent of use of modern technologies for improving their self-efficacy) is rejected. The table shows a t-value of 28.51 and a sig. or p-value of 0.00 which was less than 0.05 set for the study. Besides, the confidence interval of the difference was from 0.60 as lower bound to 0.69 as the upper bound, hence the rejection of the null hypothesis 2. The implication is that there is a significant impact of the PDP on the extent to which the lecturers use modern technologies for improving their self-efficacy after completing the program (t (268) = 28.51, p = 0.00 ˂ 0.05). The result shows that the observed mean difference in table 3 is statistically significant and that the PDP may have enhanced the extent to which the lecturers use modern technologies for improving their self-efficacy in teaching in the areas of SE, IS, and CM in the universities.

The third research question assessed the difference in the science lecturers' overall level of knowledge and extent of use of modern technologies in improving their self-efficacy in teaching in the universities after completing the PDP. Table 5: Summary of Mean and Sd of Lecturers’ Post-survey on knowledge and Use of Modern Technologies Description The overall level of Knowledge of modern technologies The overall extent of use of modern technologies

N

Mean

Std. Deviation

Std. Error Mean

269

3.51

1.22

0.07

269

3.22

1.25

0.08

Source: Authors’ elaboration.

Table 5 shows the overall mean and standard deviation of the science lecturers' level of knowledge of modern technologies in the post-survey are 3.51 and 1.22 respectively. Similarly, their overall mean and standard deviation scores on the

http://ijlter.org/index.php/ijlter

72

extent of use of modern technologies in the post-survey are 3.22 and 1.252 respectively. The table shows a mean difference of 0.29. However, the researchers could not establish from the table whether the observed mean difference is statistically significant or could be a result of sampling error.

Therefore, the result is subjected to a paired sample t-test to test the null hypothesis 3 shown in table 6. The standard deviations of 1.22 and 1.25 show that the lecturers' responses on knowledge of modern technologies were clustered around the mean more than their responses on the extent of use of modern technologies, which showed more diversity.

Table 6: Paired sample t-test on the overall Lecturers' Level of Knowledge and Extent of Utilisation of Modern Technologies

Mea n Pair 1

Paired Differences 95% Confidence Interval of the Std. Std. Difference Deviatio Error n Mean Lower Upper

Knowledge – Utilisation 0.29

0.22

0.01

0.26

0.32

T

df

Sig. (2tailed)

21.37

268

0.00

Source: Authors’ elaboration.

Table 6 shows that the null hypothesis 3 (there is no significant difference in the science lecturers' overall level of knowledge and extent of use of modern technologies for improving their self-efficacy in teaching after completing the PDP) is rejected. The table shows a t-value of 21.37 and sig. (2-tailed) or p-value of 0.00 less than 0.05 set for the study. The confidence interval of the difference was from 0.26 (lower bound) to 0.32 (upper bound) hence, the rejection of the null hypothesis 3. The result implies a statistically significant difference between the science lecturers’ overall level of knowledge and the extent of use of modern technologies for improving their self-efficacy in teaching after completing the PDP (t (268) = 21.37, p = 0.00 ˂ 0.05). Therefore, the observed mean difference in table 5 is statistically significant. The implication is that the PDP may have enhanced the lecturers’ level of knowledge more than the extent to which they use modern technologies for improving their self-efficacy in teaching in the areas of SE, IS, and CM in the universities.

6. Discussion of Findings The findings of this study have shown that the professional development program (PDP) might have enhanced the knowledge of the lecturers in modern technologies for improving their self-efficacy in teaching in the areas of students' engagement, use of diverse instructional strategies, and classroom management techniques in the universities. The above assertion arrived because the mean difference observed in the pre-survey and post-survey responses of the lecturers were statistically significant. The finding agrees with Voogt et al. (2005), Lavonen et al. (2006), Giordano (2008), and Udu (2020) who found separately that professional development programs have the capacity of enhancing participants’ knowledge and attitude towards the use of technologies for

http://ijlter.org/index.php/ijlter

73

instructional purposes. This study has, therefore, demonstrated that PDP has the capacity for effectively enhancing the lecturers’ knowledge of modern technologies for improving their self-efficacy in teaching, specifically, in engaging the students actively in the teaching and learning processes.

Furthermore, from the findings, it was revealed that the lecturers’ use of modern technologies in teaching generally increased after completing the professional development program (ADAPTI). This assertion was made on the basis that the lecturers' overall mean responses on the extent of use of modern technologies in teaching in the post-survey responses were greater than their mean responses in the pre-survey and that the mean difference recorded was statistically significant. The finding is consistent with Lavonen et al. (2006) and Giordano (2008) who found that, after completing a professional development program, the participants’ extent of utilization of technologies for instructional purposes was greatly improved. In line with Lavonen et al. and Giordano’s observations, this present study recorded a similar general significant improvement on the lecturers’ extent of use of modern technologies in teaching for improving their self-efficacy after completing the PDP. Similarly, this study has found that the lecturers’ level of knowledge of modern technologies after completing the PDP was greater than the extent to which they use modern technologies in the teaching and learning processes. This finding agrees with Brinkerhoff (2006), Fragkouli and Hammond (2007), and Glazer et al. (2009) that although professional development programs increase the participants’ acquisition of technology skills, the extent of use of such skills are still very low and had very limited impact on classroom practice. The study also established that the lecturers’ self-efficacy in students’ engagement, use of instructional strategies, and classroom management was high. This agrees with Tschannen-Moran and Hoy (2001) and Udu (2020) that professional development programs enhance professional proficiency and self-efficacy in the teaching and learning processes. The researchers observed from the findings of this study that the lecturers still experience some challenges in the use of modern technologies in the teaching and learning processes in their classrooms.

Efforts were made by the researchers to determine the reasons for the low level of use of modern technologies, after the completion of a training program by the lecturers, showed that many factors were responsible. Some of the responsible factors they listed include; non-availability of modern technologies, specifically computers; erratic power supply; lack of internet access in the schools; high cost of internet connectivity, etc. It was observed that the presence of most of these factors in the universities is seriously hampering the effective utilization of modern technologies in the teaching and learning processes in most tertiary institutions in the developing countries, including Nigeria (Ayoub, Petra & Joke, 2016).

7. The Implication for Sustainable Science Education in the Developing Nations Somayyeh et al. (2018) identified education generally and science education in particular as a key factor in human development. This is because science

http://ijlter.org/index.php/ijlter

74

education is very vital in the scientific and technological development of any nation (Udu, 2018). It has been observed that recent global changes in society, in the areas of the economy, and politics may harm education. To checkmate, these global changes require effective science and technological education in the universities. Besides, Mohammad and Khaled (2017) indicated that the changing roles of educators as a result of the global changes and expectations of the society require constant training and retraining to improve their quality. The training can be achieved through regular organization and participation in professional development programs. Moreover, regular engagement of the science lecturers to PDP has been found to enhance their capacity to achieve effective teaching and improve their self-efficacy, which may lead to sustainable science education, especially in developing nations such as Nigeria. Sustainable science education is seen in this study as the science lecturers’ acquisition of fundamental knowledge, skills, and positive attitudes in the use of technologies for effective teaching and learning processes in science education.

Furthermore, previous studies that highlighted the benefits of professional development programs emphasized that PDP enhances the participants’ ability to learn and develop the knowledge and essential skills required for good professional practices with technologies, for a sustainable education system (UNESCO, 2005; Buczynski & Hansen, 2010; Gabriel et al., 2011; Besong & Holland, 2015).

Consequently, Knapp (2003) reiterated that professional development programs are vital links to improving the effectiveness of the participants. Therefore, science lecturers should be encouraged to regularly attend PDPs for opportunities to update their professional knowledge and skills, especially in modern technologies, to help in improving their self-efficacy in science teaching and learning for sustainable science education in developing nations.

8. Conclusion and Recommendations This study has shown that professional development programs (PDPs) can help to enhance science lecturers’ knowledge and use of modern technologies for improving their self-efficacy in teaching in the areas of students' engagement, instructional strategies, and classroom management in the universities in a developing nation, Nigeria. The finding emphasizes that when lecturers have acquired the knowledge and make effective use of modern technologies, they can apply such knowledge and skills in their teaching processes. This would help to enhance the lecturers teaching effectiveness, and subsequently, result in engaging the students actively in the classrooms, as well as motivating them for effective learning. The lecturers should, therefore, strive to attend professional development programs to acquire skills for incorporating the use of modern technologies into their daily teaching processes in their classrooms. The science lecturers, as well as other lecturers in the universities, should note that teaching with technologies is not only about keeping up to date with all the latest modern teaching tools available, but it involves effective utilization of the technologies in the teaching and learning processes. The science lecturers and others should be encouraged by the university management and other higher education

http://ijlter.org/index.php/ijlter

75

stakeholders to regularly engage in PDPs for improving their self-efficacy in teaching for sustainable science education. The management of tertiary institutions should establish a feedback mechanism for regular monitoring of lecturers that have participated in PDPs for proper and regular application of the knowledge of technologies acquired during the training. Finally, the lecturers should be assisted to procure technological gadgets, internet access, constant power supply, etc. to enhance their effectiveness in the use of modern technologies in science teaching for improving their self-efficacy and sustainable science education.

9. Limitations of the study The researchers could not sample all the lecturers that participated in the professional development program due to logistics reasons. This might affect the generalization of the results of this study.

Conflict of Interest Statement The authors have no conflict of interest to declare.

10. References Ayoub, K., Petra, F., & Joke V. (2016). Factors affecting teachers’ continuation of technology use in teaching. Education and Information Technology, 21(6), 1535– 1554. doi:10.1007/s10639-015-9398-0 Barni, D., Danioni, F., & Benevene, P. (2019). Teachers’ Self-Efficacy: The Role of Personal Values and Motivations for Teaching. Frontiers in Psychology, 10, 1645. doi:10.3389/fpsyg.2019.01645 Besong, F., & Holland, C. (2015). The dispositions, abilities, and behaviors (DAB) framework for profiling learners' sustainability competencies in higher education. Journal of Teacher Education for Sustainability, 17(1), 5–22. doi:10.1515/jtes-2015-0001 Brinkerhoff, J. (2006). Effects of a long-duration, professional development academy on technology skills, computer self-efficacy, and technology integration beliefs and practices. International Society for Technology in Education, 39(1), 22–43. doi:10.1080/15391523.2006.10782471 Buczynski, S., & Hansen, C. B. (2010). Impact of professional development on teacher practices: Uncovering connections. Teaching and Teacher Education, 26(3), 599–607. doi:10.1016/j.tate.2009.09.006 Caprara, G. V., Barbaranelli, C., Steca, P., & Malone, P. S. (2006). Teachers’ self-efficacy beliefs as determinants of job satisfaction and students’ academic achievement: A study at the school level. Journal of School Psychology, 44(6), 473–490. doi:10.1016/j.jsp.2006.09.001 Chen, C. H. (2008). Why do teachers not practice what they believe regarding technology integration?

The Journal of Educational Research, 102(1), 65–75. doi:10.3200/JOER.102.1.65-75 Chong, W. H., Klassen, R. M., Huan, V. S., Wong, I., & Kates, A. D. (2010). The relationships among school types, teacher efficacy beliefs, and academic climate: Perspective from Asian middle school. Journal of Educational Research, 103(3), 183190. doi:10.1080/00220670903382954 Davies, R., Sprague, C., & New, C. (2008). Integrating technology into a science classroom: An evaluation of inquiry-based technology integration. In D. W. Sunal, E. L. Wright, & C. Sundberg (Eds.), The impact of technology and the

http://ijlter.org/index.php/ijlter

76

laboratory on K–16 science learning series: Research in science education (pp. 207–237).

Information Age Publishing.

Digital Bridge Institute, DBI. (2018). Advanced digital appreciation program for tertiary institutions (ADAPTI). Nigerian Communications Commission.

Djigic, G., Stojiljkovic, S., & Doskovic, M. (2014). Basic personality dimensions and teachers’ self-efficacy. Procedia-Social and Behavioural Sciences, 112, 593–602. doi:10.1016/j.sbspro.2014.01.1206 Fragkouli, E., & Hammond, M. (2007). Issues in developing programs to support teachers of philology in using information and communications technologies in Greek schools: A case study. Journal of In-Service Education, 33(4), 463–477. doi:10.1080/13674580701687849 Fullan, M. (2015). The new meaning of educational change (5th Ed). New York: Teachers College Press.

Gabriel, R., Day, J. P., & Allington, R. (2011). Exemplary teacher voices on their development. Phi Delta Kappan, 92(8), 37–41. doi:10.1177/003172171109200808 Glazer, E. M., Hannafin, M. J., Polly, D., & Rich, P. (2009). Factors and interactions influencing technology integration during situated professional development in an elementary school. Computers in the Schools, 26(1), 21–39. doi:10.1080/07380560802688257 Giordano, V. A. (2008). A professional development model to promote internet integration into p-12 teachers' practice: A mixed-methods study. Computers in the Schools, 24(3), 111–123. doi:10.1300/J025v24n03_08 Goktaş, Y., Yildirim, Z. & Yildirim, S. (2008). A review of ICT-related courses in preservice teacher education programs. Asia Pacific Education Review. 9(2), 168–179. doi:10.1007/BF03026497 Hennessy, S., Ruthven, K., & Brindley, S. (2005). Teacher perspectives on integrating ICT into subject teaching: commitment, constraints, caution, and change. Journal of Curriculum Studies, 37(2), 155-192. doi:10.1080/0022027032000276961 Ho, I. T., & Hau, K. T. (2004). Australian and Chinese teacher efficacy: Similarities and differences in personal instruction, discipline, guidance efficacy and beliefs in external determinants. Teaching and Teacher Education, 20(3), 313-323. doi:10.1016/j.tate.2003.09.009 Karehka, R. (2013). The use of technology – in education and the teaching process. https://www.useoftechnology.com/the-use-of-technology-in-education/ Kennedy, A. (2005). Models of Continuing Professional Development: A Framework for Analysis.

Journal of In-service Education, 31(2), 235–250. doi:10.1080/13674580500200277 Klassen, R. M., & Tze, V. M. C. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review, 12, 59–76. doi:10.1016/j.edurev.2014.06.001 Knapp, M. S. (2003). Professional development as a policy pathway. Review of Research in Education, 27(1), 109–157. doi:10.3102/0091732X027001109 Lavonen, J., Juuti, K., Aksela, M., & Meisalo, V. (2006). A professional development project for improving the use of information and communication technologies in science teaching. Technology, Pedagogy, and Education, 15(2), 159–174. doi:10.1080/14759390600769144 Lau, S., Liem, A. D., & Nie, Y. (2008). Task and self-related pathways to deep learning: The mediating role of achievement goals, classroom attentiveness, and group participation. British Journal of Educational Psychology, 78(4), 639-662. doi:10.1348/000709907x270261

http://ijlter.org/index.php/ijlter

77

Liu, S.-H. (2012). Teacher professional development for technology integration in a primary school learning community. Technology, Pedagogy, and Education, 22(1), 37–54. doi:10.1080/1475939x.2012.719398 Lynch, M. (2017). Seven (7) ways technology is impacting modern education. https://www.thetechedvocate.org/7-ways-technology-impactingmodern-education/ Marshall, S. (2010). Change, technology, and higher education: Are universities capable of organizational change? ALT-J: Research in Learning Technology. 18(3), 179– 192. doi:10.1080/09687769.2010.529107 McCarney, J. (2004). Effective models of staff development in ICT. European Journal of Teacher Education, 27(1), 61-72. doi:10.1080/0261976042000211801 Moe, A., Pazzaglia, F., & Ronconi, L. (2010). When being able is not enough: The combined value of positive affect and self-efficacy for job satisfaction in teaching. Teaching and Teacher Education, 26(5), 1145-1153. doi:10.1016/j. tate.2010.02.010 Mohammad, M., & Khaled, M. (2017). Exploring change in EFL teachers’ perceptions of professional development. Journal of Teacher Education for Sustainability, 19(1), 22– 42. doi:10.1515/jtes-2017-0002 Organization for Economic Cooperation and Development, OECD. (2009). Creating effective teaching and learning environments: First results from TALIS. https://www.oecd.org/berlin/43541636.pdf Padwad, A., & Dixit, K. (2011). Continuing professional development: An annotated bibliography.

Delhi: British Council. http://www.teachingenglish.org.uk/sites/teacheng/files/CPDbiblio.pdf Somayyeh, G., Seyed, E. M. J., & Fereydoon, S. (2018). Learning to Be: Teachers' competencies and practical solutions: A step towards sustainable development.

Journal of Teacher Education for Sustainability, 20(1), 20–45. doi:10.2478/jtes-20180002 Sousa, C. M. P., Coelho, F., & Guillamon-Saorin, E. (2012). Personal values, autonomy, and self-efficacy: evidence from frontline service employees. International Journal of Selection and Assessment, 20(2), 159–170. doi:10.1111/j.1468-2389.2012.00589.x Speck, M., & Knipe, C. (2005). Why can't we get it right? Designing high-quality professional development for standards-based schools (2nd ed.). Thousand Oaks: Corwin Press.

Tareq, M. (2020). Efficacy of teachers’ in‐service training for increasing their Knowledge of attention deficit hyperactivity disorder in Eastern Region, Saudi Arabia.

International Journal of Learning, Teaching and Educational Research, 19(12), 295-312. doi:10.26803/ijlter.19.12.16 Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783–805. doi:10.1016/S0742051X(01)00036-1 Udu, D. A. (2018). Innovative practices in science education: A panacea for improving secondary school student's academic achievement in science subjects in Nigeria.

Global Journal of Educational Research, 17(1), 23 - 30. doi:10.4314/gjedr.v17i1.4 Udu, D. A. (2020). Impact of teacher professional development programs on science lecturers' knowledge and use of modern technologies in teaching in tertiary institutions. Journal of the Nigerian Academy of Education, 16(1), 111-128.

UNESCO. (2005). Guidelines and recommendations for reorienting teacher education to address sustainability. Education for Sustainable Development in Action Technical Paper No. 2, UNESCO Education sector.

US Department of Education. (2011). Use of technology in teaching and learning. https://www.ed.gov/oii-news/use-technology-teaching-and-learning

http://ijlter.org/index.php/ijlter

78

Usher, E. L., & Pajares, F. (2008). Sources of self-efficacy in school: Critical review of the literature and future directions. Review of Educational Research, 78(4), 751- 796. doi:l0.3l02/0034654308321456 Voogt, J., Almekinders, M., van den Akker, J., & Moonen, B. (2005). A `blended' inservice arrangement for classroom technology integration: Impacts on teachers and students.

Computers in Human Behaviour, 21(3), 523–539. doi:10.1016/j.chb.2004.10.003 Whitcomb, J., Borko, H., & Liston, D. (2009). Growing talent: Promising professional development models and practices. Journal of Teacher Education, 60(3), 207–212. doi:10.1177/0022487109337280 Yoo, J. H. (2016). The effect of professional development on teacher efficacy and teachers’ self-analysis of their efficacy change. Journal of Teacher Education for Sustainability, 18(1), 84–94. doi:10.1515/jtes-2016-0007

http://ijlter.org/index.php/ijlter

79

Appendix 1: Lecturers’ Self-Efficacy Scale on Knowledge and Use of Modern Technologies

Section A: Personal/Bio-data i. ii. iii. iv. v.

Name of Institution: ..................................................................................

Department: ………………………………………………………………

Rank: ………………………………………………………………………

Age: ………………………………………………………………………..

Length/Number of Years of Teaching: ………………………………..

vi.

Gender: Male

Female

Others

Section B: Questionnaire items on Knowledge and Use of Modern Technologies in Teaching Instruction: Tick √ in the option that best describes your opinion on the level of knowledge and extent of utilization of modern technologies, using the following keys; Very Large Extent (VLE) = 5, Large Extent (LE) = 4, Moderate Extent (ME) = 3, Small Extent (SE) = 2, Very Small Extent (VSE) = 1.

Part 1: Lecturers’ Knowledge of Modern Technologies in Teaching SN

Item

VLE

LE

ME

Knowledge of Modern Technologies for Efficacy in Students’ Engagement (KESE) 1

To what extent can your knowledge of modern technologies help you to motivate students who show low interest in school work?

2.

To what extent can your knowledge of modern technologies help you to revive students’ interest and believe that they can do better in schoolwork?

3.

To what extent can your knowledge of modern technologies enable you to help your students to value learning?

4.

To what extent can your knowledge of modern technologies enable you to assist families in helping their children to do well in school?

Knowledge of Modern Technologies for Efficacy in Instructional Strategies (KEIS) 5.

To what extent can your knowledge of modern technologies assist you to craft good questions for your students?

6.

7.

8

To what extent can your knowledge of modern technologies enable you to use varieties of assessment strategies?

To what extent can your knowledge of modern technologies assist you to provide alternative explanations, or examples when students are confused?

To what extent can your knowledge of modern technologies help you to implement alternative instructional strategies in your classroom?

http://ijlter.org/index.php/ijlter

SE

VSE

80

Knowledge of Modern Technologies for Efficacy in Classroom Management (KECM) 9.

To what degree can your knowledge of modern technologies help you to control students' disruptive behavior in the classroom?

10.

To what extent can your knowledge of modern technologies help you to get students to follow classroom rules?

11 To what extent can your knowledge of modern technologies assist your students in maintaining decorum in the classroom?

12 To what extent can your knowledge of modern technologies help you to establish a classroom management system with groups of students?

Part 2: Lecturers’ Use of Modern Technologies in Teaching Use of Modern Technologies for Efficacy in Students’ Engagement (UESE) 13 To what extent do you use modern technologies to motivate students who show low interest in school work?

14

To what extent do you use modern technologies to revive students’ interest and believe that they can do better in schoolwork?

15 To what extent do you use modern technologies to help your students to value learning?

16 To what extent do you use modern technologies, to assist families in helping their children to do well in school?

Use of Modern Technologies for Efficacy in Instructional Strategies (UEIS) 17 To what extent do you use modern technologies to craft good questions for your students?

18

To what extent do you use modern technologies to access varieties of assessment strategies?

19 To what extent do you use modern technologies to provide alternative explanations or examples when students are confused?

20 To what extent do you use modern technologies to implement alternative instructional strategies in your classroom?

Use of Modern Technologies for Efficacy in Classroom Management (UCM) 21 To what extent do you use modern technologies to control disruptive behavior in your classroom?

22 23

24

To what extent do you use modern technologies to get learners to follow classroom rules?

To what extent do you use modern technologies to assist your students to maintain decorum in the classroom?

To what extent do you use modern technologies to establish a classroom management system with groups of students?

http://ijlter.org/index.php/ijlter

81

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 81-103, February 2021 https://doi.org/10.26803/ijlter.20.2.5

A Competence Model to Assess and Develop Designing Competence Assessment Tool Do Huong Tra Hanoi National University of Education, Hanoi, Vietnam https://orcid.org/0000-0001-5905-7667 Nguyen Thi Dieu Linh Hanoi National University of Education, Hanoi, Vietnam https://orcid.org/0000-0002-7342-423X Abstract. Studies showed that competency-based assessment improvement had generated a much greater impact on students’ achievements on standardized tests than other forms of educational activities. However, studies also indicated a number of challenges for teachers when designing competence assessment tools (CAT), especially in building assessment tasks that replicate real-life practice. There have been many different models for teachers’ assessment competencies, but the competency of designing competence assessment tools (CDCAT) was not paid much attention. In hope to develop a competency model that would serve as a supporting role in developing the CDCAT for preservice teachers and teachers, this study used a multi-step development process to construct a teachable model that reflectived the complexity of designing assessment tools. The model consisted of 12 behavioural indicators spreading across four dimensions informed by the existing literature and empirical findings in particular contexts. To guarantee the content value of the proposed model, the research twice used the expert method by two panels. The reliability of the model was tested by analyzing the data collected from the survey with students. Interesting findings were met, and the outlined CDCAT model assisted pre-service teachers in solving issues related to their assessment competence. The model was intended for educational researchers, educators, teachers, and policy makers to support teachers’ assessment competence concerning the current accountability model across educational systems.

Specific implications for developing pre-service teachers' CDCAT were discussed, followed by suggestions for future studies.

Keywords: assessment tool; CDCAT; competence model; designing

1. Introduction To help students develop 21st-century skills, teachers should be equipped with knowledge and skills to select, adapt, and design classroom assessment tools ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

82

following curriculum-based competencies. Black and Wiliam (1998), in their meta-analysis of 250 empirical studies, indicated that the formative assessments brought about a positive impact on teachers' everyday teaching practice which could be seen in students’ achievement on standardized tests, in which students performed better and more quickly in solving complex tasks. In other words, they proved to possess more effective strategies in dealing with problems. This study also showed firm evidence that the assessment encouraged selfmonitoring, fostered self-reflection, and generated a greater level of students' commitment to their learning. Thus, the assessment competencies contributed a great deal to the students' academic success.

A main body of study indicated that teachers have not been provided with proper training and, therefore, were incapable of designing effective assessment tools (Bol et al., 1998). It has been shown in many empirical studies worldwide (DeLuca & Klinger, 2010; Volante & Fazio, 2007) that a large number of teachers demonstrated a weak competence assessment, including CDCAT. These studies at the same time determined that these teachers needed to improve their assessment competence via training on classroom assessment and testing.

In Vietnam, there is very little time to develop assessment competence for pedagogical students in the curricula of pedagogical universities. Therefore, despite being trained by the Vietnamese Ministry of Education and Training on the innovation of assessment and testing competencies, teachers still face many difficulties in designing competency assessment tools (Linh & Tra, 2016). This indicated the important role of initial training at pedagogical universities.

Recognizing the importance of innovation in assessment and testing in general and CDCAT in particular, some pedagogical universities have included the module of "Assessment and Testing in Education" in their curricula.

According to Griffins (2015), the training and development of any competencies should be based on the model of that competence. Building a detailed model for CDCAT that included behavioural indicators and quality criteria can help to orient learning activities towards building each behavioural indicator. Such a model was also the basis for designing tools to evaluate the levels that learners achieved, thereby adjusting learning activities.

Many assessment literacy standards for teachers around the world and many studies on competence assessment models were available (Wiggins & McTighe, 2005; Brookhart, 2011; DeLuca & Klinger, 2010; Fulcher, 2012; Alonzo, 2016). In these publications, the CDCAT was only one component of such models.

However, according to the present literature review, these studies were fairly and widely conducted and therefore provided little information on behavioural indicators of CDCAT. Besides, these said models have not been appropriately considered the complexity of designing assessment tasks in competency assessments.

Therefore, this study aimed to construct a teachable model that is reflective of the complexity of designing assessment tools, including behavioural indicators

http://ijlter.org/index.php/ijlter

83

and quality criteria that can help lecturers determine the objectives and design activities for developing pre-service teachers' CDCAT. The use of the multi-step process to build the model as well as the model itself will be covered in the next sections of this article.

2. Study Methods and their Procedure Vital components of the competence assessment model proposed by Griffin (2015), which was adopted in this study, include elements, behavioural indicators, and quality criteria. There were three popular methods to build the competence model: (1) Traditional survey with (a) defining the structure to be measured, (b) building the item pool, (c) obtaining experts' review to the items, (d) testing the items, (e) making factor analysis (DeVellis, 2003); (2) Measuring competence with (a) defining the structure to be measured, (b) observing and interpreting performance, and (c) statistically modeling the reliability and validity of the scores produced (Shavelson 2013); (3) Delphi study with brainstorming, narrowing down and ranking (Okoli, 2014; Naresh Giangrande et al, 2019).

On a basis of studying and synthesizing these methods, the study has used a multi-step development process to develop a CDCAT model, in which the researchers have used all the three methods above, but the Delphi method was the mainstream. Specifically, the following steps have been performed: (i) Reviewing the existing literature on the model of assessment competencies, procedures, and literacy standards for teachers in the past and present to determine the components and behavioural indicators of the model was the first step in framing the CDCAT model. It is significant to be aware of the required national quality assurance system standards based on the perceptions of the requirements of teachers' assessment competency standards to ensure the quality of countries, review of existing studies on assessment competency and assessment tool design process, and accounting for content analysis to identify the most important criteria to form the basic structure of the CDCAT model. (ii) Consulting experts (for the first time) on the components and behavioural indicators of the model was the first phase of the Delphi method (Okoli, 2014).

Brainstorming could be an efficient way to gather and combine expert opinions without seeking convergence, but emphasizing the originality and diversity of the ideas was worthy. Therefore, the investigators have organized two sessions; brainstorming in the first session and applying the expert method to determine the value of the model in the second session. The researchers chose five experts, active lecturers, who taught the module of "Assessment and Testing in Education" or performed the study on assessments. Experts who worked independently were asked to list relevant factors in a random order. Then, the investigators consolidated the lists from all experts and the list gained from the literature review, removed exact duplicates, and guaranteed terminology inconsistency. This list was sent to five experts for validation, and they were asked to give opinions and suggestions for changes if any. When receiving the

http://ijlter.org/index.php/ijlter

84

written response from them, the researchers directly discussed with them to clarify their opinions as well as investigators’ arguments.

Experts were invited to consider whether the model had only behavioral indicators related to the basic structure of CDCAT, also whether it could provide teachers the foundation to help the student along the developmental progressions and enable the identification of indicative behaviors that could be used for interpreting students’ performance. At the end of this step, refining the final version of the consolidated lists took place. (iii) A survey on self-assessment of pre-service teachers at Hanoi National University of Education (HNUE) was conducted to determine the content validity of the initial model. The HNUE's students come from many different regions of the country and the students participating in the survey were selected at random to avoid regional bias (20% male and 80% female). The survey was conducted with 60 pre-service teachers in the second semester of the third academic year when they completed the module "Assessment and Testing in Education". Each student was required to complete a questionnaire, in which they had to assess the level of confidence for each behavioral indicator of the CDCAT. Before the survey, the questionnaire was piloted for with pre-service teachers to check whether the proposals of the scale created interpretation problems for people who were not familiar with the CDCAT model. Based on the data obtained from semi-structured interviews and first stage data, the researchers analyzed and modified, consolidated, or removed original sentence structures, terminology or wording to keep pace with sample observations.

Students involved in developing models increased their usability and helped students understand, develop, and use their understanding of task requirements. It was expected to help them "think critically about their work."

(Huba & Freed, 2000). Therefore, before filling in the questionnaire, students were asked to carefully read the behavioural indicator of the CDCAT and suggested changes in the irrational points. This step could be considered to be a single phase in the traditional survey approach. However, for the small sample, it was sufficient to calculate the Cronbach Alpha coefficient. Instead of the second step in the Delphi study, this step was intended to reduce the number of factors. (iv) The following step concerned with reviewing the literature to propose quality criteria. At this phase, each evaluation task was analized in the assessment tools to identify quality criteria that were disclosed through each assessment task. The quality criteria of each behavior indicator were aggregated and reviewed to determine if they adequately reflected the achievable levels by each behavior indicator. Typical quality criteria were chosen for reference when proposing quality criteria. (v) In this phase, the investigators analyzed the collected practical data during training and retraining of the CDCAT and pre-service teachers, in combination with interviews with pre-service teachers' cognition to propose quality criteria.

This step could be considered to be the observation phase of performance and interpretation of performance in line with the second approach. To limit shortcomings of many large-scale surveys, Shavelson (2013) indicated that even when using statistical models, there is a need for evidence of cognitive validity;

http://ijlter.org/index.php/ijlter

85

the tasks evoked the types of thinking and reasoning that became part of the inference to make judgments about competence. In this sense, such evidence was collected through the "think-aloud" method, whereby students had told their thoughts as they performed a task (Leighton, 2004). The results of think-aloud with pre-service teachers about their perceptions were used to collect information on how students participated in the design of assessment tools to identify the key factors for their success It also helped to determine the level of internal development of learners concerning their obtained product. Students' product samples represented different levels of performance and showed up observable criteria in behaviours (Tierney & Simon, 2004). Therefore, the assessment tools designed by the learner were classified according to the levels of each behavioural indicator. To determine learners’ levels, the product samples selected for analysis included students from the lower, middle, and upper groups of the class in terms of overall learning progress. The results of this analysis were combined with the results of the above stated step to propose quality criteria. (vi) With regard to the above steps, consulting experts (for the second time) on the complete CDCAT was concsidered. This step was similar to phase 3 of Delphi method. In order to strike a balance between "generalized wording" for increased applicability and "detailed description" for reliability, there should be cooperation between interdisciplinary instructors when describing the criteria (Suskie, 2004). 23 experts and active lecturers, who taught "Assessment and Testing in Education" for students from nine faculties at pedagogical universities (Math, Physics, Chemistry, Technology and Education, Biology, Literary, History, Geography, Information Technology) and experts in the Center for Educational Assessment and Quality assurance, were selected. The oldest was 60 years old and the youngest expert was 33 (M =42.69; SD = 8.99). The minimum period of teaching experience among the expert samples was eight years and the maximum was 35. In total, 11 women and 12 men from HNUE, VNU University of Education, and also Vinh University participated in the study. Experts were encouraged to examine the suitability of the model's quality criteria with the students’ product and reviewed if the model had enough difficulty levels to distinguish the students’ CDCAT levels. The following questions were asked for experts for reflect: 1. Do the criteria of the model address all aspects intended to be measured in the assignments given by you?; 2. Are all the important criteria relevant to the assessment method being evaluated through the model?; and 3. Do the criteria reflect competencies that will suggest success on future or related performances?” (Moskal & Leydens, 2000).

If the experts disagreed with any points in terms of content and expression, asking them for advice on how best to adjust it was considered. A direct discussion with the experts also took place upon receiving their written responses. Besides, the investigators shared feedback with each panelist and asked them to re-rank each list. This was repeated until the members reached an agreement or had a high consensus.

http://ijlter.org/index.php/ijlter

86

3. Content and Research Results 3.1. The concept of competence of designing assessment tools Assessment competence is related to the understanding and appropriate use of assessment practices, as well as the theoretical and philosophical background in measuring student learning (Stiggins 2002; Volante & Fazio 2007). Another simpler definition was provided by the North Central Regional Educational Laboratory (2016), in which “Assessment competence is the willingness of an educator to design, implement and discuss assessment strategies”. The above definition indicated that the CDCAT is a component of assessment competence.

In recent studies on teacher competencies, assessment competence has been mentioned (Caena, 2011; OECD, 2018). However, since assessment competence is only one dimension of teacher competence, CDCAT is also only one component of the assessment competence, this is why CDCAT has not been covered in detail in these studies.

Therefore, to define and clarify the model of CDCAT, an analysis of the Stiggins assessment competence model and 12 assessment literacy standards for teachers (from 1990 to present) from six geographic regions was conducted (USA, UK, Canada, Australia, Europe, and New Zealand). Through this analysis, four themes representing contemporary aspects of CDCAT were identified:(1) Defining the purposes and objectives of the competency assessment activities; (2) Planning the development of the assessment tool; (3) Developing assessment tool; and (4) Testing and editing the assessment tools. What had been reviewed suggested that competence of designing assessment tools was to connect assessments for clear purposes, apply proper assessment methods, and develop quality assessment exercises and scoring criteria appropriately.

3.2. Identification of Competence Components and Behavioural Indicators As stated in part 2, the study of the CDCAT model was conducted in 6 steps.

The results obtained through each step were as follows: Step 1: The literature review to determine the components and behavioral indicators of the model To determine the components and behavioural indicators of the model, all criteria related to CDCAT from previous publications were synthesized. All publications built assessment competency models to guide the activities fostering teachers' assessment competencies in general, but each publication focused on specific points, depending on the evaluation trends at that time.

Therefore, in this section, to take a complete look at the CDCAT, it was crucial to synthesize all the criteria related to the CDCAT from the publications. i. The CDCAT in the models of assessment competence The criteria relating to CDCAT in the publications were picked up and classified into topics representing contemporary aspects of CDCAT. For overlapped standards, more general criteria were chosen to include in the synthesized table, so as not to miss out on criteria related to CDCAT. None of the standards mentioned comprehensively covered all the important teacher assessment competencies. Except for the Standards for Teacher Competence in the American Educational Assessment of Students published in 1990 (ASTCEAS), most of these standards were only outlined generally, with incomplete requirements for

http://ijlter.org/index.php/ijlter

87

teachers' assessment competency, and components of the CDCAT were not clear. This study considered models of teachers' assessment competency proposed by Stiggins (1999), DeLuca and Klinger (2010), Brookhart (2011), Fulcher (2012), Alonzo (2016), and ASTCEAS (1990) to determine the components of the CDCAT. Table 1 shows the results of the analysis of those models. For each model of teachers' assessment competency, its components were classified into four groups corresponding to the four components of CDCAT (Row 1).

Table 1: Comparison of components from the existing model of Teacher Assessment competency I. Determine the purposes and objectives of the competency assessment activities ASTCEAS (1990)

II. Plan the development of the assessment tool

Choosing assessment methods.

Stiggins (1999)

Connecting assessment to clear purposes.

DeLuca & Klinger (2010)

Assessing under philosophies ofclassroom assessment and philosophies of large-scale assessment.

III. Develop anassessment tool

Finding out unethical, illegal, and inappropriate assessment methods

Conduct assessment and use assessment results

Administering, scoring, and interpreting.

Using assessment results when making decisions.

Communicating assessment results.

Applying Preparing quality Avoiding bias in Using an proper assessment the assessment. assessment asthe assessment exercises,scoring instructional methods. criteria and intervention.

Clarifying sampling in an Communicating Achievementex appropriate effectively about pectations. manner. student achievement.

Using and being Designing and Using statistical Implementing aware of learning marking the test. techniques for provincially practices and Taking assessment. mandated theory. theoretical Defining reliability assessment Using types of principles in and validity issues. practices. assessment. assessment of Giving the learning and rationale for assessment for assessment learning. decisions and Applying and practices. having technical knowledge of summative assessment practice and assessment item formats.

http://ijlter.org/index.php/ijlter

Buildingassessme nt methodsCreating valid pupil gradingprocedur es.

IV. Trial and finalize the assessment tool

88

Brookhart (2011)

Fulcher (2012)

Alonzo 2016

Defining clearlythe teachingknow ledge and method to achieve; - Identifying the learning outcomes aboutcurricul um goals andstandards

Planning strategies for discussion about the learning outcomes that are going to be assessed with the students

- Obtaining a good understanding of the efficiency of how the available assessment alternatives work; - Building a tool that can assess the students’ performance through a scoring system with helpful data.

Learning about Designing and and developing the applyinglarge- test. scalestandardiz ed testing,classroo m testing, and washback.

Designing assessment tasks.

Using rubrics to assess students’ learning.

- Learning about whether or not the assessment tools the teachers are using meet the intended learning outcomes including the required knowledge and thinking skills.

Being valid reliable

Providing effective, useful feedback on student work.

Offeringconstructi ve feedback on students’ performance; Helping students use assessment information to make sound educational decisions.

Supportingstudent s to make decision on their assessment - Giving ethical considerations on the administration of the assessment being used.

and

Considering factors that affect students' performance.

Securing task completion in any form.

Holding up dialogue/ conversation with colleagues to ensure consistent, fair, and comparable judgment of students’learning.

ii. The behavioural indicators of the CDCAT in the processes of designing assessments The models in the literature listed in table 1 did not demonstrate the behavioural indicators of the CDCAT. Therefore, a process-based approach based on analysis of the thinking process was chosen when designing an assessment tool to infer the behavioural indicators of the CDCAT. In this regard, this study analysed the processes of designing assessments that have been popularly used in research by Stiggins (1987), Grant Wiggins (2005), Brookhart (2011), Tillema (2011), and National Register for Vocational Education and Training in Australia (2012). To

http://ijlter.org/index.php/ijlter

89

draw out behavioural indicators for the CDCAT model (table 2), the researchers compared the actions for each step of one process with another.

Table 2: Comparison of steps from the existing process of assessment design I. Determine the purposes and objectives of the competency assessment activities Stiggins 1. Clarifyingreason (1987) (s) for assessment • Specifying decision(s) to be made • Nominatingde cisionmaker(s) • Giving used to be made of results • Describing students to be assessed

II. Plan the development of the assessment tool

III. Develop anassessment tool

Defining Designingexercise performance to be s evaluated • Selectingtypes • Providing the of exercises detailed content • Determining or skill focus of obtrusiveness of the assessment assessment • Selecting the • Defining the type of amount of performance to evidence needed be evaluated 2. Making • Listingperforma performance nce criteria rating plan • Choosing the type of score needed • Specifying who is to rate performance • Clarifying the score recording method Wiggins & Identifying Making out • Determining McTighe desired results performance tasks acceptable (2005) students will • the big ideas evidence completedemonstr • desired specific • that students ating the desired understandings will understanding • predictable demonstrate the misunderstanding achievement of • Setting criteria by which s the desired performances will results • the way be judged students will reflect upon and self-assess their learning Brookhart • Specifying the • Making • Designing (2010) kind of decision on assessment task. thinking and what you will Planninga of the content take as balance content and you wish to evidence that see evidence the student has thinking by an for. exhibited this assessment kind of blueprint. thinking about the

http://ijlter.org/index.php/ijlter

IV. Trial and finalize the assessment tool

Conduct assessment and use assessment results

90

appropriate content Tillema, (2011)

• Indicating purpose or goal of the assessment.

Australia( 2012)*

1.1 Identifying the target group of candidates, purpose of assessment tools, and contexts 1.2 Accessing relevant benchmarks for assessment and interpret them 1.3 Indicating, obtaining, and interpreting organizational, legal, and ethical requirements 1.4 Specifying other related documentation

• Designing or selecting an assessment task.

• Setting criteria for the assessment task.

2.1 Selecting assessment methods that support the collection of definedevidenc e.

2.2Encouraging candidates to show or support their claim through selected assessment methods 2.3 Considering different assessment instruments for the selected assessment methods 2.4 Reviewing how the assessment instruments will be administered

3.1 Developing specific assessment instruments 3.2 Defining and preparing clear and specific procedures instructing assessor 3.3 Considering requirements of assessment system policies and procedures, addressing storage and retrieval needs, and reviewing, evaluating control procedures as part of this process

• Administering the assessment.

• Scoring the assessment.

• Making appraisal or ‘‘grading of the assessment’’.

• Giving feedback and further promotion of learning 4.1 Checking draft assessment tools against evaluationcriteri a and amend as required 4.2 Testing assessment tools to validate content and applicability 4.3 Collecting and making written feedback 4.4 Amending the final toolbased on an analysis offeedback 4.5 Appropriately formatting and filing finalized assessment toolaccording to assessment system policies and procedures as well as organizational, legal, and ethical requirements

Note. *Adapted from TAEASS502B Design and develop assessment tools, Australia Goverment Department of Education, Employment and workplace Relations (2012). All rights reserved.

iii. Synthesize criteria related to the CDCAT and propose the model The model will be used to design tasks in teaching and assessing the CDCAT. Hence, the model needs to support students and teachers in tracking their competence development and analyzing the key components of their work. The model proposed based on the CAT design process can meet this requirement.

http://ijlter.org/index.php/ijlter

91

Table 2 showed the results of content analysis of the assessment tool design process. After analyzing the common and reasonable points of these processes, the following points were added as follows: - Each assessment activity should have contextualized and diversified purposes.

To ensure this, it is necessary to specify the purposes into specific goals. In this regard, teachers can only define goals that are suitable to the situation when they clearly define its characteristics. Such a conception was confirmed by Herppich et al. (2017) who claimed that an assessment-competent teacher should be able to master a wide range of assessment-related situations relevant to the teaching profession.Therefore, at this stage, after determining the purpose of the assessment, these two steps were included: (1) Identifying the characteristics of the situations; and (2) Determining the objectives of the assessment tasks.

- “Competence involves putting into action conceptual knowledge, procedural knowledge, and attitudes to be able to resolve a particular situation” (Ananiadou & Claro, 2009). Nevertheless, it is difficult to select authentic situations to design competency assessment tasks that must meet the key criteria on Accuracy, Generality, and Extrapolation (Gulikers et al., 2005). The above processes have not considered this complexity. When analyzing the thinking process in designing assessment tasks, the following two steps were put forward before the step of designing the assessment task: (1) Specifying the type of information to be used; and (2) Searching for the type of information to be used.

Corresponding to the steps in the assessment designing process, behavioural indicators in the competence model for designing assessment tools were proposed as shown in Figure 1.

Competence of designing competence assessment tools I. Identify the assessment purposes and objectives

II. Plan the development

1. Identify the purposes 2. Identify the characteristics of the situations 3. Determine the objectives of the assessment task system

III. Develop assessment tool

4. Determine the type and amount of evidence 5. Select assessment methods

IV. Trial and edit assessement tool

6. Determine the type of information to be used 7. Search for the type of information to be used

8. Draft assessment task 9. Design assessment instrucments 10. Determine the specific steps for evaluator

Figure 1: An initial tentative model of the CDCAT

http://ijlter.org/index.php/ijlter

11. Trial the assessement tool

12. Analyze feedbacks

13. Edit the assessement tool

92

Because the selected literature wasn’t a sufficient basis to offer quality criteria, a tentative model was proposed, just by including components and behavioural indicators.

Step 2: Consult experts on the components and behavioural indicators of the model The brainstorming results were analyzed and were as follows: all the ideas presented by three out of five experts matched the behavioural indicators of 1, 5, 8, 9, 10, 11, 12, 13 of the model proposed in step 1. The remaining two experts proposed adding two behavioural indicators namely: weighting and coding test results. However, after discussing, the below agreement were reached out: • For weighting, the weight of each behavioural indicator was determined in the competence model to be assessed, and the designer of the assessment tools implemented it as a simple procedure.

• As for coding, it is possible that coding is defined in the 10th behavioural indicator. Before implementing behavioural indicator 12, coding is just a set of predefined steps that did not demonstrate the use of assessment knowledge.

• As far as analyzing is concerned, when analyzing feedback from people involved in the experimental process, the designer should use knowledge of reliability, validity, difficulty, discrimination, and some data processing softwares.

From the above analysis, the model was retained as shown in Figure 1. Then, the model was sent to the five experts followed by a direct discussion. The results revealed that all the experts agreed with the model.

Step 3: Determine the structural validity of the CDCAT tentative model The results of a self-assessment of 60 pre-service teachers were analyzed to determine the structural validity. The Cronbach's Alpha coefficient was initially calculated for all 13 behavioural indicators (Figure 1). The variable VAR0007 corresponding to the seventh behavioural indicator had a low corrected itemtotal correlation (0,268), yet this behavioural indicator was found necessary for the process of designing a competency assessment tool. However, a person who can search for information but did not have the necessary knowledge of assessments cannot design competency assessment tools, while a person with an average ability to search for information can do this if they had assessment knowledge. Thus, the experimental results were appropriate. A process-based approach was then chosen to determine the behavioural indicators of the competency model for designing competency assessments. This analysis indicated that keeping track of cognitive processes and trying to introduce the full stages of the design process without appropriately considering the necessary knowledge was a mistake. Right after that, with each behavioural indicator, the investigators proceeded to identify all the necessary knowledge to carry out those behaviours to avoid the above mistake. As a result, the remaining behavioural indicators did not have the same problems as indicator 7.

After removing the VAR0007 variable, the results showed that the Cronbach's Alpha coefficient was 0.876, in the range from 0.8 to 1, indicating that the scale was very good (See figure 2). The corrected item-total correlation coefficient of all behavioural indicators was greater than 0.3 which denoted that all

http://ijlter.org/index.php/ijlter

93

behavioural indicators were satisfactory and no behavioural indicators measured another competence. The statistical analysis showed that the above 12 behavioural indicators were well correlated with each other and measured the same variable, the CDCAT of pre-service teachers.

Figure 2: Cronbach’s Alpha coefficient for 12 behavioural indicators

3.3. Identification of the Quality Criteria Step 4: Review of the literature to propose quality criteria To propose the quality criteria, the following studies on assessment tools were analyzed: • Three tools designed based on the content of the ASTCEAS were: questionnaires of Likert-type items (Zhang & Burry-stock, 1997), questionnaires of multiple-choice questions (Mertler & Campbell, 2005), and True/False questions (Daniel & King, 1998).

• An instrument (DeLuca et al., 2016) consisted of both Likert-type items and multiple-choice questions focusing on the contents of the US 2014 Standards for Educational and Psychological Testing.

• A rubric assessing the teacher’s assessment competency (Alonzo, 2016) and a questionnaire (Jarr, 2012) consisted of 15 sentences designed on Likert-type items.

• Other tools to assess teacher assessment in specific subjects (McGee, 2012; Perry, 2013; Bremner, 2014; Gutierrez, 2014; Nary, 2014; Vidacovich, 2015; Xu & Brown, 2016; Hammami, 2016; González, 2018).

The results of the synthesis of the quality criteria of each behavioural indicator evaluated by the above assessment tools were as follows: - The tools focus a lot on evaluating some content in ASTCEAS such as item analysis, select assessment methods, and tools, interpretation ofthescore, select the test sample.These are, in turn, some essential aspects of the quality criteria of the behavioral indicators of 12, 5, 10, 9, 11 in Figure 1. However, these tools had not yet fully assessed the aspects and levels of these behavioural indicators. The same thing happened with behavioural indicators in 1, 2, 3, 4, 7, and 13.

http://ijlter.org/index.php/ijlter

94

- Some quality criteria were at the high level of the behavioural indicator 8, for example: “My methods and types of assessment allowed students to demonstrate their learning in diverse ways” or “I had thought deeply about my approach to assessment”, which were evaluated in the Approaches to Classroom Assessment Instrument (DeLuca et al., 2016).

These quality criteria were useful references in proposing quality criteria for the model. The remaining quality criteria were determined through the analysis of actual data in step 5.

Step 5: Analyze Practical Data With this model, it was expected by the researchers to have 4 levels of quality criteria. Therefore, the practical data outlined below were classified into four levels. In the summer of 2014, the Ministry of Education of Vietnam organized training courses on teachers’ assessment competencies in the whole country. A survey of 382 teachers was conducted to identify the level of difficulty that teachers encountered in designing competency assessment tools (Linh & Tra, 2016). According to the assessment tools designed by those teachers, the researchers’ observations during the teacher training process, and the level of difficulty were categorized in 4 levels (beginning, developing, accomplished, and exemplary) as a basis for proposing quality criteria.

During the academic year 2015-2016, in the module of “Assessment and Testing in Education”, 56 pre-service teachers from the Physics Faculty at HNUE were required to make their portfolios. Before each class, pre-service teachers had to read the lesson documents and determine the objectives of that lesson. After class, they recorded in their portfolios what they had learned, the difficulties they encountered, and what else they could learn. As mentioned above, the assessment tools designed by pre-service teachers, observations during the teaching process, and the data in theportfolios were also classified into 4 levels.

For three consecutive years (2015-2017), data were collected during the process of supervising nine pre-service teachers to do graduation on these topics of designing competency assessment tools. A direct discussion took place with these pre-service teachers to identify their difficulties, suggest directions, and monitor changes to their problem-solving processes. These discussions happened every time they designed new tools. First, for each behavioural indicator, pre-service teachers were grouped according to the four levels. When comparing their assessment tools with the results of cognitive interviews, learners' internal development level were determined relative to their external performance and key factors for their success whenin designing assessment tools.

The analysis of student artifacts and in-depth discussions among lecturers ensured the consistency between the descriptions in the model built based on the researcher's inference and experience with practice (Reddy, 2011). Therefore, after independent analysis, quality criteria and the SOLO taxonomy were suggested and discussed respectively to propose sound quality criteria (Biggs &

http://ijlter.org/index.php/ijlter

95

Collis, 1982). Basically, the SOLO taxonomy can be used to build scores or codes to determine the subtle perception of the subjects (in this case, the designers of competency assessment tools). This rating scale was also a simple, reliable, and easy-to-use model that matched the quality rating of those behavioural indicators. At the end of this step, combined with the result of step 4, the researchers outlined 48 quality criteria of 12 behavioural indicators mentioned in step 3. To retest the claritied behavioural indicators, all the quality criteria were practiced the scoring samples of student work.

Step 6: Consult experts on the complete competence model To ensure the validity of the model, the latter was reviewed by experts for the second time. Behavioural indicators 6 and 8 (Figure 1) required creativity. Based on the practical data above and the rubric suggested by Alonzo (2016), corresponding to each behaviour, the highest level “Instructing the behavior in a professional way for colleagues” was designed. To reach this level for each behaviour, pre-service teachers needed to draw up logical methods and rules when performing the behavior. This also corresponded to the highest level in the SOLO taxonomy. However, 52% of experts said that this was not reasonable because, in achieving those quality criteria, pre-service teachers also would be able to present information. After discussing behavioral indicators 8 (Figure 1), the highest level of the SOLO taxonomy (Extended Abstract) was expressed by “Draft assessment tasks and reflect on the implementation process to withdraw appropriate rules for the next time”.

Other feedback from the experts focused on terms and expressions, particularly for indicator which called for clarification (30% experts). Analyzing these responses led to some modifications to the quality criteria for the CDCAT as shown in Table 3.

Table 3: The full model of the CDCAT Components

I. Determine the purposes and objectives of the competency assessment activities

Behavioura l indicators 1.

Identifythe purposes of using the assessment tool 2.

Determine the characteristi cs of the situation using the tool

http://ijlter.org/index.php/ijlter

Quality criteria 1.1. State the purpose of using the assessment tool in a general way.

1.2. State clearly the familiar purposes of using the assessment tool.

1.3. State clearly and fully the purposes of using the assessment tool.

1.4. State, classify and rank the purposes of using the assessment tool.

2.1. Identify some common factors (time, acquired knowledge).

2.2. Identify some basic factors, consistent with the assessment purposes (student level, reading comprehension, number of students, facilities).

2.3. Fully determine the factors to be considered, appropriate for assessment purposes (health and psychology, language, and student readiness).

96

3.

Determine the objectives of the assessment task system

II. Plan the development of the assessment tool

III. Develop anassessment tool

4.

Determine the type of evidence and the amount of evidence to be collected to assess learners’ competenci es 5.

Select assessment methods supporting the collection of such evidence

6.

Determine the type of information used to draft assessment tasks

7.

Draft assessment tasks.

http://ijlter.org/index.php/ijlter

2.4. Determine, classify and rank the factors to be considered.

3.1. Determine the objective, only focusing on some common factors (acquired knowledge and time).

3.2. Determine the objective, focusing on the competence model to be assessed.

3.3. Determine the objective, ensuring the purposes of assessment.

3.4. Determine the objective that is relevant to the situation in which the tool is being used.

4.1. Determine the type and amount of evidence needed to assess some behavioral indicators.

4.2. Determine the type and amount of evidence needed to separately assess each behavioral indicator.

4.3. Determine the type and amount of evidence with attention to the relationships between behavioral indicators.

4.4. Determine the type and amount of evidence that fully meets the assessment objectives.

5.1. Select some assessment methods in accordance with the collection of some types of evidence.

5.2. Select assessment methods supporting the separate collection of each type of evidence.

5.3. Select assessment methods with attention to the combination of collecting different types of evidence.

5.4. Select assessment methods that achieve full assessment objectives.

6.1. Only find assessment tasks that are available for use directly.

6.2. Determine the characteristics of the information that can be immediately utilized to draft assessment tasks.

6.3. Determine the characteristics of information that can be utilized to find ideas to generate other assessment tasks.

6.4. Determine the type of information and reflect on the implementation process to withdraw appropriate rules for the future.

7.1. Draft assessment tasks only to obtain some simple assessment objectives.

7.2. Draft assessment tasks to achieve the full assessment objectives.

7.3. Draft multidimensional assessment tasks that allow students to come up with different

97

8.

Design assessment instruments (scale, checklist, rubric, etc.) to assess the evidence obtained.

9.

Determine the specific steps that evaluators should take to manage and use the tool.

10. Trial of the assessment tool

IV. Trial and finalize the assessment tool

11. Analyze feedback from people involved in the experiment al process.

12. Finalize the assessment tool

http://ijlter.org/index.php/ijlter

ways to express ideas.

7.4. Draft assessment tasks and reflect on the implementation process to withdraw appropriate rules for the next time.

8.1. Select suitable tools.

8.2.

Select appropriate assessment instruments to assess the evidence obtained.

8.3. Develop assessment instruments that meet some basic criteria according to the assessment theory.

8.4.

Develop appropriate assessment instruments to assess the evidence and create learning opportunities for students.

9.1. The content of some steps to manage and use the tool is reasonable.

9.2. The content of all the steps to manage and use the tool is reasonable.

9.3. The order of steps to manage and use the tool is basically reasonable but not optimal.

9.4. The order and content of the steps are optimal (saving time, facilitating implementation, and reducing errors).

10.1. Select the appropriate test sample.

10.2. Choose the right method for collecting experimental information.

10.3. Determine the factors affecting the readiness of the subjects participating in the experiment.

10.4. Eliminate or minimize all factors that influence the readiness of the subjects participating in the experiment.

11.1. Consider feedback and provide a general comment about the tool 11.2. Select appropriate feedback analysis methods.

11.3. Separately analyze each type of feedback.

11.4. Analyze types of feedback with attention to the relationships between them.

12.1. Consider a number of factors influencing the accuracy of the assessment tool 12.2. Determine factors that may affect the accuracy and optimality of the assessment, (assessment duration, task difficulty level, language, design, and readiness of subjects in the experiment).

12.3. Modify the assessment tool in order to fix some of those factors.

12.4. Appropriately finalize the assessment tool.

98

4. Discussion of the Obtained Results According to the review in step 1, behavioural indicators 2 and 3 were not explicitly stated in studies and assessment standards. Among the publications analyzed, only the publication of National Register for Vocational Education and Training in Australia (2012) addressed identifying the group of students to be assessed when designing the assessment tool in general, but not to mention the specific factors that need attention to define goals when designing assessment tools such as: student level, reading comprehension, health and psychology, and student readiness, etc.

However, Assessment Literacy Inventory (Mertler & Campbell, 2005) designed based on ASTCEAS had assessed some aspects of behavioural indicators 2 and 3.

This proved that there was the presence of these behavioural indicators (2 and 3) in the tacit knowledge of the assessors (Mertler & Campbell, 2005). In particular, the behavioural indicators 8 proposed by the research had not been mentioned in previous studies, but it was often grouped with behavioural indicators 9. It was the grouping of many behavioural indicators together that caused failure for many lecturers to accurately identify which behavioural indicators were responsible when designing assessment tools. As a result, it made it difficult to enhance teachers’ competencies.

Thus, the proposal of the current new behavioural indicators explicitly showed the tacit knowledge of teachers. This will support teachers and learners in detecting their own problems to improve their teaching and learning practices.

These behavioural indicators were consistent with the results of the students’ product and thinking analysis. They have also been confirmed through the expert method and Cronbach alpha analysis results. Moreover, methods of collecting and analyzing students' thinking and data processing to find new content for the model in this study could suggest a new way for teacher educators to build a model of other teacher competencies and facilitates verifying or improving the competency model.

5. Conclusion To construct a reflective model on the complexity of designing assessment tools, the literature on the process of designing the assessment model and data on students' thinking were synthesized and analyzed, respectively. The validity and reliability of the model were proved by analyzing the data from the survey with students and twice using the expert method by two panels. Overall, this study provided initial validity and reliability evidence to support the usefulness of the CDCAT model. The construction of this new model was approached in a comprehensive, mixed process (including both qualitative and quantitative) taking into account all the factors related to CDCAT. Research added to the literature the model that clarify the cognitive aspects of CAT by designing and highlighting tacit knowledge used for designing CAT. The results of this research will be the basis for the development of CDCAT for pre-service teachers. Based on behavioural indicators of the model, a lecturer can plan

http://ijlter.org/index.php/ijlter

99

training and design tasks for pre-service teachers. The level achieved by each pre-service teacher can be determined through the application of the quality criteria of the CDCAT. The next step was to design learning tasks for each behavioural group to train pre-service teachers so that they could achieve higher quality criteria. The CDCAT model had a significant value in providing additional insights into the difficulties in the cognition of teachers and the nature of teachers’ development in their designing CAT and making these visible for teacher-educators and pre-service teachers.

6. Implications for Teacher Education One of the most obvious uses of the model was to provide a basis for directing teacher competency development. There is evidence that depending on teachers' identified approaches to assessment, areas of confidence, and professional development priorities and preferences, the teacher had approaches to specialized learning or maintained other learning goals in the assessment learning (Linh & Tra, 2016). The tool provided diagnostic information about teachers' CDCAT as the foundation for developing differentiated and targeted professional learning. By viewing the development of the CDCAT as a mixture of dimensions rather than aggregating scores to obtain an overall score, the model can assist educators-teachers and teachers in responding and guiding adjustments. The quality criteria that clearly defined teachers' CDCAT expectations can be used as tools for monitoring changes and, more importantly, as learning aids. Besides, models can be used to enhance instruction by providing educators, mentors, and students with descriptions of common concepts and language to foster discussion and feedback. Furthermore, the quality criteria described in the CDCAT model can elicit teacher dialogue with questions about incorporating theory and practice into classroom assessment.

In addition, by using models and looking at patterns collected during teacher development, educators-teachers can determine when a teacher's CDCAT is growing faster and changing slowly. Also, researchers can use the model and examine samples collected from different teachers, in different contexts, to discover factors that affected teachers' CDCAT development.

The strength of this model was that it complemented the model in some behavioural indicators to clarify the cognitive aspects of CAT design (indicator 8) and highlight tacit knowledge used for designing assessment tools (indicator 2,3). With the outlined 48 quality criteria, this model also provided more detailed information about the nature of teacher development as they became more competent in CAT designing. However, because it required a lot of actions when assessing teachers' CDCAT to report in detail the level of teacher achievement in each behavior indicator, using the model in teacher training was more appropriate than in summary assessment.

7. Research Limitations The limitation of the study was that the model of CDCAT was built through the analysis of small data.Although research design with this small sample can deeply analyze the inner learner’s activities through a think-aloud and analyzing

http://ijlter.org/index.php/ijlter

100

the learning portfolio, further research should continue by measuring CDCAT on a wider sample, varied pre-service teachers, and using the Rasch model for analysis.In comparison with factor analysis, the use of the Rasch model will give more meaningful results to quantify the construct being studied (William, 2016).

The strength of the Rasch model lies in its ability to determine if the items are not related and item fit (Randall & Engelhard, 2010). A factor analytic approach is more appropriate if the focus of the study is to account for and establish the multiple dimensions of the construct (Sick, 2011). The CDCAT is one dimension of assessment competence, so it was consistent with the Rasch model. If studied with larger samples, the model of CDCAT could be standardized and the developmental model could be built to support the CDCAT training for teachers and pre-service teachers.

8. References Alonzo, D. A. (2016). Development and application of a teacher assessment for learning literacy tool [Doctoral dissertation, University of New South Wales].

American Federation of Teachers, National Council on Measurement in Education, & National Education Association. (1990). Standards for teacher competence in educational assessment of students. Washington: National Council on Measurement in Education.

Ananiadou, K., & Claro, M. (2009). 21st Century skills and competences for new millennium learners in OECD countries. https://doi.org/10.1787/218525261154 Assessment Reform Group. (2008). Changing assessment practices: Process, principles and standards. http://www.aaia.org.uk/content/uploads/2010/06/ARIAChanging-Assessment-Practice-Pamphlet-Final.pdf Association for Educational Assessment-Europe. (2012). European framework of standards for educational assessment 1.0. Rome: Edizioni Nuova Cultura.

Australia Goverment Department of Education, Employment and workplace Relations.

(2012).

TAEASS502B Design and develop assessment tools. https://training.gov.au/TrainingComponentFiles/TAE10/TAEASS502B_R1.pd f Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy.

New York: Academic Press.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy, and Practice, 5, 7–74.

Bol, L., Stephenson, P. L., O'connell, A. A., & Nunnery, J. A. (1998). Influence of experience, grade level, and subject area on teachers’ assessment practices. The Journal of Educational Research, 91(6), 323–330. http://doi.org/10.1080/00220679809597562.

Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how?.

CBE Life Sci Education, 15(4). http://doi.org/10.1187/cbe.16-04-0148 Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers.

Educational Measurement: Issues and Practices, 30(1), 3-12.

Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. ASCD Alexandria, Virginia USA. Caena, F. (2011). Literature review teachers’ core competences: requirements and development. https://www.researchgate.net/publication/344906332_Literature_review_Teac hers'_core_competences_requirements_and_development

http://ijlter.org/index.php/ijlter

101

DeLuca, C., & Klinger, D. A. (2010). Assessment literacy development: Identifying gaps in teacher candidates’ learning. Assessment in Education: Principles, Policy & Practice,17, 419–438. http://doi.org/10.1080/0969594X.2010.516643 DeLuca, C., LaPointe, D. M., & Luhanga, U. (2016). Approaches to classroom assessment inventory: A new instrument to support teacher assessment literacy. Educational Assessment, 21(4), 248-266. http://doi.org/10.1080/10627197.2016.1236677.

Department for Education-United Kingdom.

(2012).

Teachers’ standards. http://www.education.gov.uk/schools/teachingandlearning/reviewofstandar ds/a00205581/teachers-standards1-sep-2012.

Department of Education-Australia. (2012). Australian professional standards for teachers. http://www.teacherstandards.aitsl.edu.au/OrganisationStandards/Organisatio n DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage Publications.

Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113-132. http://doi.org/10.1080/15434303.2011.642041 Gardner, J. (2006). Assessment for learning: a compelling conceptualization. In J. Gardner (Ed.), Assessment and learning (pp197–204). Sage.

González, E. F. (2018). The impact of assessment training on English as a foreign language [Doctoral dissertation, University of Southampton].

Griffins, P. (2015). Assessment for teaching. Cambridge University Press.

Gulikers, J. T. M., Bastiaens, T. J., & Martens, R. L. (2005). The surplus value of an authentic learning environment.Computers in Human Behavior, 21(3), 509–521. http://doi.org/10.1016/j.chb.2004.10.028 Gutierrez, S. L. (2014). From national standards to classrooms: A case study of middle level teachers’ assessment knowledge and practice [Doctoral disseration, Western Michigan University].

Hammami, A. (2016). ESL teacher profiles of ICT integration in their classroom practices and assessment activities: A portrait viewed through the lens of some Quebec teachers’ social representations [Doctoral dissertation, Université de Sherbrooke].

Herppich, S., Praetorius, A. K., Foster, N., Glogger-frey, I., Karst, K., Leutner, D., Behrmann, L., Bohmer, M., Ufer, S., Klug, G., Hetmanek, A., Ohle, A., Bohmer, I., Karing, C., Kaiser, J., & Sudkamp, A. (2017). Teachers' assessment competence: Integrating knowledge-, process-, and product-oriented approaches into a competence-oriented conceptual model. Teaching and Teacher Education, 76, 181-193. http://doi.org/10.1016/j.tate.2017.12.001 Huba, M. E., & Freed, J. E. (2000). Learner-centered cssessment on college campuses: Shifting the focus from teaching to learning. Pearson.

Jarr, K. A. (2012). Education practitioners’ interpretation and use of assessment results [Doctoral dissertation, University of Iowa]. http://ir.uiowa.edu/etd/3317 Leighton, J. P. (2004). Avoiding misconception, misuse, and missed opportunities: The collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice, 23(4), 6–15. http://doi.org/10.1111/j.17453992.2004.tb00164.x Linh, N., & Tra, D. (2016). From surveying reality to proposing some solutions to develop competence of designing assessment tool in training and improving physics teacher. HNUE journal of science, 8, 213-225.

McGee, J. R. (2012). Developing and validating a new instrument to measure the self-efficacy of elementary mathematics teachers [Doctoral dissertation, University of North Carolina].

http://ijlter.org/index.php/ijlter

102

Mertler, C. A., & Campbell, C. (2005). Measuring teachers’ knowledge and application of classroom assessment concepts: Development of the Assessment Literacy Inventory [Paper presentation at meeting]. American Educational Research Association, Montreal, Canada.

Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: validity and reliability. Practical Assessment, Research, and Evaluation, 10(7), 71-81. http://doi.org/10.7275/q7rm-gg74 Nary, T. (2014). Development and validation of classroom assessment literacy scales: English as a foreign language (EFL) instructors in a Cambodian higher education setting [Doctoral dissertation, College of Education Victoria University Melbourne, Australia].

National Register on Vocational Education and Training in Australia. (2012). Design and develop assessment tools. TAE10 Training and Education Training Package version 2.0. Commonwealth of Australia.

North Central Regional Educational Laboratory. (2016). Indicator: Assessment. http://www.ncrel.org/engauge/framewk/pro/literacy/prolitin.htm Organisation for Economic Co-operation and Development (OECD). (2018). Teaching and learning international survey. http://www.oecd.org/education/school/TALIS2018-MS-Teacher-Questionnaire-ENG.pdf Okoli, C., & Pawlowski, S.D. (2004). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42, 15–29. http://doi.org/10.1016/j.im.2003.11.002 Perry, M. L. (2013). Teacher and principal assessment literacy [Doctoral Dissertation, University of Montana].

Popham, W. J. (2013). Classroom assessment: What teachers need to know (7th ed.). Boston: Pearson.

Randall, J., & Engelhard, Jr. G. (2010). Using confirmatory factor analysis and the Rasch model to assess measurement invariance in a high stakes reading assessmen., Applied Measurement in Education, 23(3), 286-306. http://doi.org/10.1080/08957347.2010.486289 Reddy, Y. M. (2011). Design and development of rubrics to improve assessment outcomes.

Quality Assurance in Education, 19(1), 84-104. http://doi.org/10.1108/09684881111107771 Shavelson, R. R. (2013). An approach to testing and modeling competences. In S. Blömeke, O. Zlatkin-Troitschanskaia, C. Kuhn & J. Fege (Eds.), Modeling and measuring competencies in higher education: Tasks and challenges (pp. 29-43).

Sick, J. (2011). Rasch measurement and factor analysis. SHIKEN: ALT Testing and Evaluation SIG Newsletter, 15(1), 15-17.

Stiggins, R. (1987). Design and development of performance asessment. Educational Measurement: Issues and Practice, 6(3), 33-42. http://doi.org/10.1111/j.17453992.1987.tb00507.x Stiggins, R. (2002). Assessment crisis: The absence of assessment for learning. Phi Delta Kappan, 83(10), 758–765.

Stiggins, R. J. (1999b). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practice, 18(1), 23-27. http://doi.org/10.1111/j.1745-3992.1999.tb00004.x Stiggins, R. J., Conklin,N. F. (1992). In teachers’ hands: Investigating the practices of classroom assessment. Albany: State University of New York Press.

Suskie, L. (2004), Assessing student learning: A common sense guide. Anker Publishing Company, Bolton, MA.

http://ijlter.org/index.php/ijlter

103

Tierney, R., & Simon, M. (2004). What’s still wrong with rubrics: focusing on the consistency ofperformance criteria across scale levels. Practical Assessment, Research and Evaluation, 9(2). http://doi.org/10.7275/jtvt-wg68 Tillema, H., Leenknecht, M., & Segers, M. (2011). Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning – A review of research studies.

Studies in Educational Evaluation, 37, 25–34. http://doi.org/10.1016/j.stueduc.2011.03.004 Vidacovich, C. (2015). Measuring teachers' knowledge and use of data and assessments: Creating a measure as a first step toward effective professional development [Doctoral dissertation, University of Denver].

Volante, L., & Fazio, X. (2007). Exploring teacher candidates’ assessment literacy: Implications for teacher education reform and professional development.

Canadian Journal of Education, 30, 751–772. http://doi.org/10.2307/20466661 Wiggins, G., & McTighe, J. (2005). Understanding by design. Association for Supervision and Curriculum Development.

Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization.

Teaching and Teacher Education, 58, 149-162. http://doi.org/10.1016/j.tate.2016.05.010 Zhang, Z., & Burry-stock, J. A. (1997). Assessment practices inventory: A multivariate analysis of teachers’ perceived assessment competency. https://www.semanticscholar.org/paper/Assessment-Practices-Inventory%3AA-Multivariate-of-Zhang-Burry Stock/2ade782d7f56a4143f74985e87ef786d90c9eeb3

http://ijlter.org/index.php/ijlter

104

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 104-120, February 2021 https://doi.org/10.26803/ijlter.20.2.6

Cultural Language Learning Approach (CLLA): An Approach for Learners’ Integrated Language Use Acculturation Hermayawati Universitas Mercu Buana Yogyakarta, Indonesia https://orcid.org/0000-0003-2860-6322

Abstract. The current development of English Language Teaching (ELT) is gradually shifting towards learning language skills in an integrated manner that do not use separated skills. It makes teachers look for the right approach to teach the desegregated four language skills.

Accordingly, this project investigated the Students of Candidate English Teachers (SCET) learning process using the Cultural Language Learning Approach (CLLA) for acculturating their Integrated Language Learning (ILL) in use. It was an applied study involving 100 SCET consisting of 60 females and 40 males aged between 20-22 years. They were intervened purposely in the learning process, to be well-acculturated to guard their native cultures. Quasi-semester meetings were carried out online. A closed-questionnaire and in-depth observation were the data gathering tools employed in this study. The results were analyzed descriptively using learners’ engagement factors inspired by Lee, Song and Hong’s model and Sheppard’s scoring rubrics. The results revealed that 68.31 % of learners were engaged during their ILL process. Besides, in-depth observation achieved the same average score, namely 77 in each quasisemester. Such scores were respectively categorized both in their learning engagement and achievement. Thus, CLLA can be used as an alternative approach to familiarize SCET with integrated language use in acculturating their ILL process. This study recommended further research on learners' ILL acculturation through different native cultures acculturation.

Keywords: approach; candidate English teachers; Integrated Language Learning; scoring rubrics; in-depth observation

1. Introduction The demand for a "liberated learning" implementation model, that is launched

by Indonesian's Ministry of Education and Culture, has been currently arousing the ELT (English Language Teaching) teachers to find novel alternative breakthroughs in their teaching manner. In line with such a demand, the development of technology has been affecting many aspects of human life

©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

105

mainly in English Language Teaching (ELT) which was previously taught in an isolated manner. To put it differently, in its beginning ELT received a number of approaches recommended by the linguists of the 20th-century concepts such as Grammar Translation Method (GTM), Audiolingual Method/Direct Method (AM/DM), Structural Approach, Oral Approach or Situational Language Teaching, Total Physical Response (TPR), Silent Way, Community Language Learning, Suggestopedia, Whole Language, Multiple Intelligence, Neurolinguistics Programming, the Lexical Approach, Competency-Based Language Teaching, Communicative Language Teaching (CLT), Natural Language Approach (NLA), and Task-based Language Teaching (TBLT), Cooperative Language Learning, Content-Based Instruction, and The post-methods era (Richards & Rodgers, 2001; Tomlinson, 2001, Brown, 2007). Such approaches and methods were well-known and consistently used to teach foreign languages worldwide, however not all of them are used in Indonesia, particularly the CLLA which can be used to cultivate learners ILL practically.

In facts, the current development of the era, which generates the emergence of various new approaches or methods, pronounced the aforementioned approaches to become fade-away and less-popular except for the Communicative Language Teaching (CLT), Co-operative Language Learning (CLL), Total Physical Response (TPR) and Task-Based Language Teaching (TBLT) since they are popular and used by English teachers in Indonesia. It is important to mention that by the early 21st century, various new approaches such as Contextual Teaching and Learning (CTL), Quantum Learning Method (QLM), Computer Assisted Language Learning (CALL), Content and Language Integrated Learning (CLIL), Multicultural Language Approach (MLA), Blended Learning Method (BLM) have been nowadays currently more popular and used to teach during the 21st century. In this case, Waters (2012) suggests that methodology at the ‘grass-roots’ level has remained relatively stable over the last 15 years and continually updated in the forthcoming years by the growing use of electronic technology in language teaching.

By the existence of those methods and approaches, teachers have a various choice in their teaching, even they may also mix approaches together. Since the present paper is not concerned with any of these methods and approaches, yet CTL and BLM are of no exception as they were related to the Cultural Language Learning Approach (CLLA) employed to investigate in this research paper.

2. Literature Review Contextual Teaching and Learning (CTL) was found by Johnson, Ida and Ibnu in 2007, in which teaching will be only effective when the content relates to the learners' real-life (Johnson, Ida & Ibnu, 2007). In this case, the learning content is delivered in the discourse connected to the learners' daily life. In ELT practices, CTL can be actualized by providing learners with various appropriate texts according to their age (English for kids, young learners, adults) and level (beginners, false-beginners, intermediate, advanced, and professional) (Bailey, 2005). The term BLM (Blended Learning) was primarily introduced by Stein and Graham in their Book written in 2014 and then published in 2017 through

http://ijlter.org/index.php/ijlter

106

European Journal and Education Studies. They defined that the BLM may be used to cultivate both onside and online assessments by which it enables the teacher to determine an assessment model that matches with the decided learning outcomes (Stein & Graham, 2014). They also added that assessments do not just measure, but also provide learners' feedback that should be directly delivered by the teacher soon after the assessment according to be useful for them to improve their further learning.

Another relatively current novel approach related to CTL and BLM is Cultural Language Learning Approach (CLLA). It is an approach to teach language by employing traditional cultural heritages in which materials are presented in the form of texts (Setiadi, 2017). It was introduced by Hermayawati at the Oman 14th International ELT Conference hosted by SQU (Sultan Qaboos University) on 8th to 9th May 2014 in Muscat, Oman. The approach is intended to introduce the traditional cultural heritages that are being the future generation's responsibility to preserve and to have a sense of belonging towards their ancestors' valuable artifact, and to avoid getting fade away. So far, CLLA has been developed in ELT through human puppets performances that can be picked from YouTube as the teaching materials delivered in an integrated way by developing pairs-skills of listening-speaking and reading-writing (Hermayawati, 2020).

Actually, teaching the four language skills in unity has been suggested for more than six decades now , and research and practice in ELT have identified the four skills (listening, speaking, reading, writing) as paramount which in turn affected text-books and curriculum designing (Brown, 2007). Nevertheless, early observations showed that most ELT teachers in Indonesia still focus on their teaching in an isolated way, surprisingly in the last decade, they have been taught in an integrated manner to Students of Candidate English Teacher (SCET).

However, ELT still has been delivered through two integrated pairs of listeningspeaking and reading-writing skills, due to the effect of curriculum development.

Integrated Language Learning (ILL) pedagogical paradigms started to emerge into practice as the effect of the communicative language learning paradigm (Gautam, 2019; Tajzad & Ostovar-Namagi, 2014), which was popular since two decades ago.

Regarding ILL, Brown (2007) suggested two previous language learning models, namely: content-based instruction (content-centred) and task-based instruction (CBI and TBI). CBI practically involves learners’ activities to read, discuss, solve problems, analyze data, and write opinions and reports. TBI implies ILL in its focus on language in the real world mainly for functional purposes. CBI focuses on subject matter content, while TBI focuses on the whole set of real-world tasks.

Besides, he also considered the use of Theme-based Instruction and Experiential Learning models. Those two models are proposed since most current ELT pedagogical materials (such as textbooks) and the authentic materials (such as teachers’ hand-outs) have been used as ELT materials for intermediate and advanced learners. They are delivered thematically or topically in which

http://ijlter.org/index.php/ijlter

107

learning activities need ILL for the sake of cultivating learners language learning experiences.

It is wiser the record that the English language enjoys the status of a foreign language and not as a second language (ESL) since it has its second language, namely, Bahasa Indonesia. It has various unique cultures, including the dominant ethnic, Javanese. Based on the ethnic classification, there are approximately 40% of Javanese where the total population concentrated on the island of Java (Yuniari, 2016; Leinbah, 2020). Thus, Javanese culture is used as the focus of CLLA study for the ILL approach. In this study, ILL involves desegregated four language skills focusing on both the learners' linguistic acquisition and the new knowledge embedded in the learned materials.

This study is addressed to acculturate the students of candidate English teachers (SCET) ILL acculturation using CLLA wherein the latter is as an approach with Javanese cultural heritages as the learning discourses. The previous studies have not found the ILL process involving the four language skills (listening-speakingreading-writing) in desegregating use. It might be because of the current existing ELT curricula that have not put the four ILL as a unity nor contextually. This study also purposefully introduced CLLA as a novel approach that has not been well-acknowledged by the relevant teachers thought it is considered as an appropriate approach to introduce and preserve traditional cultures to develop learners ILL through the learning process acculturation. Based on the above considerations, there were two under-observed variables, namely the acculturation of CLLA and SCET’s ILL in the use process.

2.1. Acculturating Learners’ Integrated Language Use Using CLLA By definition, acculturation is a process of acquiring certain cultural aspects of a dominant culture by the non-cultural members (Berry, 2003). Marsiglia et al.

(2011) reported that universal prevention interventions for the minor ethnic need to adapt to their linguistic acculturation trajectories. It can be triggered by intercultural contact, which may affect attitudes, norms, behaviours, knowledge, and identity changes (Berry, 2007). The above findings implied that acculturating a certain dominant culture can be well-acquired by the other nondominant cultures. Hence, this study cultivated Javanese cultural heritages as the dominant culture to intervene in the ILL of the non-dominant cultures that joined the ELT courses. As an illustration, Figure 1 provides the research paradigm of this study.

http://ijlter.org/index.php/ijlter

108

Early data access: Analysis of the needs of cultures' preservation using CLLA

Disseminated CLLA for ILL development

ELT development through cultural discourses, ILL use Intervention & Assessment

Learners’ wellacculturated in using CLLA for ILL development

Figure 1: The Research paradigm

Figure 1 shows the research paradigm that starts from accessing the research problems, namely the needs for cultural heritage preservation through ILL development using CLLA acculturation. By doing so, ILL which is (in this case) conducted in ELT intervention by employing CLLA can be developed and disseminated. Acculturation and dissemination processes are used as an attempt to preserve cultural heritage.

Several types of Javanese culture heritages may be used as learning materials.

They are, among others, in the forms of building, art performance, songs, foods, ceremonies, kinds of music, legends, folklore, clothing and heirlooms. Among those heritages, one may mention "wayang" (human and leather puppets), "batik" (a name of especially Javanese fashionable clothing), "reyog" (traditional art performance), "keris" or "kris" (a traditional Javanese heirloom) which have been recognized by UNESCO (United Nations of Educational, Scientific, and Cultural Organization) as the world's cultural heritages since 2003 (UNESCO, 2020). When investigating "wayang orang" or human puppets (HP) stories as the learning media, results revealed that HP can be benefited as learning materials for they contained hidden messages with moral values. More significantly, it can be used as a media for embedding learners' ILL. Nevertheless, HP had not been used as learning materials in ELT in Indonesia. Wayang stories are not only useful as an introduction to new knowledge, but also as a medium of entertainment and for moral education (Suyanto, 2013; Marsaid, 2016; Purwanto, 2018). Another study focused on Mahabharata and the Ramayana by looking at their benefits. All stories were originally performed in the Javanese language but completed with the Indonesian language subtitles in every story. The learners' tasks were respectively watching, understanding, rewriting and peers' assessing the group stories' synopses written in English in which the main intention was to acculturate learners' meta-cognitive dimension.

In contrast to the author's previous studies, this study examined the different types of traditional Javanese cultural heritage, primarily addressed to the Javanese traditional "building" and "traditional ceremony". The benefited building types included "Joglo House", "Kraton", "Benteng Vredeburg" or "Vredeburg Fort" (as the Dutch colonial heritage) and "Sonobudoyo Museum" (a museum of cultural heritage storage). Whereas, traditional ceremonies covered

http://ijlter.org/index.php/ijlter

109

"Mitoni", "Tedhak Siten" and "Ruwatan". "Mitoni", "Tedhak Siten", "Ruwatan" and "Labuhan" which are still popular today, especially by people with middle and upper economic status, just as an effort to preserve them in Indonesia.

"Mitoni" is a ceremony especially addressed to a woman who is pregnant for seven months. She should be ceremonially performed for the sake of her own and her baby’s safeness. "Tedhak Siten" is a ceremonial performance to train the first stepping down a baby on the ground when starting to walk. "Labuhan" is one of the Javanese traditional ceremonies commonly done by Kraton (Sultanate relatives) by carrying out food and clothes that are neatly made and arranged in a mountain-like shape as an expression of gratitude for all God gifts. The two types of heritages were employed both as the knowledge content (such as listening-speaking-reading-writing) and its linguistic aspects (such as sentence structure, grammar, vocabulary, pronunciation, spelling and comprehension).

3. Methodology The present research paper used applied study as a type of research to find the specific solution of individuals, organizations, and industries (Baimyrzaeva, 2018). In this type of research, the researcher regulates environmental variables to minimize excessive behaviour or to increase deficits and evaluates the results using a combination of single-subject experimental designs and social validation tools (Jessel, 2020). Newton-Rex (2019) proposed six principles to be employed in an applied study. First, it ensures a clear and agreed-on product aim. Second, it involves the product in setting research objectives. Third, it estimates the length of the study. Fourth, it lets researchers choose their methods. Fifth, it has regular reviews of progress and considers whether to continue the work at each review. Sixth, it rewards good choices and speed to conclusions.

Accordingly, this study performed to gain the specific solution of the current issues concerning the cultural heritages that have been worried to be fade away extinct and need preservation, besides they can be very useful materials to develop Learners' ILL. It purposely investigated two observable variables, namely the use of CLLA (which functioned as the learning approach) and the ILL intervention process by involving two classes with different courses.

3.1. The Participants Because the current paper aimed to address a critical point, it is crucial to care about the participants by defining important characteristics for data validity and reliability. Therefore, Students of Candidate English Teachers (SCETs) are the research informants in this study. They are selected because they enjoy a certain age and are adult learners with an advanced level of study. They are generally autonomous, goal-oriented, practical, competent and mastery, learning by experience, knowledgeable, purposeful, have emotional barriers, result-oriented, outside responsibilities, potential physical limitations, the big picture, selfresponsible, and need for community (Ann & Strong, 2002; Malamed, 2009; Makhlouf, 2019). Considering such specified characteristics, they should be treated using andragogy, that is, a study of adult learning proposed by Knowles in 1950.

http://ijlter.org/index.php/ijlter

110

Andragogy is principally used to teach adult learners who need to be treated as follows: (1) being involved in their instruction; (2) doing trial and error in their learning activities; (3) learning should be related to their personal real-life; (4) learning should be directed to problem-solving rather than the content-oriented (Knowles, 1984; Culatta, 2020). In a nutshell, it is practically instructed to more focus on the process rather than on the content being learned. Furthermore, Malcolm (1984) proposed 7 (seven) points of adult learning outcomes by enabling them to (1) acquire their understanding maturity; (2) respect toward others; (3) develop a life dynamic attitude; (4) learn to react to the causes, not the symptoms, of behaviour; (5) acquire the skills of their personalities potentials; (6) understand the essential values in human capital experience; and (7) understand their society to be skilful in directing social change. All of the above principles were considered as the basis of the participants' language learning process in integrated use, both receptively (in listening and reading activities) and productively (in speaking and writing activities).

Based on the above highlighted, 100 students of candidate English teachers (SCET) were research participants aged between 20 to 22 consisting of 60 females and 40 males who were purposely involved in the research project (Purposive sampling). They were intervened by purpose in the learning process, namely to be well-acculturated to guard and/or preserve their native cultures. 50 of them joined writing in a professional context course, and the rest joined the Materials and Media Designs (MMD) course. They were all categorized into adult learners and treated as pre-advanced English capacities. The two different classes were taken as research participants with the same intervention since they have similar needs in using practical ILL. Besides, they were all approached as the future teachers and the young generations all at once who have the responsibility in developing both ELT quality and cultural heritage preservation.

3.2. Research Instruments In this study, CLLA which was the product of the author's previous research was employed as an approach within the SCET's ILL process for a semester. It was considered as the most appropriate approach to fulfill the needs of cultural heritage preservation efforts through the SCET's ILL. It was employed as an approach to acculturate the SCET's ILL process mixed with the BLM. A closedquestionnaire was distributed to gather learners' needs analyses as qualitative data. The in-depth observation was carried out during the learning process to gather quantitative data. The latter were collected by analyzing the gain score of the mid-term and the final tests achievements of the learners. Both data types were analyzed descriptively through (respectively) theoretical tree-angulation (data validation by comparing the data to the relevant theories) and the SCET's ILL test results assessment using Sheppard's (2015) scoring rubric.

Table 1 provides the example of using such scoring rubric that includes 7 assessment aspects, namely "overall impact", "clarity", "organization", "mechanic", "grammar", "style", and "presentation". Each aspect can be scored into 5 levels or categories of scores, namely "not yet" (1-2), "emerging" (3-4), "developing" (5-6), "effective" (7-8) and "strong" (9-10) with numerical scores

http://ijlter.org/index.php/ijlter

111

spread from 1 as the lowest score until 10 as the highest score. The test taker will have completed a 70 score with the strong level category whenever s/he achieves a 10 point for the seven assessment aspects. Vice versa, s/he will get a 7 point when getting the lowest score at all of the 7 aspects being assessed. In other words, the total score spreads from 7 to 70 or from "not yet' to "strong" level category. A test-taker will achieve a score of 7 with "not yet" category level, if s/he gets 1 for all assessment aspects and will get "strong" when s/he gets score 70 for all assessment aspects. Table 1 illustrates a sample of a test taker who gets a total score of 65.7. It includes "effective" level for having: 6 for each in his "clarity", "organization" and "mechanic" aspects, and score 7 for each "overall impact", "grammar", "style" and "presentation" aspects.

Table 1: Sample of a test-taker achievement inspired by Sheppard’s scoring rubric Assessment Aspects

Not Yet 1-2

Emerging

Developing

3-4

5-6

Overall Impact

Effective 7-8 7

Strong 9-10

Total

7

Clarity

6

6

Organization

6

6

Mechanic

6

6

Grammar

7

7

Style

7

7

Presentation

7

7

28

46

Ʃ Comment

18 46/70 x 100 = 65.7

√

In connection with this study, the tests were assigned in groups of 5, so that there were 20 groups involved in this study. Results of all the test-takers with the use of ILL were computed using the Mean Difference (Md) formula to find the gain score between the first (mid-term) and the second (final) tests. The following was the formula used to compute the Md that shows the numerical or coefficient number of the gain score. The higher the coefficient number is, the more significant the students' well-acculturated (in their ILL process using CLLA) will be.

Md = Ʃ X1 /n1 - Ʃ X2 /n2 or Md = 𝑋1 - 𝑋2 Notes: Md is Mean Difference; Ʃ X1 is the total scores of Mid-term test takers; Ʃ X2 is the total scores of Final-test takers;

http://ijlter.org/index.php/ijlter

112

n1 is the number of Mid-term tests; n2 is the number of Final-test takers; 𝑋1 = The average number of Mid-term test; 𝑋2 = The average number of Final-test takers.

4. Findings As stated above, this study aimed at acculturating SCET's learning process by benefiting CLLA as an alternative approach to preserve the existence of Javanese cultural heritages as the most dominant ethnic in Indonesia. This section presents the results of the two prior under-observed variables, namely the acculturation of CLLA materials types and SCET's ILL in the use process. The followings are the details of the findings.

4.1. Students’ Engagement in the Online Learning Environment Using CLLA This study employed CLLA as an approach for the SCET’s learning intervention with traditional culture buildings and ceremonies as the knowledge content.

Seven types of Javanese cultural heritages were employed as the learners’ learning materials namely “Joglo House”, “Kraton”, “Benteng Vredeburg” or “Vredeburg Fort” (as the Dutch colonial heritage) and “Sonobudoyo Museum” (a museum of cultural heritage storage) as the building types. The included "traditional ceremonies" were “Mitoni”, “Tedhak Siten”, “Ruwatan” and “Labuhan”. Those seven material topics were delivered using the FUN procedure. FUN stands for “Finding (materials)”, “Understanding (the target content)”, and “Naturally- discussion”. The FUN stages assigned the learners to cope with all activities that were carried out in groups of 5 for each class. There were 20 (groups) in two classes.

The “Finding (materials)” stage included the learners’ involvement in finding their materials. Such an activity was purposely designed to activate the learners’ creativity in selecting qualified discourse both in written and oral language. The “Understanding (the target content)” stage, included the learners’ comprehensions towards the materials they have found through the discussion forum among their groups. The intention is to acculturate them to work together along with preventing them from being selfish. The “Natural-discussion” stage called learners to problems solving activities regarding the difficulties found during their discussion process. This activity aimed at acculturating them to solve their difficulties either individually or in groups. The final work of each group was in the form of written English text which was then, exchanged to other group’s work to assess both its content and the language use. The groups’ written language use was scored by using the scoring rubric, as previously highlighted (See Table 1).

http://ijlter.org/index.php/ijlter

113

Table 2: The FUN stages of CLLA implementation

Learning Stages Using CLLA

Building Types

Ceremony Types

FUN

Joglo House

Mitoni

Finding (materials)

Kraton

Tedhak Siten

Understanding (the target content)

Benteng Vredeburg

Ruwatan

Sonobudoyo Museum

Labuhan

Naturally-discussion

Regarding the in-depth observation that was carried out during the process of learning, findings revealed that CLLA was appropriately used to enable learners to be well-acquired both in understanding the given materials and in using the target desegregated language use. Such an observation was attentively conducted for seven virtual meetings during the learning process intervention.

All of the learning processes were tested and scores were given for each group .

The observation activities focused on the participants' ILL practice progress using CLLA. The ILL was delivered together within the use of such an approach.

In this case, ILL functioned as an aggregate to comprehend the content of the materials being learned. The materials main topics were as follows: "building" and "traditional ceremony" with “Joglo House”, “Kraton”, “Benteng Vredeburg” (Vredeburg Fort), “Sonobudoyo Museum”, “Mitoni”, “Tedhak Siten”, “Ruwatan” and “Labuhan” as the sub-topics. Table 2 illustrates the FUN stages of CLLA implementation.

All of the eight subtopics shown in Table 2 were assigned in 20 groups of 5 members who were not all Javanese. There were around 40 participants who were native Javanese, and the rest came from various other ethnics with different cultures spread in many other Indonesian islands. They come to Yogyakarta to study, as it is an education city centre located on Java island, the most populated city in Indonesia. Even though they come from non-dominant ethnics, they were enthusiastically involved in the learning process. Their learning engagement was observed from their activities, mainly as they performed groups discussion through virtual courses using G-meet-links.

Regarding learners' engagement measurement, this study was inspired by Dixon's (2015) and Lee et al. (2019) who asserted that learning engagement emerges by considering six factors, namely: psychological motivation, peer collaboration, cognitive problem solving, interaction with instructors, community support and learning management. Lee et al. (2019) proposed a 25item scale to improve student engagement. Such six factors, including the scale of the items, were considered for measuring the learners' learning engagement.

They include "psychological motivation (consists of 6 items)", "peer collaboration (consists of 5 items)", "cognitive problem solving (consists of 5 items)",

http://ijlter.org/index.php/ijlter

114

"interactions with instructors (consists of 2 items)", "community support (consists of 3 items)" and "learning management (consists of 4 items)". The followings are the details of six factors with each item categories.

Firstly, “psychological motivation” includes six items that are: interest enhancement in their virtual classes learning, motivation to study when taking virtual classes, feeling of usefulness to take virtual classes, interest to take the online course, looking forward to the next virtual meeting, after taking an online lesson ,and Learners’ satisfaction with the online class they are taking. Secondly, “peer collaboration” includes five items: study the lesson contents with other students, efforts to solve difficult problems with other learners when encountering them, working with other learners on online projects or assignments, efforts to ask other learners for help when they cannot understand the concept taught in the online classes, responses to answer the other students’ questions. Third, “cognitive problem solving” consists of five items, namely: deriving new interpretations and ideas from the knowledge they have learned in the online classes, ability to analyze thoughts, experiences, and theories about the knowledge they have learned in the online course, ability to judge the value of the information related to the knowledge learned in the online classes, the tendency to apply the knowledge they have learned in the online course to real problems or new situations, efforts to approach the subject of the online course with other new perspectives.

Fourth, "interactions with instructors" consists of two items;: communication with the instructor privately for extra help and the frequency to ask the instructor concerning the course content. Fifth, "community support" consists of three items, including: feeling a connection with the other learners in the online classes, feeling a sense of belonging to the online class community and frequent interaction with other learners in the online course. Sixth, "learning management" consists of four items: study-related learning contents by themselves after the online lesson, tendency to remove all distracting environmental factors when taking the online course, management of their learning using the online system, and their necessities to plan a learning schedule when joining an online class.

All six factors, which include 25 engagement items, were delivered through the closed-questionnaire to collect data on the learners’ engagement in the virtual learning environment using CLLA. A hundred learners were assigned to fill out the online questionnaire after completing a semester of the virtual meeting courses, conducted within 14 (meetings) through G-meet links. Data analysis yielded that 68.31 % (of a hundred participants) were engaged with online learning using CLLA, while the rests (31.69 %) were not. This fact was primarily as the effect of their psychological motivation which shows that 52.22 % were psychologically not engaged during their online course. This was not because of the use of CLLA, rather it was the effect of the online system they commonly underwent, by the time they joined the G-meet links to participate in the virtual courses. Most learners also complained about the affordability and interference

http://ijlter.org/index.php/ijlter

115

of the internet signal during lectures, especially for students living in remote areas.

Such conditions were known when the lecture was taking place, as they had tried to join the link but failed and finally took part in the course via the chat application available on G-meet. On the positive side, learners who got problems with the internet network were still trying to attend lectures, which were indicated by the relatively high percentage of the three other observable aspects, namely "peers collaboration", "interaction with instructors" and "learning management" that reached above 81.84%, 72.35 %, and 79.22 %. Table 3 provides the learners’ engagement in the virtual learning environment using CLLA. The other aspects of engagement, such as the “cognitive problem solving” and the “community support” were respectively achieved with 69.24 % and 59.47 %. It means that there was 30.76 % and 40.53 % of learners got problems in such two aspects. The issues concerning those two aspects are related to their difficulties accessing lectures through the online system (See Table 3).

Table 3. Learners’ engagement in the virtual learning environment using CLLA (An analysis of results inspired by Lee et al., 2019 model) Observed Items Number

Factors

Learners ’

Yes (%)

No ( %)

Engagement in the e-Learning Environment

Psychological motivation

6

47.78

52.22

Peer collaboration

5

81.84

18.16

Cognitive problem solving

5

69.24

30.76

Interactions with instructors

2

72.35

27.65

Community support

3

59.47

40.53

Learning management

4

79.22

20.78

Total

25

409.86

190.14

68.31

31.69

Average (Among participants)

100

4.2. ILL Intervention Process In this study, ILL was employed to comprehend the whole target materials topics, namely building and traditional ceremony with “Joglo House”, “Kraton (Palace)”, “Benteng Vredeburg” (Vredeburg Fort), “Sonobudoyo Museum”, “Mitoni (Seven Months of Pregnancy)”, “Tedhak Siten (the first time of a Baby’s stepping on the Ground)”, “Ruwatan (Name of a traditional ceremony to reject evil crime)” and “Labuhan (An offering ceremony to the ruler of the oceans)” as sub-topics. The FUN (finding, understanding, and naturally-discussing) stages were procedurally implemented to achieve learners’ target language acquisition through the use of ILL acculturation. The seven sub-topics were intervened

http://ijlter.org/index.php/ijlter

116

through FUN in groups of five and scored by the peers-group (peers assessment).

The results were accumulated as the group’s portfolio scores.

Table 4. Learners’ ILL average achievement Topic Category

Sub-topic Category

Group’s Participant

Portfolio Achievement

Buildings

𝑿2

1

78

75

2

78

78

3

78

80

➢ Joglo House

4

78

75

➢ Kraton

5

76

84

➢ Benteng Vredeburg

6

76

70

➢ Sonobudoyo Museum

7

76

78

8

76

85

9

79

77

10

79

80

11

79

79

12

79

70

13

76

67

14

76

74

15

76

80

16

76

76

17

78

87

18

73

68

19

75

77

20

75

80

77

77

Portfolio 1

Portfolio 2 ➢ Mitoni ➢ Tedhak Siten ➢ Ruwatan ➢ Labuhan Ceremonies

ILL Using CLLA (Cultural Language Learning Approach) Materials

𝑿1

𝑋 for each term

Referring to Sheppard's (2015) scoring rubrics, both scores include the "effective" category. By such findings, it is safe to mention that the two investigated variables were accomplished in this study. Firstly, it reflected the learners' wellacknowledged with CLLA as an approach to preserve traditional cultures through ILL acculturation. Secondly, it indicated their well-acculturated in using the target language in an integrated manner.

5. Discussion This section elaborated the interpretation and implication concerning the two findings above which implied that the use of an approach in the ILL was intertwined and cannot be separated from one another. Previous studies revealed that an approach is possible to highlight the attempts for learners’

http://ijlter.org/index.php/ijlter

117

learning enhancement quality (Beckwith, 1991; Trigwell et al., 1999; Rosário et al., 2013).

The current study revealed that there was not any significant difference among the four of six factors: “peer collaboration”, “cognitive problem solving”, “interaction with instructors”, and “learning management” because of a set of parameters used for measuring the learners’ engagement in the virtual learning environment. Concerning the use of CLLA, it was matched to carry out the ILL process. Even though it was found that the two factors of “psychological motivation” and “community support” have low percentages (that imply the low of the learners’ psychological and community supports), it did not mean that it was due to the effect of using CLLA. Such a condition was experienced by most learners because of the lack of internet signals availability as the main means of online learning.

Another finding is related to the result of the ILL process. Previous studies asserted that ILL needs the learners’ good understanding of the sources being learned, as using the language for communication must be automatically interwoven somewhat like a tapestry (Oxford, 2001; Orellana, 2001; Pardede, 2017). Oxford (ibid) proposes that to integrate the language skills instruction, teachers should consider the following stages: (1) Learning more about the various ways to integrate language skills in the classroom (e.g., content-based, task-based, or a combination); (2) Reflecting on their current approach and evaluate the extent to which the skills are integrated; (3) Choosing instructional materials, textbooks and technologies that promote the integration of listening, reading, speaking and writing as well as the associated skills of syntax, vocabulary and so on; (4) Integrating the other language skills through appropriate tasks; (5) Teaching language learning strategies and emphasizing that a given strategy can often enhance performance in multiple skills. This study has coincidentally considered such five stages as written above to carry out the learning process.

The current finding showed that intervened learners were well-acculturated in using the target language in an integrated manner. It is proved by the results of data analyses towards their learning achievement collected through a portfolio system consisting of two quasi-semesters that includes four sub-topics for each half-semester. It implied that CLLA was useful to acculturate learners’ language learning in an interwoven manner, like a tapestry. Learners’ acculturation in using integrated language using CLLA was also possible to cultivate their learning-centred strategy. Cultural representation contained in the learned discourses can also change the viewpoints of the language learners (Lustyantie & Dewi, 2020). In this case, Loi et al. (2021) claimed that classroom instruction deployed more learner-centred activities and formative assessment to push the use of learning strategies.

6. Conclusion This study was intended to acculturate the students of candidate teachers’ ILL by using CLLA that was warranted appropriate use for such cases. It can be seen

http://ijlter.org/index.php/ijlter

118

from the results of both the qualitative and quantitative data analyses that the understudied learners need to be well-acculturated in their ILL using CLLA. It is, therefore, recommended to use it as an alternative approach to ILL teaching practice. Besides, it can be also beneficial to instil noble moral values in students.

Add to this, dindings showed that the average portfolio of assignments resulted in a 77 score, which includes an "effective" score (See Page 13).

Therefore, this applied research concluded that CLLA can be an alternative approach to acculturate learners’ ILL acquisition, mainly to preserve traditional cultures. It is recommended for teachers, practitioners and language program designers to utilize CLLA as an approach in their language teaching programs.

Such an approach should be used in the ILL setting for it was proved as an appropriate approach to acculturate students as the next young generation who have the responsibility to guard and preserve their native cultures. This must be realized and acculturated through ILL for the sake of preserving such valuable cultural heritages extinction.

7. Limitations and Further Studies The limitation of the current study was that the ILL process which was carried out during the Covid-19 pandemic, since it affected most learners who have difficulty in adapting to the internet signal coverage. Such a condition influenced both the learners' psychological motivation and their community support during the learning process. Further studies should include an investigation on how to enhance the above two aspects by acculturating the use of CLLA with other topics for the sake of developing learners' ILL acquisition.

Acknowledgements We acknowledge special thanks to all of the parties who supported this article publication.

8. References Ann, F. V. S., & Strong, G. (2002). Adult language learners: An overview. Harbour Press. https://www.tesol.org/docs/books/bk_CP_AdultLL_615 Bailey, K. M. (2005). Practical english language teaching speaking. McGraw-Hill.

Baimyrzaeva, M. (2018). Beginners’ guide for applied research process: What is it, and why and how to do it? University of Central Asia: Graduate School of Development, 4, 1–43.

Beckwith, J. B. (1991). Approaches to learning, their context and relationship to assessment performance.

Higher Education, 22(1), 17–30. https://www.jstor.org/stable/3447151 Berry, J. W. (2003). Conceptual approaches to acculturation. American Psychological Association. https://doi.org/10.1037/10472-004 Berry, J. W. (2007). Acculturation strategies and adaptation. Queen’s University. https://www.researchgate.net/publication/232564006_Acculturation_strategies _and_adaptation Brown, H. D. (2007). Teaching by principles: An interactive approach to language pedagogy.

Pearson Education, Inc. Culatta, R. (2020).

Andragogy (Malcolm Knowles). https://www.instructionaldesign.org/theories/andragogy/

http://ijlter.org/index.php/ijlter

119

Dixson, M. D. (2015). Measuring student engagement in the online course: The online student engagement scale (OSE).

Online Learning, 19(4), 51–65. https://doi.org/10.24059/olj.v19i4.561 Gautam, P. (2019). Integrated and segregated teaching of language skills: An exploration.

Journal of NELTA Gandaki (JoNG), 1(1), 100–107.

Hermayawati. (2020). Cultural language learning approach (CLLA): An approach for language pedagogy. Universitas Mercu Buana Yogyakarta.

Jessel, J. (2020).

Applied research (1st ed.). https://doi.org/10.1002/ 9781119171492.wecad058 Johnson, E. B., Ida, S., & Ibnu, S. (2007). Contextual teaching and learning (Edisi Terjemahan). MLC. Knowles, M. (1984a). Andragogy in action. Jossey-Bass.

Knowles, M. (1984b). Malcolm knowles, informal adult education, self-direction and andragogy. https://infed.org/mobi/malcolm-knowles-informal-adult-education-selfdirection-and-andragogy/ Knowles, M. (1984c). The adult learner: A neglected species (3rd ed.). Gulf Publishing.

Lee, J., Song, H., & Hong, A. (2019). Exploring factors, and indicators for measuring students’ sustainable engagement in e-learning. Department of Education, College of Education, Chung-Ang University.

Loi, N. Van, Thi, C., Hang, T., & Tho, C. (2021). English proficiency gain and mediating factors in training : A self-evaluation of Vietnamese pre-service EFL teachers.

International Journal of Learning, Teaching and Educational Research, 20(1), 22–47. https://doi.org/10.26803/ijlter.20.1.2 Lustyantie, N., & Dewi, E. R. (2020). Representation of French culture as a foreign language through textbooks. International Journal of Learning, Teaching and Educational Research, 19(3), 404–421. https://doi.org/10.26803/ijlter.19.3.22 Makhlouf, J. (2019). Pedagogy vs. Andragogy: Where many get it wrong In their learning strategy. https://elmlearning.com/pedagogy-vs-andragogy/ Malamed, C. (2009).

Characteristics of adult learners. https://theelearningcoach.com/learning/characteristics-of-adult-learners/ Marsaid. (2016). Islam dan kebudayaan: Wayang sebagai media pendidikan islam di nusantara.

Kontemplasi, 4, (1). https://media.neliti.com/media/publications/67434-ID-islam-dan-kebudayaan -wayang-sebagai-medi.pdf Marsiglia, F. F., Yabiku, S. T., Kulis, S., Nieri, T., Parsai, M., & Becerra, D. (2011). The influence of linguistic acculturation and gender on the initiation of substance use among Mexican heritage preadolescents in the borderlands. The Journal of Early Adolescence, 31(2), 271–299. https://doi.org/10.1177/0272431610363157 Newton-Rex, E. (2019). Six principles of applied research. Https://Medium.Com/. https://medium.com/on-coding/six-principles-of-applied-research875de3b44ba0 Orellana, E. por. (n.d.). The importance of integrating skills in the teaching of english as a foreign language. https://www.monografias.com/trabajos17/integratedskills/integrated-skills.shtml Ostovar-Namaghi, S. A., & Tajzad, M. (2014). Exploring EFL learners’ perceptions of integrated skills approach: A grounded theory. English Language Teaching, 7(11), 92–98. https://doi.org/10.5539/elt.v7n11p92 Oxford, R. (2000). Integrating the language Skills. The Journal TESOL France, 5–12. https://www.tesol-france.org/ Pardede, P. (2017). Integrated skills approach in EFL classrooms: A literature review. UKI

http://ijlter.org/index.php/ijlter

120

Press. https://eeduki.com/ Purwanto, S. (2018). Pendidikan nilai dalam pagelaran wayang kulit. Ta Allum Jurnal Pendidikan Islam, 6(1), 1–30. https://doi.org/10.21274/taalum.2018.6.1.

Richards, J. C., & Rodgers, T. S. (2001). Approaches and methods in language teaching.

Cambridge University Press.

Rosário, P., Núñez, J. C., Ferrando, P. J., & Al., E. (2013). The relationship between approaches to teaching and approaches to studying: a two-level structural equation model for biology achievement in high school. Metacognition Learning, 8, 47–77. https://doi.org/doi.org/10.1007/s11409-013-9095-6 Setiadi, H. (2017). Designing a supplementary reading using cultural Language learning approach (CLLA). International Journal for Innovation Education and Research (IJIER), 8(5), 111–128. https://doi.org/10.31686/ijier.vol5.iss8.794 Sheppard, R. (2015).

Rubrics for integrated assessment. https://blogs.newschool.edu/unchartedtesol Stein, J., & Graham, C. R. (2014a). Considerations for blended course designs. In M. Vai, (Ed), Essentials for blended learning: A standards-based guide. Routledge.

Stein, J., & Graham, C. R. (2014b). What is Blended Learning? In M. Vai (Ed.), Essentials for blended learning: A standards-based guide. Routledge.

Trigwell, K., Waterhouse, F., & Prosser, M. (1999). Relations between teachers’ approaches to teaching and students’ approaches to learning. Higher Education, 37(1), 57–70. https://doi.org/10.1023/A:1003548313194 UKEssays. (2018). History of english language teaching. https://www.ukessays.com/ Unesco. (2020). Browse the lists of intangible cultural heritage and the register of good safeguarding practices. Ich.Unesco.Org. https://ich.unesco.org/en/lists Waters, A. (2012). Trends and issues in ELT methods and methodology. ELT Journal, 66(4), 440–449. https://doi.org/10.1093/elt/ccs038

http://ijlter.org/index.php/ijlter

121

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 121-136, February 2021 https://doi.org/10.26803/ijlter.20.2.7

The Impact of Using VAKT Strategy on Oral Reading and Reading Comprehension Skills of Elementary Students with Dyslexia Ayed H. Ziadat Department of Special Education, Princess Rahman College, Al-Balqa Applied University, Al-Salt, Jordan http://orcid.org/0000-0002-7920-2419

Abstract. Dyslexia is a learning disability associated with a deficiency of spell, read, write, and recognize words. The key purpose of the study was to examine the desirable enhancing of oral reading and reading comprehension levels by testing the advantage of Visual, Auditory, Kinesthetic, and Tactile (VAKT) strategy on oral reading and reading comprehension among students with dyslexia registering at the governate source classroom. The study adopted a Quasi-experimental approach with the design of control and experimental groups and two measurement phases (pre and post-tests). The study assessed oral reading and reading comprehension levels by the diagnostic scale of Arabic language Basic skills. Thirty-nine subjects were drawn using the purposive sampling technique (control N=19 and experimental N=20), and the intervention VAKT technique was used for third-grade students with dyslexia for one semester consecutively followed by the postassessment. The results revealed that the control group showed enhancement in comprehension reading and oral reading level, but less significant than the experimental group. Thus, VAKT is a sufficient approach to advance the comprehension reading and oral reading levels of students with dyslexia. The study findings contribute to motivating VAKT strategy using for students representing dyslexia for the sake of reading learning. Further, the study contributes to implementing VAKT in Arabic language learning settings. The researcher recommends considering learning style, motivation, and settings in future studies.

Keywords: Dyslexia; oral reading; reading comprehension; source classrooms; VAKT

1. Introduction In the last 50 years, various research and literature accurately identified and categorized learning disabilities based on a plausible broad of cognitive, neurobiological, and environmental aetiologias of these deficient. The most authorized results of global efforts are shown in the Diagnostic and Statistical ©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

122

Manual of Mental Disorders (DSM) editions (American Psychological Association [APA], 2020), and the International Statistical Classification of Diseases and Related Health Problems (ICD) revisions (World Health Organization [WHO], 2019). Through these manual learning disabilities are recognized as a heterogeneous group of learning competencies and need to receive special supports and treatments (Grigorenko et al., 2020). Across countries, the definition and classification of learning disabilities have slight differences. In contrast, some countries have not yet formally defined and recognized learning disabilities like African countries and Indian governments (Hayes et al., 2018). In Jordan, there is no specific category or definition for learning disabled in the Law on the Rights of Persons with Disabilities No. 20 (2017). The only legislated definition is for a person with disabilities, which defined as (Law No. 20, 2017): “Person who has long-term1 physical, sensory, intellectual, mental, psychological, or neurological impairment, which, as a result of interaction with other physical and behavioral barriers, may hinder performance by such person of one of the major life activities2 or hinder the exercise by such person of any right or basic freedom independently.”

From the definition, learning is considered one of the "major life activities". Thus, any genre of impairments hindering a person from learning is covered by this act, such as learning disabilities. Accordingly, learning disabilities are generally included in the definitions without classification. Locally, students are diagnosed academically with learning disabilities if they do not exhibit the corresponding approved age -or -grade-level standards in, at least, one of the following skills: oral reading, oral expression, reading comprehension, listening comprehension, reading fluency, essential reading skills, written expression, mathematics and calculation, problem-solving in math. In Jordan, the population census reported that around 11% of the population diagnoses with functional deficiency, 22.3% of them have never enrolled in any educational institutions (DOS, 2015). In 2018, the disability prevalence raised to 13% of the population. Unfortunately, the national rate approximates 15% of the global disability prevalence rate estimated by the World Health Organization. Moreover, 35.5% of the disabled population is illiterate. Only 21859 (1.6%) of disabled students receive special education services in the academic year of 2018/2019 (Thompson, 2018). Although that there is no accurate static about the distribution percentage of disabled students according to the disability category needed to integrate the national figure of disability (AlZboon & Hatmal, 2015), the Jordanian ministry of education and the Higher Council for the Rights of Persons with Disabilities (HCD) has worked and keened with a solid basis for effective and reliable mechanisms and strategies to increase the enrolling percentage of students with disabilities in public schools from 5% in 2017 to 10% in 2022 (HCD, 2018). However, there is constant screening, assessment, identification, and treatment of learning disabilities at all levels of interest, including global and local levels. The study contributes to examine the 1 not expected to 2

disappear in at least (24) months

Major life activities mentioned in Clause (a) of this Article include the following: Eating, drinking, administering, self-care, reading, and writing. Movement and mobility. Interaction and concentration, expression, and verbal, visual and written communication. Learning, rehabilitation, and training. Work.

http://ijlter.org/index.php/ijlter

123

VAKT strategy’s effectiveness as a teaching strategy for this population, which aligns with global and domestic efforts.

Moreover, the current study concerns dyslexia and examines the effectiveness of a teaching strategy based on multisensory discipline within the context of developing oral reading and reading comprehension abilities.

2. Review of Literature Dyslexia is a term derived from Greek words “dys” and “lexis” which meaning poor or inadequate, words or language, respectively. Accordingly, this term is inferred difficulties or problems in expression, reception, oral, or written functions (Jeyasekaran, 2015). These difficulties translated in reading, spelling, speaking, listening, and writing skills obviously. The International Dyslexia Association defines dyslexia as a neurobiological learning disability that induces a deficiency in the fluency and accuracy level of word recognition, pronunciation, spelling, and decoding competencies (Subramaniam & Nasir, 2020). Even the accurate definition of dyslexia, but professional particularly, teachers still misunderstand dyslexia. For instance, if teachers are asked to define dyslexia, they more likely to answer, “Dyslexia is an impairment that students see words backward." The truth is that students with dyslexia see words normally as their normally developing peers, but their brains process the information differently and make the response as they see the term backward (Norton, Beach & Gabrieli, 2015). Abuzaid and Kayed (2020) defined dyslexia as a condition hinders students from reading at the level of their normally developing peers at the same age, learning language and words within normal teaching techniques, and reading sentences correctly. Thus, dyslexia is known as a reading disorder. It is worth mentioning that people who lose their reading competencies after previously acquiring known as alexia. Dyslexia including various types of reading faults such as dropping, moving, insertion, replacement, reversal, considering, and estimation (Peter, Albert & Gray, 2020). However, treatment of dyslexia mainly depends on adjusting teaching methods or implementing special teaching methods to encounter students’ special needs and conditions (AL-Qatawneh, 2020). The teachers’ job is to tailor proper pedagogical practices with students to advance their awkward abilities in reading, speaking, spelling, and writing.

Rather than changes their abilities or functioning because students with dyslexia are not disabled, but they have a different way of functioning and processing information (Hanif et al., 2019; Mills, 2018).

VAKT strategy It is academically approved that individuals learn better using more than one sense, especially for improving language learning (Korkmaz & Karatepe, 2018; Mathias et al., 2019; Rao, 2018). The predictive coding theory established that the brain smoothly encodes and recognizes multi-sensory information. Multi-sensory is a teaching approach in which students activate all human faculties, namely, hearing, smelling, touching, seeing, feeling, and tasting in learning settings (Suryaratri, Prayitno & Wuryani 2019). These strategies are known to be natural because it simulates the innate learning process of infant and toddler. The infant in the early stages uses his taste and toddler depends on the tactile sense (touch

http://ijlter.org/index.php/ijlter

124

or grab) to recognize the surroundings (Sarudin, Hashim & Yunus, 2019). Thus, teachers do not need to exert an extra-ordinary effort to implement multi-sensory strategies.

VAKT has been developed by Fernald and Orton-Gillingham, which is a multisensory teaching method that activates visual, auditory, kinesthetic, and tactile sensing in order to encourage learning and retrieving (Jeyasekaran, 2015).

Fernald’s method activates four main sensory inputs, namely, visual, auditory, kinesthetic, and tactile, and known by VAKT (Chither, 2020). Meanwhile, the Orton-Gillingham method activates only three sensory inputs, namely, visual, auditory, and kinesthetic, which is known as (VAK) (VAK) (Hardiana & Suyata, 2018). The currents study used Fernald’s method, which consists of four stages, see figure (1) (Hanif et al., 2019). First, the students selected the words they want to learn, which the teacher wrote on boards by large letters, then students were asked to trace the letters with their fingers and writing them on their wrists. While tracing, he read it aloud. Accordingly, the students see (visual), say (auditory), and trace and write on the wrist (tactile and kinesthetic). In this stage, students activate four senses. This process has to be repetitive until students master and add the word in their minds’ word bank. The second stage is the stage of learning upgrade. In which the students no more selected the word to learn, they are asked to see, recognize by themselves the word that teachers wrote on cards or whiteboard, and make stories with these mastered words. The third stage is where students have to recognize, see, and read a word from book text rather than ones written on board. In the last phase, (the fourth), the students must have to go through and becomes able to recognize new words and retrieve the old ones.

Several studies examined the effectiveness of the VAKT strategy on reading skills.

Prasetyaningrum and Faradila (2019) found that the VAKT approach can be adopted effectively to improve the reading abilities of mild retardation children compared to the conventional teaching methods. They explained the increment of learning achievements due to practicing the VAKT strategy, which applies and extends their acquired knowledge. Agustia and Arifin (2018) targeted the same population of Prasetyaningrum and Faradila (2019) and assessed that the VAKT strategy developed 26.45% of students’ alphabetic learning and 17.60% of their computational learning. Agreed to Sarudin et al. (2019), the VAKT strategy not only enhances students’ acquiring and retrieving but also enables students to associate ideas with previously acquired knowledge. Jeyasekaran (2015) added that VAKT strategies enhanced reading levels of children with dyslexia and explained that VAKT strategy uses all learning pathways in the brain that why learning, and memorizing were enhanced by a mean score value of 12%. Similar to Arbi, Rianto and Murtadlo (2019), who conducted research of one group pre and post-test design and supported that the VAKT strategy has been influenced the reading ability of children with learning disabilities. Further, Gohar (2019) established that the VAKT strategy, which is used in English as a foreign language (EFL) program, improves phonological awareness and spelling abilities of dyslexia students according to the obtained results of the two groups research design.

http://ijlter.org/index.php/ijlter

125

All above reported studies used the multisensory strategies without defined which methods were adopted Orton-Gillingham's method or Fernald’s method.

However, the current study complies with Fernald’s method. Furthermore, the previous studies, such as Arbi et al. (2019), Prasetyaningrum and Faradila (2019), Agustia and Arifin (2018), Sarudin et al. (2019), targeted learning disabilities in general without defining specific learning disability. Similar to the current study, Jeyasekaran (2015) only targets students presenting dyslexia. But he detected the effect of the VAKT on reading skills without determined the effectiveness of the strategy in each reading skill. No one of the previous studies assesses the VAKT recruiting in Arabic language learning as the present study does.

Previous Arabic studies assessed the VAKT strategy to enhance the attention and perception of students with learning disabilities (see Mahmoud et al., 2020) but no study targets Arabic language teaching for students with dyslexia using the VAKT strategy according to the best knowledge of the author. Also, there is a lack of studies conducting each component of reading skills distinctly.

Accordingly, the study contributes to examine the VAKT strategy’s effectiveness for students representing dyslexia. Further, the study contributes to the implementation of VAKT in Arabic language learning settings.

Figure 1: Theoretical Framework using visual auditory kinesthetic tactile (VAKT) strategy on oral reading and reading comprehension skills .

Study Objectives The study aims to assess the VAKT strategy's effectiveness in developing oral reading and comprehension skills of students with learning disabilities. Hence, the researcher Hypothesizes that: H1: There is a significant mean score difference in the oral reading level of students with learning disabilities among control and experimental groups.

H2: There is a significant mean score difference in the reading comprehension level of students with learning disabilities among control and experimental groups.

H3: There is a significant mean score difference in the oral reading level of students with learning disabilities between pre and post-test measurement for the experimental group.

http://ijlter.org/index.php/ijlter

126

H4: There is a significant mean score difference in the reading comprehension level of students with learning disabilities between pre and post-test measurement of the experimental group.

3. Method The study has adhered to a quasi-experimental two groups’ pre and post-test design and quantitative approach (Gopalan et al., 2020) in the sake to answer the study question, namely,: “Does using VAKT Strategy impact the oral reading and reading comprehension skills of elementary students presenting dyslexia?”

Study Sample The study recruited 39 students with learning disabilities in third grades who enrolling Amman elementary schools using purposive sampling technique. The parents of all students were assigned a consent form of participation. All selected students were diagnosed with reading and reading comprehension dysfunctions (dyslexia). Also, they were with visual learning style. The students have distributed equivalently over two groups: the experimental group (20 students) and the control group (19 students). The samples were limited due to global infection of Covid-19 that hinders parents from engaging their children in the intervention. Experimental group students were taught Arabic corpus using the VAKT strategy (During the entire semester). In contrast, the control group was taught by a conventional scheme used in the source classrooms at this period. The study design composites of three main phases: Baseline, intervention, and followup (repeated measures), which will be illustrated next.

Instrument The study adopts the diagnostic scale of Arabic language basic skills developed by (Al-Waqfi et al., 2008). The Jordanian Ministry of education formally accredits this developed scale for screening Arabic language basic skills of pupils presenting learning difficulties. It is worth mentioning that the scale is an Arabic equivalent standardized version of the Kaufman Test of Educational Achievement-II (KTEA-II, (Kaufman & Kaufman, 2004)). KTEA-II measures phonological awareness, oral fluency, and reading fluency. KTEA-II is a screening instrument of the oral reading level (verbal fluency) and reading comprehension (RC) skills. Likewise, the Arabic-standardized version is designed for elementary and primary school ages (1st to 9th grades). The current study utilizes only the part for second and third elementary ages. However, the scale aims to assess reading skills. But the study targets the oral reading and the reading comprehension age of students only as succeeding.

The comprehension age scale consists of five questions that followed one passage.

The examiner asks an examinee to read the paragraph silently and answer the followed multiple-choice questions, which the examiner loudly read it. The student can either answer the question verbally or circle the correct answer on paper. Examiner calculates corresponding comprehension age based on the accuracy score. The accuracy score represents a percentage of correct answers to total questions. For example, the accuracy score of a student answering four

http://ijlter.org/index.php/ijlter

127

questions out of 5 is 80%. Accordingly, the examiner determines the comprehension age of a student using a scale scoring sheet.

Likewise, the instrument assesses the oral reading age of a student based on his reading accuracy. The student has to read a paragraph consisting of around 50 words. Accuracy score represents the percentage of corrected words corresponding to the number of reading errors (for example, if a student faults twice, the accuracy score will be 96%). Accordingly, the examiner determined the oral reading age of a student with the help of a scoring sheet.

Instrument administration times vary according to the examinee’s age, mood, and personality. The average time of students in the second and third grade-ages is 30 to 50 minutes.

Examiner starts using the lowest reading level (first semester of the first grade).

For each level, the examiner calculates the accuracy score of students. If the calculated score exceeds or equals the succeeding threshold, the examiner will continue with students to the next level. The process is continuous to the first score below the threshold. Accordingly, the examiner defines the reader's age based on the last level, which the student has scored above the threshold. Succeeded thresholds are 80% on the oral reading scale and 98% on the reading comprehension scale. For instance, if a student achieves succeeded age level score on the scale, he will pass to the next age level. The examiner defines the student’s age level based on the last succeeded age-level.

Furthermore, the instrument has two forms; form (A) and form (B) of both reading skill facets, in which form (B) is an alternative equivalent form of (A). In the study, the examiner administrated form (A) to the experimental group and form (B) to the control group.

Al-Waqfi et al. (2008) validated the instrument over a sample of 477 female and male students from different grade ages. The correlation between test and retest scores were significant and coefficients were above 0.7.

Study procedure The study conducted in the earliest period of emerged Covid-19 era. However, the students with learning disabilities, such as those presenting dyslexia, attending schools for remedial classes. HCD asked the government to permit students with learning difficulties to acquire learning through offline settings.

The researcher designed an individualized program based on the VAKT strategy, G. Fernald’s methods. The research implemented the VAKT program during the first semester of the 2020 academic year consisting of 64 sessions, four sessions each week. Because remediating reading skills for students with dyslexia require an intensive one-to-one program over an extended period (Mills, 2018).

However, the session duration is 35-40 minutes. The targeted subject is the Arabic language subject content (Arabic corpus) taught to the third elementary grade

http://ijlter.org/index.php/ijlter

128

students in the curriculum of Jordanian public schools. The program was conducted in the source classrooms of five public schools.

To validate the program, the researcher reported the VAKT program to an advisory panel that includes 15 academic specialists who have extensive expertise in teaching disabled students and professionals in learning disabilities. All their suggestions and recommendations have been taken into account.

The study recruited five remedial teachers implementing the VAKT program.

Required teachers hold at least a high diploma certificate on learning disabilities, and work in the capacity of a source teacher for at least two practical experience years. These qualified remedial teachers were selected purposively and recruited in the two weeks VAKT training program (30hrs). It is worth mentioning remedial teachers were naive to study objectives and hypotheses to sustain the validity and reliability of research results. Furthermore, each remedial teacher has to complete a session checklist after each session.

In the baseline stage, the participants (students) completed oral reading and comprehension reading tests. Both tests were administrated in separate sessions for one week. One teacher collects data and determines the oral reading and reading comprehension ages of all participants. The following chart (see fig.1) represents the oral reading and reading comprehension levels of both the experimental and control groups.

Figure (2) shows that the distribution of students based on their level in both the oral reading and the reading comprehension levels are at a first-grade level, with a minority of students have the third-grade level.

Reading _Comprehension

Oral reading 20 18 16 14 12 10 8 6 4 2 0

Exp

Control

Figure 2: Students' oral reading and reading comprehension levels for both experimental and control group

Accordingly, the intervention phase (treatment) started and last for three months.

However, teachers implemented a total of 60 sessions of intervention for each

http://ijlter.org/index.php/ijlter

129

participant discriminately (5 sessions per week, intensive program). As mentioned previously, the program instructed based on Fernald’s method of VAKT strategy, which incorporated specific instructional activities, such as word tracing, skywriting, and writing on the wrist, utilize the word in passages, and visual modulization of term, and instructional materials, such as word cards, highlighted and color pens, crayon, mirror, sand, and whiteboard.

Remedial teachers adhere to invest all modalities of students learning through teaching. For example, teachers introduce the word visually to students, make students tracing the term in their hands or via modeled terms juxtaposes to the oral presentation of terms to the student, skywriting the word, sand writing, mirror using to look to his self when sounded out the phoneme, and simultaneously tapping his fingers to the thumb.

The follow-up phase consists of two sessions in one week for repeated measurement after one week of intervention. Both oral reading and reading comprehension tests were re-administrated for each student individually.

Equivalence Analysis Before analyzing data the frequencies and percentages were extracted and the chisquare test (Chi2) was used to detect differences between control and experimental groups since the data type is categorical nor continuous (Rutkowski & Svetina, 2017) in order to identify the equivalence of groups to premeasurement for oral and comprehension Reading as shown in Table (1) below: Table 1: The percentage and frequency of experimental and control groups in each grade level and Chi-square on oral reading levels of students in the pre-measurement (N = 39) Pre_Oral_Reading 1st 1st 2nd 2nd 3rd grade grade grade grade grade (S1) (S1) (S2) (S1) (S2)

Group

Experimental

Control

Total

N P N P N P

9

7

2

1

45.0% 35.0% 10.0% 5.0% 12

7

63.2% 36.8% 21

14

53.8% 35.9%

Total

1

20

5.0%

100.0%

0

0

0

19

0.0%

0.0%

0.0%

100.0%

2

1

1

39

5.1%

2.6%

2.6%

100.0%

Chi2

Sig.

4.406

0.354

The results in table 1 supported that the students were equivalent in the premeasurement of oral reading, (Chi2) = (4.406) and it is not significant, which indicates the possibility of conducting a treatment to identify its effect in raising the level of oral reading among the students.

http://ijlter.org/index.php/ijlter

130

Table 2 :The percentage and frequency of experimental and control groups in each grade level and Chi-square on reading comprehension levels of students in the premeasurement (N = 39) Pre_Comprehension_Reading 1st 1st 2nd 2nd 3rd grade grade grade, grade, grade (S1) (S2) (S1) (S2) (S1)

Group Experimental N

6

P Control

2

1

20

30.0% 40.0% 15.0% 10.0%

5.0%

100.0%

N

8

P Total

N

3

9

1

1

0

19

42.1% 47.4%

5.3%

5.3%

0.0%

100.0%

4

3

1

39

35.9% 43.6% 10.3%

7.7%

2.6%

100.0%

14

P

8

Total

17

Chi2

Sig.

2.654a .617

Also, it was found that there were no statistically significant differences between control and experimental groups on the pre-measurement of reading comprehension skills, as the value of (Chi2) was (2.654) (see table 2) and it is not statistically significant, which indicates the permissibility of applying the treatment.

4. Result H1: There is a significant mean score difference in the oral reading level of students with learning disabilities among control and experimental groups.

Table 3: The percentage and frequency of experimental and control groups in each grade level and Chi-square on oral reading levels of students in the post-measurement (N = 39) Post_Oral_Reading 1st

1st 2nd 2nd 3rd grade grade grade, grade, grade (S1) (S2) (S1) (S2) (S1)

Group

Experimental

Control

N

0

P

0.0%

N

5

P

P

13

5

4

12

2

13

15

0 0.0% 4

12.8% 33.3% 38.5% 10.3%

*: significant at the level of (0.05).

http://ijlter.org/index.php/ijlter

2

Chi2

Sig.

20

5.0% 65.0% 20.0% 10.0% 100.0%

26.3% 63.2% 10.5%

N Total

1

Total

0

19

0.0% 100.0% 2

39

5.1% 100.0%

28.367a 0.00*

131

According to the table (3), the result illustrates the effectiveness of the VAKT strategy on oral reading levels. it is noted that results show an increment in oral reading levels of control group students like the experimental group. But the majority of the control group passed the first semester in the first grade, and only two students of the control group reached the first semester of the second grade (see table 1 and 3). However, the majority of experimental group students reached the first-semester level in the second grade. Further, around 30% of the experimental group scored above the first-semester level in the second grade.

Where it was noticed that there was an enhancement of the control group, it was noted that there were statistically significant differences among the experimental group (Chi2) = (28.367) and it is significance at the level of (0.05). Accordingly, hypothesis H1 is validated.

H2: There is a significant mean score difference in the reading comprehension level of students with learning disabilities among control and experimental groups.

Table 4: The percentage and frequency of experimental and control groups in each grade level and Chi-square on reading comprehension levels of students in the postmeasurement (N = 39) Post_Comprehension_Reading 1st

1st 2nd 2nd 3rd 3rd grade grade grade, grade, grade grade, (S1) (S2) (S1) (S2) (S1) (S2)

Group

Experimental

Control

N

0

P

0.0%

N

3

3

P

3

12

3

2

Chi2

Sig.

20

0.0% 15.0% 60.0% 15.0% 10.0% 100.0% 11

4

P 15.8% 57.9% 21.1% N

Total

0

Total

11

7

0

1

0

19

0.0%

5.3%

0.0%

100.0%

12

4

2

39

7.7% 28.2% 17.9% 30.8% 10.3% 5.1%

29.136a 0.00*

100.0%

It is noted from table (4) that the effectiveness of VAKT strategy on post comprehension reading among the experimental group since it showed an increase in the levels of second grade (the second semester), third grade (the first semester), and the third grade (second semester).

While the control group showed enhancement in comprehension reading, but less significant than the experimental group, as resulted in analyzing the VAKT strategy, Chi2 = (29.136) with a significant level less of (0.05). Accordingly, hypothesis H2 is validated.

http://ijlter.org/index.php/ijlter

132

H3: There is a significant mean score difference in the oral reading level of students with learning disabilities between pre and post-test measurements of the experimental group.

Table 5 :The difference between pre and post measurement in oral reading levels of the experimental group (N = 20) Oral Reading 1st 1st 2nd 2nd 3rd 3rd Measurement grade grade grade, grade, grade grade, (S1) (S2) (S1) (S2) (S1) (S2) Pre-test N 9 7 2 1 1 0 Post-test N 0 1 13 4 0 2 Total N 9 8 15 5 1 2

Total 20 20 40

Chi2

Sig.

26.36* 0.00*

As it is noted from table 5 that the VAKT strategy is effective; since the post oral reading among the experimental group mainly distributed on higher grade levels (first and second semesters in second grade) compared to pre-oral reading, which distributed in the lower grade levels (first and second semesters in first-grade).

However, the difference between pre and post measurements was significant since Chi2 = (26.36) with a significant level less of (0.05). Accordingly, hypothesis H3 is validated. This result supported that enhancement in the oral reading level is imputed to the effective VAKT strategy.

H4: There is a significant mean score difference in the reading comprehension level of students with learning disabilities between pre and post-test measurement of the experimental group.

Table 6: The difference between pre and post measurements in reading comprehension levels of the experimental group (N = 20) Reading Comprehension 1st 2nd 2nd 3rd 3rd Measurement grade grade grade, grade, grade grade, (S1) (S2) (S1) (S2) (S1) (S2) Pre-test N 6 8 3 2 1 0 Post-test N 0 0 3 12 3 2 Total N 6 8 6 14 4 2 1st

Total 20 20 40

Chi2

Sig.

24.14* 0.00*

It is clear from the table 6 that the post-reading comprehension levels of the experimental group mainly distributed on higher grade levels (first and second semesters in second grade and first semester in the third grade) compared to prereading comprehension levels, which distributed in the lower grade levels (first and second semesters in the first-grade). Hence, the VAKT strategy is effective.

However, the difference between pre and post measurements was significant since Chi2 = (24.14) with a significant level less of (0.05). Accordingly, hypothesis H4 is validated. This result supported that enhancement in the reading comprehension level is imputed to the effectiveness of VAKT strategy.

http://ijlter.org/index.php/ijlter

133

5. Discussion Succinctly, the results addressed that there are differences in students’ level in both comprehension and oral reading skills. Students taught by the VAKT strategy acquired higher levels compared to those students taught conventionally in both oral reading and reading comprehension skills (see tables 3, 4).

Furthermore, the results emphasized differences in the students’ oral reading and reading comprehension levels in the experimental group. Thus, the students presenting dyslexia developed their oral and reading comprehension skills sufficiently through the VAKT strategy adoption. In terms of reading comprehension, results are in accordance with that found by (Arbi et al., 2019; Gohar; 2019; Jeyasekaran; 2015). The statistically significant improvements in comprehension reading of the experimental group referred to the implemented multi-sensory approach (VAKT strategy) since students establish a reciprocal relationship among all sense inputs because the VAKT activates all senses.

Through the sessions, the student engages his kinesthetic sense by moving (such as in skywriting), tactile sense (such as in tracing words), visual sense (such as in word cards), and auditory (such as in listing activity). In turn, the student can easily acquire and consistently retain words and meanings.

In the context of oral reading skills, the results are in accordance with that found by (Arbi et al., 2019; Gohar, 2019; Jeyasekaran, 2015). Statistically significant improvements of oral readings in the experimental group referred to the implemented multi-sensory approach (VAKT strategy) since the students establish a reciprocal relationship among all sense inputs because the method activates all senses. Through the sessions, the student engages his kinesthetic sense by moving (such as in tapping and skywriting), tactile sense (such as in tracing words), visual sense (such as in word cards), and auditory (such as in listing activity). In turns, the student can easily acquire and consistently retain words and spellings. Using all sense of kinesthetic approach combined with hearing, seeing, observing make students utilize both sides of the brain to process multi-sensory information rather than one neurological pathway. Likewise, VAKT may provide a fun learning experience (Prasetyaningrum & Faradila, 2019; Sarudin et al., 2019). Thus, the VAKT strategy may motivate students to learn and read (Chither, 2020).

The descriptive statistical results pinpointed that the estimated average level development was between one and a half and two grade-levels in reading comprehension and between one to one and a half grade levels in oral reading.

Previous studies explained that increment of the reading comprehension level of experimental group associates with their augmentation level in oral readings skills (Solari et al., 2017; Yildirim & Ates; 2012). This supported that oral reading level is a significant predictor of the reading comprehension, which agreed to the verbal efficiency theory that points out the significance of word-level accuracy to reading comprehension. This can explain the ostensible difference in the level of development between comprehension and oral reading skills.

From another perspective, the increment of oral reading and comprehension can be explained according to results, which Mahmoud et al. (2020) obtained. They

http://ijlter.org/index.php/ijlter

134

established that the VAKT strategy improved and developed students’ perception and attention, which leads to the development of students acquiring and consequently comprehending since attention is a paramount factor in acquiring and learning.

6. Conclusion Dyslexia is a learning disability associated with a deficiency of spell, read, write, and recognize words. Students presenting dyslexia only need specific and individual teaching approaches aligned with the way information processed in their brain. Literature proposed the VAKT technique to effectively enhance language learning skills, such as reading, speaking, and writing. Thus, the current study examined the effectiveness of the VAKT strategy to improve oral reading and reading comprehension of the Arabic language among students with dyslexia. The intervention is based on specific instructional activities, such as word tracing, skywriting, and writing on the wrist, utilize the word in passages, and visual modulization of term, and instructional materials, such as word cards, highlighted and color pens, crayon, mirror, sand, and whiteboard. The researcher purposively selected 39 students at the third-grade level who are diagnosed with dyslexia divided into two groups (control=19; Exp.=20) with pre and post measurements. In the post-measurement, there is a notable and substantial improvement of oral reading and reading comprehension levels among groups.

Furthermore, the experimental group exhibited higher grade levels compared to control and in post measurements than pre-measurement in oral reading and reading comprehension skills. The obtained results support the VAKT strategy effectiveness for improving oral reading and reading comprehension of students with dyslexia. The research data was limited to a categorical data type. Therefore, it is recommended to adopt continuous scale measurements to assess further the effect size of VAKT further statistically on reading skills and making a comparison with normally developing peers of the same age. The researcher recommends considering learning style in future studies as variable. Motivation for reading also can be measured due implication of the VAKT strategy. Moreover, there is a need for further examining the applicability of the VAKT approach in both schools, home, and online settings.

7. References Abuzaid, H., & Kayed, M. A. (2020). The impact of using storyboards on Improving reading skills of third-grade students with reading disabilities in Jordanian context. International Journal of Learning, Teaching and Educational Research, 19(1), 172-187. https://doi.org/10.26803/ijlter.19.1.10 AL-Qatawneh, Y. H. (2020). Effectiveness of a computerized program in the treatment of Dyslexia in a sample of fourth-grade students. Asian Social Science, 16(7), 15-29. https://doi.org/10.5539/ass.v16n7p15 Al-Waqfi, R., Alkilani, A., Jarar, S., & Aldawi, M. (2008). Basic skills of Arabic language: A diagnostic scale (4th ed.).The princess Sarvath community college.

Al-Zboon, E., & Hatmal, M. M. (2015). Attitudes of dentists toward persons with intellectual disabilities in Jordanian hospitals. Special Care in Dentistry, 25-31. https://doi.org/10.1111/scd.12149 APA. (2020). Diagnostic and statistical manual of mental disorders (DSM-5). American Psychiatric Association.

http://ijlter.org/index.php/ijlter

135

Arbi, R. P., Rianto, E., & Murtadlo, B. (2019). Influence of VAKT method toward reading ability to learning difficulty children in Galuh Handayani elementary school. 3rd International Conference on Special Education (ICSE 2019), Vol. 338, (pp. 225-228).

Surabaya, Indonesia: Atlantis Press SARL. Chither, H. R. (2020). Effect of Vernald way (VAKT) for multiple senses, health care, and clinical learning in dealing with hardship and written to the reading first stage pupils. Indian Journal of Public Health Research & Development, 11(2). http:/doi.org/10.37506/v11/i2/2020/ijphrd/195205 DOS. (2015). General population and housing census 2015. The Department of Statistics.

Gohar, R. (2019). The effect of multisensory structured EFL program on developing primary pupils' phonological awareness and spelling. Educational Sciences Journal, 27(2), 1-35. http://search.shamaa.org/PDF/Articles/EGJes/JesVol27No2P2Y2019/jes_2019 -v27-n2-p2_001-036_eng.pdf Grigorenko, E. L., Compton, D. L., Fuchs, L. S., Wagner, R. K., Willcutt, E. G., & Fletcher, J. M. (2020). Understanding, educating, and supporting children with specific learning disabilities: 50 Years of science and practice. American Psychologist, 75(1), 37-51. http://dx.doi.org/10.1037/amp0000452 Hanif, S., Achmad, L., Madjdi, H., & Utom, S. (2019). The Vakt model-based on psycholinguistic review for overcoming Dyslexia children. CONECT 2019: Proceeding of the 2nd International Conference Education Culture and Technology, ICONECT 2019 (pp. 46-56). Kudus, Indonesia: European Alliance for Innovation.

Hardiana, M. T., & Suyata, P. (2018). The effectiveness of VAK (visual, auditory, kinesthetic) model in learning of summary writing. International Journal of Research & Review, 5(8), 43-49. http://www.ijrrjournal.org/IJRR_Vol.5_Issue.8_Aug2018/IJRR008.pdf Hayes, A. M., Dombrowski, E., Shefcyk, A., & Bulat, J. (2018). Learning disabilities screening and evaluation guide for low- and middle-income countries. RTI Press.

HCD. (2018). First annual report on the status of persons with disabilities and their rights in the Hashemite Kingdom of Jordan. The Higher Council for the Rights of Persons with Disabilities (HCD).

Jeyasekaran, J. M. (2015). Effectiveness of visual auditory kinesthetic tactile technique on reading level among dyslexic children at Helikx open school and learning center, Salem. International Journal of Medical Science and Public Health, 4(3), 315-318. https://dx.doi.org/10.5455/ijmsph.2015.0511201467 Kaufman, A. N., & Kaufman, S. L. (2004). Kaufman test of educational achievement (2nd ed.).

American Guidance Services.

Korkmaz, Ş. Ç., & Karatepe, Ç. (2018). The Impact of multi-sensory language teaching on young English learners’ achievement in reading skills. Novitas-ROYAL (Research on Youth and Language), 12(2), 80-95. https://eric.ed.gov/?id=EJ1195281 Law No. 20. (2017). Law on the rights of persons with disabilities No. 20. Jordanian Government.

Mathias, B., Klingebiel, A., Hartwigsen, G., Sureth, L., Macedonia, M., Mayer, K., & Kriegstein, K. V. (2019). Sensorimotor cortices casually contribute to auditory foreign language vocabulary translation following multisensory learning. Brain Stimulation, 12(2), 401-402. https://doi.org/10.1016/j.brs.2018.12.295.

Mills, J. R. (2018). Effective multi-sensory strategies for students with Dyslexia. Kappa Delta Pi Record, 54(1), 36-40. https://doi.org/10.1080/00228958.2018.1407181 Norton, E. S., Beach, S. D., & Gabrieli, J. D. (2015). Neurobiology of dyslexia. Current Opinion in Neurobiology, 30(1), 73-78. https://doi.org/10.1016/j.conb.2014.09.007

http://ijlter.org/index.php/ijlter

136

Peter, B., Albert, A., & Gray, S. (2020). Spelling errors reveal underlying sequential and spatial processing deficits in adults with dyslexia. Clinical Linguistics & Phonetics, 1-29. doi:https://doi.org/10.1080/02699206.2020.1780322 Prasetyaningrum, S., & Faradila, A. (2019). Application of VAKT methods (visual, auditory, kinestetic, and tactile) to improve the reading ability for mild mental retardation. Advances in Social Science, Education, and Humanities Research (ASSEHR), 304, 379-385. https://doi.org/10.2991/acpch-18.2019.91 Rao, A. R. (2018). An oscillatory neural network model that demonstrates the benefits of multisensory learning.

Cogn Neurodyn, 12(5), 481-449. https://doi.org/10.1007/s11571-018-9489-x Sarudin, N. A., Hashim, H., & Yunus, M. M. (2019). Multi-sensory approach: How it helps in improving word recognition? Creative Education, 10(12), 3186-3194. https://doi.org/10.4236/ce.2019.1012242 Solari, E., Grimm, R., McIntyre, N., Lerro, L. S., Zajic, M., & Mundy, P. C. (2017). The relation between text reading fluency and reading comprehension for students with autism spectrum disorders. Research in Autism Spectrum Disorders, 4142(2017), 8-19. https://doi.org/10.1016/j.rasd.2017.07.002 Subramaniam, V., & Nasir, N. S. (2020). Multisensory therapy in letter reversal of Dyslexic pupils. Universal Journal of Educational Research, 8(12), 7118-7130. https://doi.org/10.13189/ujer.2020.081279 Suryaratri, R. D., Prayitno, E. H., & Wuryani. (2019). The implementation of multi-sensory learning at elementary schools in Jakarta. Jurnal Pendidikan Usia Dini, 13(1), 100113. https://doi.org/10.21009/10.21009/JPUD.131.08 Thompson, S. (2018). The current situation of persons with disabilities in Jordan, K4D helpdesk report. Institute of Development Studies.

WHO. (2019). International statistical classification of diseases and related health problems(11th revision) (Vol. 11th revision). World Health Organization.

Yildirim, K., & Ates, S. (2012). Silent and oral reading fluency: Which one is the predictor of reading comprehension of Turkish elemntary students?. International Journal on New Trends in Education and Their Implications, 3(4), 79-91. http://ijonte.org/FileUpload/ks63207/File/07.yildirim.pdf

http://ijlter.org/index.php/ijlter

137

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 137-151, February 2021 https://doi.org/10.26803/ijlter.20.2.8

The Influence of Architecture Students’ Learning Approaches on their Academic Performance in Two Nigeria Universities Gabriel Sen* Covenant University, Nigeria https://orcid.org/0000-0002-7039-1903 Albert Adeboye Olabisi Onabanjo University, Nigeria https://orcid.org/0000-0003-2985-1426 Oluwole Alagbe Covenant University, Nigeria https://orcid.org/0000-0003-2779-1861

Abstract. The paper was a pilot study that examined learning approaches of architecture students; variability of approaches by university type and gender and; influence of architecture students’ learning approaches on their academic performance. The sample was 349 architecture students from two universities. Descriptive and statistical analyses were used. Results revealed predominant use of deep learning approaches by students. Furthermore, learning approaches neither significantly differed by university type nor gender. Regression analysis revealed that demographic factors accounted for 2.9% of variation in academic performance (F (2,346) = 6.2, p = 0.002, R2 = 0.029, f2 = 0.029) and when learning approaches were also entered the model accounted for 4.4% of variation in academic performance (F (14,334) =2.2, p =0.009, R2 = 0.044, f2=0.044). Deep learning approaches significantly and positively influenced variation in academic performance while surface learning approaches significantly and negatively influenced academic performance. This implies that architectural educators should use instructional methods that encourage deep approaches. Future research needs to use larger and more heterogeneous samples for confirmation of results.

Keywords: architecture students; students’ learning approaches; deep learning approaches; surface learning approaches; academic performance *Corresponding

Author: Gabriel Sen; Email: gpnlsen@gmail.com; gabriel.sen@stu.cu.edu.ng. ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

138

1. Introduction Universities in Nigeria and even worldwide desire that students pass their examinations in order to graduate on time. Failure of students to graduate on time results in substantial financial losses and several administrative problems.

Architectural educators are, therefore, always looking for strategies that would help students complete their educational careers on time. One of such strategies is in studying the learning approaches of architecture students. Results from previous studies have shown that, learning approaches of students in higher education significantly influence academic performance (Carstensen, Ødegaard and Bonsaksen, 2018; Öhrstedt & Lindfors, 2019). Puteh et al. (2018) on the other hand reported that learning approaches do not predict academic performance.

Knowledge of the relationship between architecture students’ learning approaches and their academic performance will undoubtedly reveal the priority that architectural education should give to learning approaches. This pilot study is meant to achieve this aim.

1.1 Student Learning Approach Learning approach is the intention and strategy adopted by students in acquisition of knowledge (Biggs & Tang, 2007). Learning approaches are not an inherent but acquired trait or strategy determined by contextual demands (Chonkar, Ha & Chu, 2018). Furthermore, the learning approach of a student is characterised by a change in study behaviours dictated by the student’s perception of the learning task (Dilek & Noor, 2015). Previous studies have categorised learning approaches into two: surface learning approach and deep learning approach (Ayalp, 2015; Howie & Bagnell, 2013).

Surface learning approach is depicted by the students’ behaviour of adopting strategies of memorisation of ideas and facts during studies. Information is studied as unrelated and isolated facts (Shaik et al., 2017).The emphasis is not that of understanding the underlying principles of the information, but rather on rote learning. Ayalp (2015) opines that such a strategy results in the student hardly retaining knowledge after examinations. The surface learning approach is, therefore, extrinsically motivated, which is a kind of motivation due to what the student sees around him (Noori et al., 2020). The student that focuses on the surface approach seeks to overcome tasks with minimum efforts.

Deep learning approach on the other hand is depicted by the student’s adoption of aggressive engagement with the materials being studied. The student seeks to relate newly acquired information with precious ideas (Dilek & Noor, 2015). The student tries to understand the patterns and underlying principles of the study material. In this approach, there is the promotion of deeper reasoning; higher order thinking that is followed by self-directed learning (Rakibul-Islam & Shahriar Shafiq, 2016). The student adopting a deep learning approach, therefore, goes beyond mere acquisition of knowledge, but analyses, synthesises and evaluates the subject matter. The subject matter is integrated with real life situations and compared with the student’s personal experience. Although this method focuses on the subject matter, the student is still mindful that meaning is conveyed by words, text and formulae. The deep learning approach is intrinsically motivated, which is a type of motivation that is from the student,

http://ijlter.org/index.php/ijlter

139

but not from what the student sees externally (Noori et al., 2020). Danaci (2014) reports that, in architectural education, the failure of students in adopting deep learning approach results in their inability to defend their designs. This obviously affects their ability of passing their examinations and also results to deficiencies in their future careers. Suffice it to mention that the architectural curriculum emphasises vocational skills which, according to Danaci (2014), should require a deep learning approach.

1.2 Student Learning Approach and Academic Performance Previous studies have shown that learning approach is a process factor of Biggs’s 3P theory that affects the learning outcome of the students (Song, 2018).

According to the theory, three factors of presage, process and product are responsible for learning in educational institutions. The presage factor could refer to demographic variables like university type and gender. It could also include learning environment. The process factor, on the other hand, refers to students’ learning approaches while the product factor refers to the student’s learning outcome such as the student’s academic performance. The academic performance could be expressed in form of students’ Cumulative Grade Point Average (CGPA). The theory further postulates that the personal factor can directly affect the students’ learning outcome. Also, that the process factor (students’ learning approach) can also directly affect the students’ learning outcome. Furthermore, the students’ learning approach can influence the students’ learning outcome even when the presage factor is acting as the intervening variable. Based on the 3P theory, studies have been carried out on learning approaches on the premise that the approach scores of the students could be an index of their teaching quality. By implication, the learning approach scores of the student are useful in helping the teacher realise better ways of meeting the students’ needs. This could be in terms of assessment, curriculum, overload, teaching methods and learning environment (Entwistle, 1988).

Previous studies have, therefore, investigated the predominant learning approaches of students in higher education. Hussin, Hamed and Md-Jam (2017) investigated the predominant learning approaches of engineering students at University of Technology Mara from four different faculties. The findings revealed that engineering students predominantly use deep learning approach.

Kamath et al. (2018) also investigated the learning approaches of second year medical students and the findings also revealed that the students adopted a deep learning approach. Arsaythamby et al. (2015), on the contrary, reported a predominant use of surface learning approaches by English undergraduate students among University Utara, Malaysia.

The findings from previous studies have also shown that students’ learning approaches could be affected by the school type. Khan et al. (2020) investigated how peculiarities in school leadership and teaching could affect students’ learning experiences, which will inadvertently affect their learning approaches.

Concomitantly, Leung, Fang and Lu (2015) compared the learning approaches of universities in mainland China and Hong Kong City and found learning approaches to differ significantly in terms of university type. This was, however,

http://ijlter.org/index.php/ijlter

140

contrary to Ayalp (2015) who compared engineering students’ learning approaches from four universities and found no significant difference in approaches.

Previous studies have also identified the need to explore gender differences in learning approach research. Rosander and Martin (2012) have shown that gender stratification affects school experiences. It is, therefore, essential that this pilot study is done with due recognition of gender stratification. Several studies have previously found no differences of learning approaches by gender. Keithia et al. (2011) reported no gender differences in learning approaches in first year psychological students. Such was also the findings of Arteche et al. (2009) on psychology students, both from five British and two American universities.

Other previous studies, however, have identified differences in learning approaches based on gender (Biggs, 1987).

Previous studies have further examined the impact of learning approaches on academic performance and have found varied results. Salamonsun et al. (2013) reported that deep and surface learning approaches from across five different disciplines of Nursing, Engineering, Mechanical, Health Sciences and Medical Chemistry significantly affected the students’ academic performance. Surface learning approach negatively predicted students’ performance (B = -0.13, p = 0.001) while deep learning approach positively predicted the students’ performance. Also, in a study on a sample of education students, Cetin (2016) reported a positive relationship between deep learning approach and students Grade Point Average (GPA) and a negative relationship between the surface approach and the student’s GPA. In a sample of undergraduate students of philosophy, education and psychology, Karagiannopoulou and Milienos (2015) also found that surface learning approach statistically, significantly and negatively predicted students’ GPA, whereas the deep learning approach had no effect on the students ‘GPA. Choy, O’Grady and Rotgans (2012) found that approaches to learning are weak predictors to academic achievement. Using a sample of students drawn from four faculties that included architecture, planning and survey, science computer and mathematics, arts and accounting, Moh’d et al. (2015) reported no prediction of learning approaches on academic performance. Such were also the findings of Bonsaksen et al. (2017) in a study which involved a sample of undergraduate occupational therapy students from four countries, Australia, Norway, Hong-Kong and Singapore.

As could be seen from the above literature, previous studies on student learning approaches are seemingly conducted mainly in areas outside the discipline of architecture. Even when such studies included architecture students in their sample, the proportion of students from other disciplines remained dominant.

Moreover, the apparent discrepancies in results created a need for more research in learning approaches and particularly in the architectural discipline. The present pilot study is meant to fill this gap.

http://ijlter.org/index.php/ijlter

141

1.3 Research Questions and Hypotheses This pilot study is meant to address three questions: 1) What are the predominant learning approaches of architecture students?

2) What are the disparities in students’ learning approaches in terms of demographic factors of university types and gender?

3) What is the influence of the architecture students’ learning approaches on the cumulative grade points average (CGPA) after accounting for demographic factors of university type and gender?

The study made use of the following hypotheses; Hypothesis H1: There is a statistically significant difference between learning approach scores in terms of university type. The associated hypothesis (H10) is that there is no statistically significant difference between learning approach scores in terms of university type.

Hypothesis H2: There is a statistically significant difference between male and female students’ learning approaches scores. The associated null hypothesis (H20) is that there is no statistically significant difference between male and female learning approach scores.

Hypothesis H3: Learning approaches account for a variance in academic performance when demographic variables of gender and universities types are controlled for. The associated null hypothesis (H30) is that learning approaches do not account for any variance in academic performance when the demographic variables of gender and university type are controlled for.

2. Materials and Methods This section of the paper refers to the procedure and the specific tools that were used in carrying out this study. This is presented in the following subheadings.

2.1 Research Strategy A cross-sectional survey was adopted for this pilot study. According to Sedgwick (2014), this method of survey has the advantage of not requiring too much time in capturing data. Data were, therefore, collected once for each of the two universities studied. This was during the first semester of 2018/2019 academic year.

2.2 Participants Census sampling techniques was used. The technique involved all the undergraduate architecture students in second, third and fourth years. The sample also included postgraduate students of architecture in the second year.

These were all from Rivers State University, Portharcourt (RSU) and University of Uyo, Uyo (UNIUYO). The first-year undergraduate students were deliberately left out of the study since the school curricular indicated that the real architectural education starts in the second year. The first-year postgraduate students were also left out of the study since at the time of study the universities

http://ijlter.org/index.php/ijlter

142

had not fully admitted their first-year postgraduate students. Moreover, even those admitted had not obtained any CGPA. The choice of the universities was purposive. RSU as a state government sponsored university was chosen to represent the state government owned universities. University of Uyo, Uyo (UNIUYO) as a federal government owned university was chosen to represent the federal government owned universities. The emphasis of the pilot study was on public universities.

2.3 Data Collection Tools Quantitative data for the study were collected using anonymous questionnaire that had two parts, as displayed in appendix I. The first part collected information on the students’ personal factors of university type, gender and their current CGPA. The self-report method was adopted due to the administrative difficulties encountered in obtaining access to students’ academic records. The second part of the questionnaire had the Biggs, Kember and Leung (2001) Revised Student Study Process Questionnaire (R-SPQ-2F).

This questionnaire has 20 items that are equally divided into two main subscales, surface learning approach and deep learning approach. The subscale for surface learning approach has 10 items with five items meant for surface motive and the other five items meant for surface strategy. An example of items measuring the surface motive approach is; “My aim is to pass the course while doing as little work as possible” and an example of the item measuring surface strategy is “I only study seriously what’s given out in class or in the course outlines”. The subscale for measuring deep learning approach also has 10 items with five items measuring deep motive and the other five items measuring deep strategy. Example of items measuring the deep motive is; “I find that sometimes studying gives me a feeling of deep personal satisfaction” and the example of the items measuring the deep strategy is; “I find that I have to do enough work on a topic so that I can perform my own conclusions before I am satisfied”. The measures are on a five point Likert scale rated from 1-5. ‘1’ indicates Never or only rarely true of me while 5 indicate salways or almost always true of me. The scoring system is as prescribed by Biggs et al.

(2001): Surface approach score:∑ All surface motive scores + all surface strategy scores.

Deep approach score:∑ All deep motive scores + all deep strategy scores.

2.4 Data Collection Method A total of 444 questionnaires were administered on the students, some before their lecture session and others after such sessions. The participants filled the questionnaires within the space of twenty minutes. Only 349 of the questionnaires that were returned and completed were used for the analysis.

2.5 Data Analysis The Cronbach’s alpha of the R-SPQ-2F was fully determined to ascertain the reliability of the scale. Descriptive analysis was used to determine the predominant learning approaches of the participants, which answered the first research question. The second and third research questions were transformed into hypotheses [(H1-H3) and (H10– H30)]. In order to test the first two null hypotheses (H10 and H20), one way multivariate analysis of variance (MANOVA) was used. Categorical regression was used to test the third null

http://ijlter.org/index.php/ijlter

143

hypotheses (H30). This was because the dependent variable, CGPA was measured categorically. Students indicated the ranges of their current CGPA. Categorical regression could, therefore, be used as submitted by Choy et al.

(2012). The program used for analysis was IBM, SPSS version 21.

3. Result The demographics of the respondents in terms of university type and gender are shown in Table 1: Table 1: Demographic Analysis of the Respondents Demographic Information

Frequency

Percentage

University Type

Rivers State University Portharcourt University of Uyo, Uyo Total

152 197 349

43.6% 56.4% 100%

Gender

Male Female Total

259 90 349

74.2% 25.8% 100%

The Cronbach’s alpha value of the R-SPQ-2F instrument revealed 0.781 for deep learning approach and 0.70 for surface learning approach. This, according to Pallant (2011), is good for analysis. When descriptive analysis was used to determine the mean scale of the two learning approaches, the result was as presented in Table 2. Deep learning approach (DA) had predominant scores (M =33.56, SD = 6.87) over surface learning approach (M = 27.96, SD = 6.65).

Table 2. Descriptive Statistics of Approaches Scores Deep approach Surface approach

N

Mean

Standard Deviation

349 349

33.56 27.97

6.87 6.65

To test for the first null hypotheses (H10), a one-way multivariate analysis of variance (MANOVA) was also carried out to determine whether the combined students learning approach scores differed in terms of university type. Preliminary checking of assumption for the above analysis sufficed. There was, however, homogeneity of variance-covariance matrices as assessed by Box’s M test (p =0.193). Students in RSU and UNIUYO had higher deep learning approach scores (M = 33.4, SD =6.52; M =33.7, SD = 7.14) than their surface learning approach scores (M = 26.7, SD = 7.1; M =28.2, SD = 6.2), respectively. The difference between universities on the combined learning approaches scores was not statistically significant F(2,346) = 0.446, p =0.641; Wilk’s ^= 0.997; partial η2 = 0.003. This shows that the null hypothesis that there is no statistically significant difference between learning approach scores in terms of university type cannot be rejected. The implication is that learning score approaches do not vary by the type of university.

To test for the second null hypotheses (H20), a one-way multivariate analysis of variance (MANOVA) was carried out to determine whether the differences in learning approaches scale were statistically significant in terms of gender. There was the

http://ijlter.org/index.php/ijlter

144

preliminary checking of assumptions for suitability of analysis which revealed that data were normally distributed as assessed by Q-Q plot; there was no serious violation on univariate outliers when trimmed means were inspected against the actual means of the dependent variables [deep approach (mean= 33.56, trimmed mean=33.80; surface approach (mean =27.96, trimmed means =28.01]. There were no multivariate outliers as revealed by Mahalanobis distance (p>0.001). There was linear relationship as revealed by scatter plot and no multi-collinearity (r = 0.203, p> 0.0005), and there was also homogeneity of variance-covariance matrices as assessed by Box’s M test (P = 0.155). Male and females had higher deep approach score (M =33.5, SD = 71; M =33.7, SD = 6.2) than surface approaches scales (M = 28.1, SD = 6.5; M = 27.4, SD = 7.2), respectively. The difference between male and female students on the combined learning approach scores was not statistically significantly as F(2,346) = 0.497, p = 0.609; Wilks’ ^=0.997; partial η2 = 0.003. This shows that the null hypothesis that there is no statistically significant difference between male and female learning approach scores cannot be rejected. The implication is that learning approach score do not vary by gender.

In testing for the third null hypothesis (H30), categorical regression was first carried out using the students’ CGPA as the response and demographic factors of age and university type as predictors. The results as shown in Table 2 revealed that the demographic variable of university type was significant (β=0.178, p< 0.005) and this accounted for 2.9% of variance in academic performance (F(2,346) = 6.2, p = 0.002, adjusted R2 = 0.029, f2 = 0.029) and when learning approaches were added as variables, the influence of deep learning approach was significant (β=0.22, p< 0.005), while surface learning approach did not show any significant influence in academic performance (β= 0.134, p =0.773).These all accounted for 4.4% of the variance in academic performance (F(14,334) =2.2, p =0.009, Adjusted R2 = 0.044, f2 =0.044). This shows that the null hypothesis (H3) that learning approaches do not account for any variance in academic performance when the demographic variables of gender and university type are controlled should be rejected. This shows that learning approaches do influence academic performance even when demographic factor of gender and university are acting as intervening variables.

Table 3. Categorical Regression Results Independent variables

Standardised Coefficients Beta (β)

F

p

f2

Adjusted R2

F

p

Demographic Factors University 0.178 Gender 0.059 Student Learning Approaches Deep learning 0.229 approaches Surface learning -0.134 approaches

*P<0.05

http://ijlter.org/index.php/ijlter

12.8 1.3

0.000* 0.252

0.029

0.029

6.2

0.002

6.9

0.000*

0.44

0.44

2.2

0.009

-0.08

.773

145

4. Discussion The result of this study has shown that architecture students predominantly use a deep learning approach. This is consistent with the findings of Moh’d et al. (2015) on a study that had a sample that included students of architecture, planning and survey. It revealed predominant scores of deep learning approach (M = 35.01, SD=5.715) as against the surface learning approach (M = 30.17, SD=6.33). Ayalp (2015) also reported the predominant use of deep learning approach (M = 32.21, SD=5.4) over the surface learning approach (M= 29.07, SD=6.35) on a sample of engineering students. Such also were the findings of Hussin et al. (2017) who used a sample of engineering students and found deep learning approach scores with a mean (M = 32.8) higher than the surface learning approaches scores mean (M=27.27). Comparing this study with other studies in the architectural discipline is rather difficult due to the dearth of similar studies in architectural education. The similarity of this study’s result with that of engineering students might not be surprising. This might be due to the similar emphasis on problem-based learning in the two disciplines. Dilek and Noor (2015) have reported that problem-based learning, among other factors like the nature of curriculum and assessment methods, encourages the use of deep learning approaches.

It could also mean that the students’ predominant use of deep learning approaches is as a result of the collaborative learning occasioned by the design studio which, according to Oluwatayo, Aderonmu and Aduwo (2015), is the sine qua non of architectural education. Collaborative learning, according to Tee-Meng et al. (2016), enhances the use of deep learning approach among students. The implication of the results in this study is that architectural education for these data is on the right course.

The schools only need to heighten the use of deep learning approaches.

The results have also shown that the use of learning approach also did not differ in terms of university type. Similarly, Ayalp (2015) studied disparities in learning approaches among three universities and revealed no differences in learning approach scores. The result is, however, contrary to that of Colak and Kaya (2014) who found that school type influences student learning approaches. The lack of difference in learning approaches based on university type could be due to the similarities of curriculum which is centrally controlled by the Nigeria University Commission (NUC). The implication is that common measures could be adopted across the two universities in fostering appropriate student learning approaches.

The results have further shown that learning approaches of the students did not differ by gender. This is similar to the findings of Hussin et al. (2017) who used a sample of engineering students and also to Kamath et al. (2018) using a sample of medical students. This is, however, contrary to the report of Arsaythamby et al. (2015) in a study carried out on a sample of English undergraduate students. The lack of gender difference in learning approaches of architecture students could mean that architectural educators need not bother about evolving special instructional methods that will avoid gender bias in the learning approaches of the students.

The results of the study have also revealed that deep learning approach positively impacts the academic performance of the students. This is both directly and even when demographic variables like university type and gender are acting as intervening variables. This is in line with Biggs’s 3P theory that the process factor has a direct and indirect impact on academic performance. It is also similar to other studies like Ladan

http://ijlter.org/index.php/ijlter

146

et al. (2014) who found learning approaches as a predictor of academic performance.

This shows the importance of student approaches to academic performance. The results have further shown that, while deep leaning approaches positively impact academic achievement, surface learning approach negatively impacts academic achievement. This is similar to the findings of Lizzio, Wilson and Simons (2002) and Moore (2015) that university students with deep learning approaches obtain higher grades in academic performance. Such is also the finding of Belaineh (2017). This implies that both architectural educators and students, in order to facilitate success in examinations that will enable their students to graduate on time, should adopt instructional strategies that would encourage deep learning approaches of architecture students. According to Moh’d et al. (2015), when instructors fail to actively involve students in class discussions, students tend to apply rote learning, which hinders them from acquiring critical thinking skills. Al-Busaidi, Yusuf and Reinders (2021) reported that research has also established collaboration, which promotes a deep learning approach can be achieved in the classroom through the introduction of problem-based learning. Tee-Meng (2016) has suggested that some of the techniques instructors could adopt in order to assist students imbibe deep learning approaches include role playing, and small and large group discussions. Additionally, teachers in architectural education should also ensure that the assessment and evaluation is tailored to align with the desire for the adoption of deep learning approaches. Furthermore, universities should also ensure that workload for the students should allow the students to engage enough time in explaining ideas and developing interest in their studies. The problem of the distraction of academic programmes, which in the Nigerian experience is mainly caused by incessant strikes in the Nigerian university education, should be seriously discouraged. It affects student workload, which adversely discourages the students’ adoption of deep learning approaches in their studies.

5. Conclusion The study has examined the predominant learning approaches of architectural students of two public universities in Nigeria in sub-Saharan Africa. The study has identified the deep learning approach as the predominant approach for the architecture students studied. Furthermore, the study has shown that student learning approaches do not differ in terms of gender and university type. The study had the overarching aim of establishing whether there exists any impact of architecture students’ learning approaches on academic performance with a view of suggesting ways that will ensure success in examinations in architectural education. This will undoubtedly facilitate their graduation on time. The study has, therefore, shown that students’ learning approach has an impact on their academic performance. Also that, while deep learning approach positively affects academic performance, surface learning negatively affects academic performance. The study, however, has a sample that involved only public universities and even so from a location that is restricted in a particular part of the country, Nigeria. There is need to use bigger and more heterogeneous samples for future research that would involve private universities as well. Accordingly, this study serves as the research base for a study the researcher intends to carry out on the entire Southsouth geo-political zone in Nigeria.

Furthermore, there is need to carryout qualitative studies to further confirm the quantitative results in the study. This is on the premise that some researchers have

http://ijlter.org/index.php/ijlter

147

claimed that a mixed research that combines qualitative and quantitative results could be more valid (Lochmiller & Lester, 2017).

Acknowledgements: The authors are thankful to all partners; architecture students of RSU and UNIUYO; lecturers in the architecture department of RSU and UNIUYO; and all members of the research team who supported our study.

6. References Al-Busaidi, S., Yusuf, T., & Reinders, H. (2021). A model for implementing problem-based language learning: experience from a seven-year journey. International Journal of Learning, Teaching and Educational Research, 20(1),1-21. https://doi.org/10.26803/ijlter.20.1.1 Arsaythamby, V., Hariharan, N., Krishnasamy, H., & Harun, M. (2015). What are the learning approaches applied by undergraduate students in English process writing based on gender?.

International Education Studies, 8(6), 50-54. https://doi.org/10.5539/ies.v8n6p46 Arteche, A., Chamorro‐Premuzic, T., Ackerman, P., & Furnham, A. (2009). Typical intellectual engagement as a by product of openness, learning approaches, and self‐assessedintelligence. Educational Psychology, 29(3), 357-367. https://doi.org/10.1080/01443410902927833 Ayalp, G. G. (2015). Relationships between learning approaches of civil engineering undergraduate in three Turkish universities and success in construction management courses. International Journal of Engineering Education, 31(6A), 154–251.

Belaineh, M. S. (2017). Students’ conception of learning environment and their approach to learning and its implication on quality education. Educational Research and Reviews, 12, 14-16. https://doi.org/10.5897/ERR2017.3258 Biggs, J. B. (1987). Student approaches to learning and studying. Australian Council for Educational Research.

Biggs, J. B., Kember, D., & Leung, D.Y. (2001). The revised two factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149. https://doi.org/10.1348/000709901158433 Biggs, J. B., & Tang C. (2007). Teaching for quality learning at university (3rd ed.). Berkshire: Open University Press.

Bonsaksen, T., Brown, T., Fong, K. N., Lan, T. H., Murdolo, Y., Gonzalez, P. C., & Beng, L. H. (2017). Approaches to learning among occupational therapy undergraduate students: A cross-cultural study. Scandinavian Journal of Occupational Therapy, 24(4), 299-310. https://doi.org/10.1080/11038128.2016.1229811 Carstensen, T., Ødegaard, N. B., & Bonsaksen, T. (2018). Approaches to studying: Associations with learning conceptions and preferences for teaching, cogent education. Journal of Social Science Research, 6(1), 746-752.

Cetin, B. (2016). Approaches to learning and age in predicting college students’ academic achievement. Journal of College Teaching and Learning (TLC), 13(1), 21-28. https//doi. org/10.19030/tlc.v13i1.9568 Chonkar, S. P., Ha, T. C., & Chu, S. H. (2018). The predominant learning approaches of medical students. BMC Medical Education, 18(17), 11-25. https://doi.org/10.1186/s12909-0181122-5 Choy, J. L. F., O’Grady, G., & Rotgans, J. I. (2012). Is the Study Process Questionnaire (SPQ) a good predictor of academic achievement? Examining the mediating role of achievement-related classroom behaviours. Instructional Science, 40, 159–172. https://doi.org/10.1007/s11251-011-9171-8.

http://ijlter.org/index.php/ijlter

148

Colak, E., & Kaya, D. (2014). Learning approaches of vocational high school students: grade level and school type influences. Procedia-Social and Behavioral Sciences, 116(15), 15561561. https://doi.org/10.1016/j.sbspro.2014.01.434 Danaci, H. M. (2014). Creativity and knowledge in architecture education. Procedia – Social and Behavioural Science, 174, 1309 -1312. https://doi.org/10.1016/j.sbspro.2015.01.752 Dilek, I. B., & Noor, S. (2015). Learning approaches of successful students and factors affecting their learning approaches.

Education and Science, 40(17), 193-216. https://doi.org/10.15390/EB.2015.4214 Entwistle, N. (1988). Motivational factors in students’ approaches to learning. Learning strategies and learning style. New York: Plenum Press.

Howie, P., & Bagnell, R. (2013). A critique of the deep and surface approaches to learning model and teaching.

Journal of Higher Education, 18(4), 389-400. https://doi.org/10.1080/13562517.2012.733689 Hussin, F., Hamed, S., & Md-Jam, S. (2017). Approaches to learning of engineering students: Deep or surface? International Academic Research Journal of Social Science, 3(1), 122-127.

Kamath, A., Rao, R. R., Shenoy, P. J., & Ullal, S. D. (2018). Approaches to learning and academic performance in pharmacology among second-year undergraduate medical students.

Journal of Medical Science, 28, 4-23. https://doi.org/10.15448/1980-6108.2018 Karagiannopoulou, E., & Milienos, F. S. (2015). Testing two path models to explore relationships between students’ experiences of the teaching-learning environment, approaches to learning and academic achievement. International Journal of Experimental Educational Psychology, 35(1), 26-52. https://doi.org/10.1080/01443410.2014.895800 Keithia, L., Wilson, R., Smart, M. J., & Robert, W. (2011).Gender differences in approaches to learning in first year psychology students. British Journal of Educational Psychology, 66(1), 59-71. https://doi.org/10.1111/j.2044-8279.1996.tb01176.x Khan, A. A., Asimiran, S., Kadir, S. A., Alias, S. N., Atta, B., Bularafa, B. A., & Rehman, M. U. (2020). International Journal of Learning, Teaching and Educational Research, 19(10), 233247. https://doi.org/10.26803/ijlter.19.10.13 Ladan, M. A., Balarabe, F., Sani, D. K., Musa, H. A., & Salihu A. A. (2014). Learning approaches as predictors of academic performance of undergraduate students in Ahmadu Bello University, Zaria. Journal of Nursing and Health Science, 3(3), 45-50. https://doi.org/10.9790/1959-03344550 Leung, M., Li, J., Fang, Z., & Lu, M. (2015). Learning approaches of construction engineering students: A comparative study between Honk-Kong and Mainland China. Journal of Education in the Built Environment, 1(1), 112-131. https://doi.org/10.11120/jebe.2006.01010112 Lizzio, A., Wilson, K., & Simons, R. (2002). University students’ perceptions of the learning environment and academic outcomes: implications for theory and practice. Studies in Higher Education, 27(1), 27-52. https://doi.org/10.1080/03075070120099359 Lochmiller, C. R., & Lester, J. N. (2017). An introduction to research connecting methods to practices.

London. Sage Publications Inc. Moh’d, T., Nor, A., Othman, A., Hamid, M. Y., Ibrahim, N. A., Ismail, N. A., Hamimi, A., & Tharim, A. (2015). The influence of learning approach on students’ academic achievement: A study on UiTM (Perak) Students. https://files.eric.ed.gov/fulltext/EJ1238844.pdf Moore, L. (2015). The relationship between approaches to learning and assessment outcomes in undergraduate optometry students [MA Dissertation, Dublin Institute of Technology]. https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1033&context=ltcdis Noori, A. Q., Said, H. F., Nor, M., & Ghani, F. A. (2020). The relationship between university lecturers’ behavior and students’ motivation. Universal Journal of Educational Research, 8(11C), 15-22. https://doi.org/10.13189/ujer.2020.082303

http://ijlter.org/index.php/ijlter

149

Öhrstedt, M., & Lindfors, P. (2019). First-semester students’ capacity to predict academic achievement as related to approaches to learning. Journal of Further and Higher Education, 43(10), 1420-1432. https://doi.org/10.1080/0309877X.2018.1490950 Oluwatayo, A., Aderonmu, P., & Aduwo, B. (2015). Architecture students’ perceptions of their environment and their academic performance. Journal of Learning Environment Research, 18, 129-142.

Pallent, J. (2011). SPSS Survival Manual. Allen University Australia. https://doi.org/10.4236/oa lib.1100973 Puteh, S. M., Abdul Latif, N. E., Mansor, N., Hamid, Y. S., & Halid, Z. (2018). Learning Approaches and Academic Performance. Journal of Social Sciences Research and Academic Research Publishing Group, 6, 746-752.

Rakibul-Islam, G., & Shahriar Shafiq, M. (2016). Surface and deep approaches to learning in higher education. Bangladesh Journal of Educational Research, 2(1), 45-56.

Rosander, P., & Martin, B. (2012). The unique contribution of learning approaches to academic performance, after controlling for IQ and personality: Are there gender differences?

Journal of Learning and Individual Differences, 22(6), 820-826. https://doi.org/10.1016/j.lindif.2012.05.011 Salamonsun, Y., Weaver, R., Chang, S., Koch, J., Bhathal, R., Khoo, C., & Wilson, I. (2013).

Learning approaches as predictors of academic performance in first year health and science students. Nurse Education Today, 33, 729–733. https://doi.org/10.1016/j.nedt.2010.10.019 Sedgwick, P. (2014). Cross sectional studies: advantages and disadvantages. BMJ, 348:g2276. https://doi.org/10.1136/bmj.g2276 Shaik, S. A., Almarzuqi, A., Almogheer, R., Alharbi, O., Jalal, A., &Alorainy, M. (2017).

Assessing Saudi medical students learning approach using the revised two-factor study process questionnaire. International Journal of Medical Education, 8, 292–296. https://doi.org/10.5116/ijme.5974.7a06 Song, J. (2018). Elements in Mol-based College English learning environment- based on Biggs' 3P Model. Advances in social science, education and information research, 89, 5-14.

Tee-Meng, Y., Kannaki, F. D., Narayansany, S., M. Kamala, P., Manickam, S. J., & Leong, K. (2016).

Stimulating deep learning using active learning techniques, Malaysian. Online Journal of Educational Science, 4(6), 6-29. https://files.eric.ed.gov/fulltext/EJ1106447.pdf

http://ijlter.org/index.php/ijlter

150

APPENDIX I SECTION A: Demographic characteristics Please tick as appropriate Name of University…………………………………………………………………..

Gender: Male …….

Female…….

Yours CGPA: (0.00-1.49)____ (1.50-2.49)____ (2.50-3.49)_____ (3.50-4.49)_____ (4.50-5.00)______

SECTION B: STUDENTS’ LEARNING APPROACHES Please choose the letter most appropriate to you: A – Never or only rarely true of me (1) B – Sometimes true of me (2) C – True of me about half the time (3) D –Frequently true of me (4) E-- Always or almost always true of me (5) S/N ITEMS A 1 I find that sometimes studying gives me a feeling of deep personal satisfaction.

2 I find that I have to do enough work on a topic so that I can perform my own conclusions before I am satisfied.

3 My aim is to pass the course while doing as little work as possible.

4 I only study seriously what’s given out in class or in the course outlines.

5 I feel that virtually any topic can be highly interesting once I get into it.

6 I find the most new topics interesting and often spend extra time trying to obtain more information about them.

7 I do not find my course very interesting so I keep my work to the minimum.

8 I learn some things by rote, going over and over them until I know them by heart even if I do not understand them.

9 I find that studying academics topics can at times be as exciting as a good novel or movie.

10 11

I test myself on important topics until I understand them completely.

I find I can get by in most assessments by memorizing key sections rather than trying to understand them.

http://ijlter.org/index.php/ijlter

B

C

D

E

151

12 13 14

15

16

17 18 19 20

I generally restrict my study to what is specifically set as I think it is unnecessary to do anything extra.

I work hard at my studies because I find the material interesting.

I spend a lot of my free time finding out more about interesting topics which have been discussed in different classes.

I find it is not helpful to study topics in depth. It confuses and wastes time, when all you need is passing acquaintance with topics.

I believe that lecturers shouldn’t expect students to spend significant amounts of time studying material everyone knows won’t be examined.

I come to most classes with questions in mind that I want answering.

I make a point of looking at most of the suggested readings that go with the lectures.

I see no point in learning material which is not likely to be in the examination.

I find the best way to pass examinations is to try to remember answers to likely questions.

Scoring is the following cyclical order: Deep Motive, 2. Deep Strategy,3. Surface Motive, 4. Surface Strategy 5. etc.

Deep Approach Score:

∑All Deep Motive scores + all Deep Strategy sores

Surface Approach Score:

∑All Surface Motive scores + all Surface Strategy scores.

http://ijlter.org/index.php/ijlter

152

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 152-173, February 2021 https://doi.org/10.26803/ijlter.20.2.9

Blended Learning in Rural Primary ESL Classroom: Do or Don’t Melanie Jerry Sekolah Kebangsaan Dit, Debak, Sarawak http://orcid.org/0000-0002-6960-5994 Melor Md Yunus Universiti Kebangsaan Malaysia, Bangi, Selangor http://orcid.org/0000-0001-7504-7143

Abstract. The emergence of the Fourth Industrial Revolution (4IR) had led to discussions of the constructive relationship between the integration of technology and teaching and learning to meet society's needs in the innovative era. However, despite the current technology advancement, rural schools' circumstances propelled teachers to shy away from integrating technology in their instruction. Therefore, this paper presented a research to examine primary school teachers' experiences and views on the application of blended learning in their English language instruction using a mixed-method explanatory design.

A survey questionnaire was employed to collect data from 86 teachers from schools in the Betong district, while a semi-structured interview was designed and done with five teachers. Data from the questionnaire were analysed descriptively, while the interview was analysed thematically. The results illustrated that teachers lacked exposure and adequate knowledge of blended learning. It was deduced that teachers had mixed reactions towards blended learning, varying based on their experiences. Teachers emphasised that the lack of facilities and technological constraints, time, limited exposure and skills in ICT use, and teacher readiness were the main challenges they encountered. This study may serve as a building block towards the broad dissemination of blended learning among teachers and serious consideration and effective solutions for the problems faced in its adoption, particularly in rural schools.

Keywords: blended learning; English teaching and learning; educational technology; teachers experiences; Integration of technology

1. Introduction Recently, due to the COVID-19 pandemic outbreak, “the global disruption of pre-existing curriculum resulted in the reinvention of educational practices aimed at minimising the learning losses” (Polushkina & Tareva, 2021, p. 38). ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

153

Such a fact could be witnessed across nations, and the Malaysian context is not an exception. Accordingly, the Malaysian government had taken the measure to pose Movement Control Order (MCO) throughout the country, which affected society's norms, including the schooling period. During MCO's duration, teachers across Malaysia were instructed to conduct online classes with their students using Google Classroom or any digital modalities accessible and convenient for both the teachers and students. This led to the discussions over how online learning can be optimised primarily when it pertains to those in the rural demographic setting. As Fitriani et al. (2021) said, teachers and students may enjoy the benefits and face the challenges imposed by online teaching and learning activities.

The merge of traditional face-to-face interaction with technology-mediated learning known as blended learning conveyed the impression of an ideal approach for education in the current digitally governed world. As elucidated

by Kim (2007), both traditional and online instruction's strengths and flaws can be regulated by combining the potentials of the two into blended learning. In the same vein, Hariharasudan and Kot (2018) corroborated the constructive relationship between the integration of technology and teaching and learning to meet society's needs in the innovative era. The dynamics of blended learning should cater to the ever-changing perspectives on how a modern classroom should serve as a learning space and what it can offer. The incorporation of technology in the myriad scope of education, including English language learning, conveyed the impression that it could be a catalyst to an effective education system in the debate of preparing our youth for the demands of the future world. Razali (2016) stated that evolving technologies had a major influence on educational development.

Despite the initiatives taken by the Malaysian Ministry of Education (MoE) to facilitate English language teaching and learning in the country, there is a minimal significant effect on students' proficiency, particularly in rural schools.

English is commonly viewed as a difficult subject by students in this context, which can be observed based on their classroom performances and assessments.

Undoubtedly, the English curriculum revision had considered students of diverse learning settings (urban, suburban and rural) and allowed teachers to adopt and adapt their lessons accordingly. However, inevitable issues and concerns such as inadequate facilities and infrastructure, limited digital accessibility, unfitting learning materials and resources, and insufficient technological knowledge and skills in both educators and learners were in existence.

Studies on blended learning in the context of tertiary education were widely explored, but there is a need to widen the research in the context of primary levels of education, as well as in English as a Second Language (ESL) setting, particularly in rural schools in Malaysia. Therefore, the constructs in this research explored teachers' experiences in implementing blended learning in terms of its influences, successes and challenges. Administrators and other decision-makers would strategize implementation protocols that augmented

http://ijlter.org/index.php/ijlter

154

optimistic teacher expectations and blended learning pedagogy implementation through learning about their perspectives on the approach's essence in their teaching (Raymond, 2019). This is in line with the researcher’s objective to observe the deliberation of blended learning as an avenue for the teaching and learning of English in rural classrooms based on these experiences. From there, a prospect to educate teachers, including other chief stakeholders, on the approach can be initiated to optimise English language instruction effectiveness.

2. Literature Review 2.1. Technology Acceptance Model Access did not necessarily instigate utilisation (Murphy, 2019). The TAM (technology acceptance model) was used in this research to see whether elementary school teachers were adamant about incorporating blended learning (BL). TAM was created by Davis (1989) as a more accurate method of forecasting technology acceptance and use. The original model had two key constructs: perceived ease of use (PEU) and perceived utility (PU), which were used to predict an individual's behavioural intention to implement technology, which ultimately contributed to actual use. As discovered by Al-Azawei, Parslow and Lundquist (2017), the influences on PEU and PU may indicate the indirect impact on perceived satisfaction (PS) and intention to use (ITU). Based on this, TAM was used to examine whether teachers perceived blended learning as a relevant and useful instructional strategy and review how difficult it is to develop blended learning opportunities. The aim of this study was to look at the perspectives, causes, and challenges that primary school teachers face while using blended learning instructional methods. To meet these aims, computer self-efficacy, prior online learning experience, and teaching beliefs were the three variables in correspondence to TAM constructs to gather information on teacher’s perceptions based on their experiences, factors and challenges posed.

2.2. Teacher Perceptions of Blended Learning Teacher acceptance could be defined as the teacher's willingness to explore and understand how a contemporary method, strategy or technique fits best for the betterment of their teaching and learning. It could reflect their knowledge and skills advancement in the subject matter, to develop professionally and become true teachers of current times. Teacher acceptance is an important element for the adoption of emerging technological tools and innovations. Thus, research into which variables influenced the technology perspectives of teachers leading to their intention to use them or not to use them was significant (Murphy 2019).

This implies that teacher acceptance can be determined based on the diverse features of BL that could either increase or hamper their inclination to adopt and use the approach. Gough et al. (2017) supported the notion of teacher acceptance as they believed that teachers' satisfaction and acceptance of emerging technologies, processes, and how they fit into their teaching were imperative for acceptance and subsequent usage.

2.3. Influences and Successes of Blended Learning Sorbie (2015) outlined teachers' perceptions towards the approach, which indicated a direct relationship between the influences of blended learning on

http://ijlter.org/index.php/ijlter

155

teaching and learning and the successes of its implementation. As teachers became aware of the potential of blended learning in elevating teaching practice and instruction, they could see the relevance in adopting and aligning the approach in suitable ways for their curriculum and students. In this line of thought, it should be noted that becoming aware of a pedagogical need for change is the first step in the blended learning implementation process (Bruggeman et al., 2021). Because teachers had the first-hand contact with students as the front liners in the education setting and the main implementers of educational policies, thus, it was crucial that teachers took proactive measures to be well-informed of the educational transformations to ensure that their teaching styles stayed relevant and they continued to develop professionally.

With adequate knowledge and skills, they would be able to guide students and build competent individuals who could thrive through the era of digitalisation.

The need to improve the acquisition of blended knowledge and digital skills by teachers was associated with the provision of quality education and the promotion of versatility for teachers (Jachin & Usagawa, 2017). Ultimately, teaching values had a strong impact on the educational decisions of the teacher.

Cleveland-Innes and Wilton (2018) described blended learning as a platform for instruction and learning experiences by merging traditional face-to-face and online learning. In the blended approach, a teacher took on the role of a facilitator as students were expected to direct the pace and timing of their learning. This is in line with the current CEFR-aligned English language curriculum, whereby student autonomy in learning was emphasised.

Padmadewi, Artini & Agustini (2020) advocated the importance of autonomous learning to equip students in order to survive in the 21st century. Apart from developing students’ ICT skills, blended learning could assist in developing teachers’ digital skills by allowing them to use devices, technological-based tools and resources in its implementation. Consequently, learning was made easy as information can be accessed online anywhere and at any time along with the device to use and good internet connection (Hamouda, 2018; Wahab, Zain & Yunus, 2018). In regard to face-to-face instruction, time-saving was discovered as an advantage of blended learning (Sorbie, 2015; Oweis, 2018). When teachers were adept at using the blended approach, they could organise their time and manage lesson-related work more effectively. The cost-effectiveness of blended learning was also highlighted, as agreed by Oweis (2018) and Xu et al. (2020).

The achievements recorded in previous blended learning studies demonstrated a clear indicator of the positive effects of blended learning on students' achievement and performance across the four major skills of the English language (Huang, 2019; Challob, Nadzrah & Hafizah, 2016; Ghazidadeh & Fatemipour, 2017; Krishnan & Yunus, 2019; Shih et al., 2015; Said et al., 2013).

Students with blended learning experiences showed improvement and developed proficiency at a higher rate as compared to their peers who did not.

Furthermore, a boost in students’ motivation, participation and engagement during English lessons could be observed (Oweis, 2018; Albiladi & Alshareef, 2019; Krishnan &Yunus, 2019; Mustapha et al., 2010). The positive attitudes could be attributed to the integration of technology in teaching and learning, as it steered away from the common classroom environment with added fun and

http://ijlter.org/index.php/ijlter

156

interactive elements. Another significant impact of blended learning on English teaching and learning was the increase in engagement and cooperation among students with their peers and teachers. In this respect, Majid and Stapa (2017) stated that blended learning helped to establish positive relationships within the learning space. It is noteworthy to mention that the success of the approach depended on the rapport between students and teachers (Krishnan & Yunus, 2019; Kintu, Zhu & Kagambe, 2017).

Blended learning also has a positive effect on the learning environment and experience. The approach allowed for personalisation and students' interests to take place (Krishnan &Yunus, 2019; Huang, 2019). In other terms, lessons were designed to cater to various learning preferences and styles. Students are able to enjoy the process through technology-mediated learning in blended lessons (Ying & Yang, 2016). Blended learning also offered the opportunity to have direct experience in using digital devices and tools. Moreover, scaffold learning was promoted through the blended approach, where constant communication and feedback exchanges occured to guide the less proficient students (Majid & Stapa, 2017). Hence, blended learning can transform the learning atmosphere into a more meaningful experience and conducive environment. Additionally, Wahab et al. (2017) depicted blended learning as a solid example of studentcentred learning that could produce independent and life-long learners, as they were given the prospect of managing their own learning.

2.4. Challenges Faced by Teachers in Blended Learning Implementation The challenges in blended learning implementation provided sound justifications as to why a blended learning endeavour was ineffective and why teachers resisted technology infusion in teaching and learning, leading towards their disinclination to adopt the system. Blended learning required satisfactory technological facilities and resources for online learning. Johnson et al. (2016) elucidated that they still find it difficult to implement modern education technology seamlessly and efficiently, even though teachers typically embraced and appreciated the advantages of educational technologies.

Oweis (2018) investigated the effects of blended learning on tertiary level students' achievement and motivation to learn English. They found several barriers that impeded blended learning implementation. These barriers were observed to have an effect on both learners and teachers. As the use of technological tools was the essence of blended learning, preparation and availability of such resources were vital in order to ensure its smooth operation of the instruction. However, it was discovered that the technical resources were inadequate and internet connectivity was poor, apart from the apparent high maintenance cost of the facilities. Furthermore, teachers commented that blended learning instruction preparation and evaluation took a large amount of time, which posed time constraints to the teachers. Another barrier pinpointed in the study was the concern over plagiarism regarding students' work and the credibility of materials accessed from the Internet.

Conversely, Hamouda (2018) found contrasting challenges to blended learning implementation. The previous research, that considered student-teacher

http://ijlter.org/index.php/ijlter

157

interaction as a vital factor to blended learning success, reported that lack of face-to-face interactions and social isolation played a part to impede a good blended learning experience. Due to the limited interaction and communication, students faced difficulty in following instructions given by the teachers apart.

From the opinions and feelings towards the blended approach, students had the tendency to cheat in their tasks and were not interested in attending classes, be it face-to-face or online, as they saw it as a waste of time. Students' lack of technological skills and great need for support also contributed to their indifference towards blended learning instruction. Similar to Oweis' (2018) findings, slow connectivity was an obstruction for the ease of blended learning employment. On the other hand, teachers expressed dissatisfaction as there was a decline in teacher's efficiency due to pressure and workload, which led to unorganised blended learning materials.

Parallel to Oweis (2018) and Hamouda (2018), Albiladi and Alshareef (2019) offered a comprehensive explication on the stumbling blocks of blended learning implementation, wherein the digital divide and technology challenges were significant hindrances in blended learning adoption. As clarified by the researchers, the digital divide referred to the ICT available to individuals or societies of the different socioeconomic spectrum. As for teachers, they needed support in terms of technological and pedagogical aspects, including instructional teaching. They also had difficulties in managing materials that were culturally appropriate and familiar for students, setting attainable goals and objectives for students and having insufficient time to conduct the lesson and complete their work. Moreover, outside-of-classroom challenges were a matter for teachers to monitor consistently. The researchers also identified that finding the balance between using new technology innovation and producing cost-effective results was strenuous for teachers. Other challenges consisted of the amount of interaction in both face-to-face and online learning modes, and the types of blended learning activities students participatef in and how their teachers can facilitate them.

In addition, lack of understanding of blended learning hindered blended learning adoption by teachers (Shebansky, 2018). Likewise, the lack of training led to uncertainty and doubts in implementing BL in their instruction. This is in synchronization with Johnson et al.'s (2016) proposition, who suggested that teachers would be incapable of using technologies to their full potential without effective and continuous professional development, which in turn would lead to interminable citation of inadequate professional development as a major hindrance to technology integration. Xu et al. (2020) stated that insufficient highquality English instructors were the cause of reluctance in blended learning adoption. Besides, the search and selection process for relevant materials were considered tedious, while the material development was perceived as timeconsuming. The lack of technical support, as a recurring barrier, was a substantial reason for the hampered adoption of blended learning. It was also deduced that high time commitment impeded the willingness to implement blended learning.

http://ijlter.org/index.php/ijlter

158

Hence, this research hoped to gain richer insights into teachers' experiences blended in English language teaching and learning, mainly in rural Sarawak.

Such an endeavour would contribute to the body of knowledge as there were limited studies on teacher blended experiences in their teaching and learning of English, especially at the primary school level in Malaysia. Therefore, this research aimed to explore the teachers’ experiences and challenges in the implementation of blended learning in primary ESL classrooms by answering the following research questions: RQ1: What are the teachers’ experiences in the implementation of blended learning in the teaching and learning of English?

RQ2: What are the challenges faced by teachers in the implementation of blended learning in primary ESL classrooms?

3. Methodology This research employed a mixed-method design in an explanatory sequential structure. The triangulation and integration of both quantitative and qualitative data analysis were observed throughout the research process to authenticate the gathered data from both means of data collection used in this research work. The conceptual framework for this study was as follows: Technology Acceptance Model (Davis, 1085) WEBLEI (Chang & Fisher, 2009)

Blended learning in rural primary ESL classroom Teacher Experiences

Larsen (2012)

Factors Challenges Figure 1. Conceptual Framework

3.1. Participants Purposive sampling was used to select samples of the study from a population of 108 primary school English teachers. It was done based on the Krejcie and Morgan’s (1970) table. A total of 86 English teachers teaching in primary schools under the Betong District Education Office (PPD Betong) participated in the survey. Following the COVID-19 outbreak leading to the closures of schools nationwide, teachers had been conducting online classes. They were encouraged by the Pejabat Pendidikan Daerah (PPD) to participate in online workshops, webinars, and meetings with experts and high-performing teachers to up-skill themselves, particularly in managing online and blended learning. This was dependent on the teacher’s will and determination to do so. The next step included screening and determining teachers who were willing to participate in

http://ijlter.org/index.php/ijlter

159

the interview. Five teachers were interviewed voluntarily, who responded in the survey that they were willing to be interviewed.

3.2. Research Instruments Two research instruments were used to collect data for this research: a questionnaire and interview. The study instrument was replicated from Larsen (2012). The questionnaire was divided into a pre-questionnaire and Web-Based Learning Environment Inventory (WEBLEI) by Chandra and Fisher (2009). The quantitative data were collected using a questionnaire, which was administered via Google Form. The answers to these questions gave insight into formulating questions for the interview. A background questionnaire was mandatory conducted in order to get an accurate and intricate view of teacher participants.

In the survey, teachers were asked to fill in a set of preliminary questions to view their teaching experiences and how familiar they were with ICT use and blended learning. Regarding the reliability of WEBLEI, the questionnaire had been tested twice for slightly different applications in Chang and Fisher (2003) and Chandra and Fisher (2009). The Cronbach alpha coefficient for the four scales, namely 'Emancipatory activities', 'Co-participatory activities', 'Qualia' and 'Information structure and design' in the survey ranged from 0.78 to 0.86, and the fifth scale (facilitation scale) showed a very high level of reliability at α = .902 (Larsen, 2012) An individual semi-structured interview which was adopted from Sorbie (2015) was carried out after collecting data from the survey. The interviews were done using Google Meet and Zoom applications due to meeting restrictions and adaptation to the new normal following the COVID-19 pandemic outbreak. Each interview lasted from 30 to 45 minutes in length, for those interviewees who were comfortable and approved of the time spent. The participants were asked open-ended questions created by the researcher, followed by probes. The broad nature of the interview questions encouraged teachers to expand on their experiences with blended learning and how it had affected them as teachers and their students.

3.3. Research Procedure There were several steps taken by the researcher prior to conducting the research: Approval was sought from the Malaysian Ministry of Education (MoE) by applying through the Education Research Application System 2.0 (eRAS 2.0). i.

Jabatan Pendidikan Negeri Sarawak (JPNS) was notified of the approval. ii.

The approval was forwarded to PPD Betong to get the final consent by the Head of PPD to proceed. iii.

A letter addressed to school headmasters was issued for them to be notified. The letter contained the Google Form link to the survey and the survey details. iv.

A letter that was addressed to respondents who agreed to be interviewed was issued. The preferred meeting platform and time were discussed and decided between the researcher and respondents.

http://ijlter.org/index.php/ijlter

160

The research data collection duration took four weeks, whereby both the survey and interview were to be completed within a two-week time frame. This gave them ample time to complete the survey and prepare themselves for the interview. Once the data were collected, they were analysed quantitatively and qualitatively. Data from the survey were coded and entered as 1 (Strongly Disagree), 2 (Disagree), 3 (Neither Agree nor Disagree), 4 (Agree), and 5 (Strongly Agree) using SPSS. Statistical measurements such as frequency and percentage were determined. The interviews were recorded, transcribed, and tabulated. Transcriptions were compared, allowing for categories and themes to emerge through the six-step thematic coding (Braun & Clarke, 2006).

4. Findings 4.1. Data from the Survey Table 1. Teachers’ Experiences in the Implementation of Blended Learning in the Teaching and Learning of English Item

1.

2.

3.

4.

5.

6.

7.

8.

After getting the BL training, I felt pedagogically ready to teach English.

I received the BL pedagogical support I needed during the teaching and learning of English.

I had enough influence on English content and activities.

There was a good balance between online and classroom activities.

The online and classroom activities integrated well.

I made an effort to integrate classroom and lab activities with each other.

I felt technically prepared to teach English.

I received the technical support I needed during the teaching and learning of English.

http://ijlter.org/index.php/ijlter

Strongly Disagree (%)

Somewhat Disagree (%)

Agree (%)

Strongly Agree (%)

11.6 (10)

Neither Agree or Disagree (%) 37.2 (32)

0

48.8(42)

2.3 (2)

0

23.3 (20)

50.0 (43)

26.7 (23)

0

0

7.0 (6)

40.7 (35)

44.2 (38)

8.1 (7)

0

22.1 (19)

37.2 (32)

29.1 (25)

11.6 (10)

2.3 (2)

19.8 (17)

46.5 (40)

24.4 (21)

7.0 (6)

0

16.3 (14)

39.5 (34)

36.0 (31)

8.1 (7)

2.3 (2)

7.0 (6)

37.2 (32)

39.5 (34)

14.0 (12)

0

15.1 (13)

45.3 (39)

34.9 (30)

4.7 (4)

161

9.

Using BL did not make English more demanding to teach.

10. My teaching style matches well with BL. 11. The online activities worked well.

12. The classroom activities worked well.

0

7.0 (6)

47.7 (41)

36.0 (31)

9.3 (8)

0

14.0 (12)

46.5 (40)

19.8 (17)

19.8 (17)

0

23.3 (20)

46.5 (40)

24.4 (21)

5.8 (5)

0

7.0 (6)

39.5 (34)

40.7 (35)

12.8 (11)

The results can be summarised by saying that neutral responses were given to half of the items in this section of the questionnaire, accounting to six out of 12 items (Items 2, 5, 8, 9, 10, 11). Even so, 6 out of 12 items demonstrated positive feedback from the participants, in which they either agreed or strongly agreed to the statements. All 12 items received negative responses amounting from 2.3% at a minimum and up to 23.3% at a maximum, to which their responses were either to disagree or strongly disagree.

Table 2. Challenges in Implementing Blended Learning Item

Difficult (%)

Somewhat Difficult (%)

3.5 (3)

13. Getting technical support was…

14. Managing the online activities was…

15. Managing the classroom activities was…

16. Integrating the online and classroom activities was…

Somewhat Easy (%)

Strongly Easy (%)

22.1 (19)

Neither Easy or Difficult (%) 55.8 (48)

12.8 (11)

5.8 (5)

8.1 (7)

31.4 (27)

33.7 (29)

36.7 (23)

0

0

4.7 (4)

37.2 (32)

48.8 (42)

9.3 (8)

2.3 (2)

22.1 (19)

44.2 (38)

29.1 (25)

2.3 (2)

Similar to the findings gathered from the previous part of the questionnaire, respondents once again showed mixed responses towards the statements on blended learning challenges. They were mostly neutral towards item 13 and contributed higher percentage to a negative response, indicating that it was difficult to get technical support. This could be attributed to the lack of technical experts amongst them. Item 16 also mostly gained neutral responses, signifying the respondents' doubt towards combining the two modes of classroom activities. They showed a strong positive reaction towards item 15, in which they indicated the simplicity and familiarity in managing classroom activities. There was a clear picture of how the respondents felt about managing the two types of activities. While they reacted positively towards classroom activities, they responded discouragingly towards item 14 (managing online activities).

http://ijlter.org/index.php/ijlter

162

4.2. Data from Semi-Structured Interview 4.2.1. Teachers’ Experiences in the Implementation of Blended Learning in the Teaching and Learning of English Table 3. Teacher Perceptions

Sub-themes

Selected Interview Excerpts "…allows me to diversify my teaching strategies…motivate, discipline, self-direct and manage my pupils' time wisely."

Teacher Perceptions

Development of teacher’s knowledge and skills

"…allows me to be more technology savvy by employing an appropriate amount of IT to improve my teaching practice." "…deliver contents in various and interesting ways." “…helps a lot in enhancing my assessment style…” “…select appropriate materials to be used” “…enjoy learning with the presence of all those technological tools…”

Use of technology and gadgets

“…can use laptops in learning…” “…flexible enough to allow pupils to have access to internet…”

“Teachers and students are also exposed with advanced collaboration tools and technology during online discussions, quick messages and feedback from teachers and students.”

Cost-effectiveness

"…more cost-effective compared to that of physical learning" “…activities can be conducted online which saves a lot of money”

Easy access to information

“…could get a lot of examples online…could access any platform…” “…high quality online educational websites and applications…” “…get the information directly from the internet and…can immediately share the ideas” “…can easily and quickly access everything as long as we have internet connection” “Content, materials and references can be accessed any time.”

http://ijlter.org/index.php/ijlter

163

There are two main themes discussed in this section, namely (i) Teacher Perceptions and (ii) Successes in Blended Learning. The first recorded sub-theme was the development of teacher's knowledge and skills. The teachers' feedback gave a description of how teachers viewed blended learning implementation as a chance for them to upgrade professionally by broadening their knowledge on the latest teaching pedagogy and methodology, and the necessity to possess commendable ICT skills in order to use technology effectively and smoothly.

This could be demonstrated by the responses given by the teachers such as "…allowed me to be more technology savvy by employing an appropriate amount of IT to improve my teaching practice". It was made known during the interviews that blended learning promoted the use of technology and gadgets. Students are inclined to be more engaged and enjoy learning more as technology offered interactive features and colourful visuals, which led to teacher satisfaction upon seeing students enjoying the lessons. On an important note, the interactive nature of the technology allowed for active collaboration and immediate feedback.

Another sub-theme considered was the cost-effectiveness of blended learning.

Online learning could help to reduce the cost of material preparation. Instead of spending money to make posters and print worksheets, teachers can opt for the use of technology to deliver their lesson and run their classroom activities.

YouTube videos, interactive PowerPoint slides, and online quiz applications such as Quizizz and Kahoot! were among the tools teachers commonly used. A respondent stated that using blended learning is "…more cost-effective compared to that of physical learning”. Easy access to information was also a theme revealed from the interviews. There are unlimited resources to English language learning, especially now when technology integration in education continued to grow. An example pertaining to this theme is that “Students can explore information easily on their own”.

Table 4. Successes of Blended Learning

Increase in participation, engagement and motivation to learn English Successes of Blended Learning

Enhancement of Learning Environment and Experience

http://ijlter.org/index.php/ijlter

“…more engaged to the assessment…” "…motivate them to participate actively in the classroom." “…paid full attention as much as their interest towards the learning” “attract learners’ attention and interest towards their learning” “…more motivated to learn and compete healthily…” “…helps to increase my pupils’ motivation and interest in learning” “…motivated to learn everyday as they have fun during the lesson” “…allows pupils to enjoy themselves…” “provides learners more opportunities to learn…” “…interesting to learn…not familiar with all those technology materials and online media” “…nature of BL is multisensory and

164

Increase in students’ language achievement in English language

Interaction and Collaboration among Students and Teachers

Autonomous and Selfpaced Learning

interesting for pupils…”

“It makes my lesson and teaching/learning environment more interactive and attractive than the traditional way.” “…learn better through visuals or pictures” “…motivation & confidence from the students themselves when voicing out their ideas whenever they find something interesting on the web” “Lesson can be delivered in various fun ways virtually.” “…understand the learning content easily…deepens their understanding…” “…enhance learners’ understanding” “indirectly being exposed to a lot of learning techniques that can help … my students to comprehend the lesson better…” “…help students to understand the contents of the lesson better as they can easily and quickly access to the unknown knowledge…” “…broadens classroom discussion…” “…improves learners’ collaboration” “…increasing interaction with peers” “…ample of opportunities to interact and to express ideas…” “…compete not only with their friends in the classroom…with other pupils all around Malaysia…around the world” “…more interactive as the students are eager to share the information that they get from the internet to the class…” “…allows pupils to move at their own pace” “…online learning environment promises learners a place where they have controlled over their own learning” “…learn at their own comfortable and healthy pace” “Students can move through content and tasks at their own pace.”

The first theme that surfaced frequently was the increase in participation, engagement and motivation to learn English. Some examples of that illustrated how blended learning could improve students’ motivation were: “…more motivated to learn and compete healthily…”, and “…helped to increase my pupils’ motivation and interest in learning”. A respondent also mentioned that her students were“…more engaged to the assessment…” when it was done online. This implied that students were able to concentrate on the task better through online means of assessment as compared to written ones. Also, blended learning can attract students’ attention towards learning English as illustrated by the comment that it “attracted learners’ attention and interest towards their learning”.

http://ijlter.org/index.php/ijlter

165

Blended learning can enhance learning environment and experience. Online learning did not only spice up students learning through fun and meaningful media, but also gave them the opportunity to use gadgets and devices. As shared by a respondent, it has been expressed that her students “learn better through visuals or pictures.”In parallel, students found security and comfort in their learning environment, leading them to be active and involved during the activities. By way of illustration, the interviewees highlighted that blended learning “provided learners more opportunities to learn…” and “…the nature of BL was multisensory and interesting for pupils…”.

Blended learning had a positive effect on students’ language achievement in the English language. In this concern, teachers agreed that blended learning could enhance students’ understanding in English language learning, in which they stated that students “…understood the learning content easily…, and deepened their understanding…”. A teacher added that blended learning led her to discover new methods, strategies and techniques in English language teaching, in which she said that it “indirectly exposed to a lot of learning techniques that can help … my students to comprehend the lesson better…”. Another prominent theme highlighted was autonomous and self-paced learning. Teachers agreed that blended learning encouraged students to learn at their own pace and timing. Some examples of feedback indicating this were the fact that “…online learning environment promised learners a place where they had control over their own learning” and “…learned at their own comfortable and healthy pace.” Additionally, "interaction and collaboration among students and teachers" is another theme recognised, whereby blended learning allowed them to remain in contact to discuss and collaborate on their learning tasks. This was congruent with teachers’ feedback statements saying that “Blended learning helped to connect teachers with students without physically being present”, and “increased interaction with peers”.

4.2.2. Challenges Faced by the Teachers in Implementing Blended Learning in Rural Primary ESL Classrooms Four challenges were identified based on teachers’ feedback and responses: (i) inadequate facilities and technological constraints, (ii) time consumption, (iii) lack of exposure and skills in ICT use, and (iv) teacher readiness. Table 5 summarized these challenges, while table 6 covered some responses gathered from the interviewees in respect the challenges being highlighted.

Table 5. Challenges Faced by Teachers in the Implementation of Bended Learning Challenges Inadequate facilities and technological constraints

Time-consuming and increased workload Lack of exposure and skills towards the use of ICT

http://ijlter.org/index.php/ijlter

Teacher’s Concerns i. poor internet connection ii. inadequacy of facilities iii. expensive maintenance fee iv. lack of technical experts i. need more time to arrange the activities and select tasks appropriate for the two modes of learning ii. a decline in teacher's efficiency i. inability to perform tasks and participate actively among students ii. inability to conduct online classes effectively among teachers

166

Teacher readiness

i. inclined to fall back to familiar teaching styles; ones that they have practised for a long time and have seen results with

Table 6. Challenges of Blended Learning with Interviewees’ Statements Challenges

Inadequate facilities and technological constraints

Time consumption

Lack of exposure and skills in ICT use

Teacher Readiness (Preference of traditional face-to-face instruction over online learning)

Selected Interview Excerpts “…have limited numbers of laptop…”

“The school facilities…there are no LCD and computer provided for every classroom” “…poor internet connection in my school area…affects the delivery of the lesson” ““…it depends on the classroom size as well as I’ve mentioned earlier, with the lack of devices to use, my students have to share with one another…” “…expenses to afford them, technical issues…”

“The school facilities and internet connection” “…poor internet or technology access…”

“Using blended learning in my classroom needs me to prepare more for the lesson…” “…need to adapt the content for blended learning…ensure the quality of the instructional processes…takes a lot of time” “…in terms of time management,… still struggling…” “…Time consuming in preparing materials and to get feedback and outcomes…” “…Extend timeframe for students to complete tasks given due to low self-motivation among students…”

“Pupils were not exposed to technology…” “…most of them are not familiar with all those technology materials and online media” “If every student is provided with good technical aspects and IT literacy, blended learning can be easily implemented and arouse students’ interest in learning…meaningful impact on learning experience or they will fall behind due to family economy support and challenges with management in an online setting.” “…comfortable…enriches learning experiences…suits their current learning environment… convenient to learners who…hardly to use technologies throughout their learning” “…blended learning functions as an ‘enhancement tool’ you know…to the traditional face-to-face interaction” “Teacher may struggle with as the workload is significantly higher due to number of classes and subjects to be taught….”

“Teacher prefers face-to-face class to make sure the contents of the lesson are delivered fluently and students achieve the learning objectives during the live time lesson because not all students have strong communication and collaboration skills in technology…Direct supervision…increase motivation”

The main challenge faced was inadequate facilities and technological constraints.

Among the responses received were those related to “limited numbers of laptop…”and “…poor internet connection in my school area” as claimed by the interviewee. The same participant also talked about how the class size affected the smoothness of blended lesson delivery, and this is felt in the following

http://ijlter.org/index.php/ijlter

167

excerpts wherein “…it was depended on the classroom size as well as I’ve mentioned earlier, with the lack of devices to use, my students had to share with one another…”.

The larger the class size, the more facilities such as laptops were needed for students’ individual use. A teacher shared that revisitation of a particular task was bound to happen due to the lack of technological infrastructure and access; as the participant had to “redo the same learning content in a hardcopy (worksheet) for students when it came to poor internet/technology access in order to ensure every student was updated with the content or knowledge.” Bearing in mind that the teachers were working in rural schools, and consequently students may not had “…expenses to afford them…” and there may be difficulties in getting experts to resolve “technical issues”.

Next, the challenge reported by teachers was that implementing blended learning was time consuming. The responses recorded that teachers “…were required to adapt the content for blended learning…ensure the quality of the instructional processes…took a lot of time” and “…in terms of time management, …still struggling…”. However, two respondents mentioned that blended learning helped to save time in lesson delivery. Planning blended lessons may require a lot of time but teachers perceived to reduce delivery or teacher’s presentation time. A response signifying this was that it “…helped save time rather than giving long explanations”. Another challenge posed towards students was the lack of exposure and skills in ICT use, as indicated by three respondents. From their point of view; “If every student was provided with good technical aspects and IT literacy, blended learning can be easily implemented and arouse students’ interest in learning…meaningful impact on learning experience or they will fall behind due to family economy support and challenges with management in an online setting.” This implied that without adequate competency in handling ICT among students, blended learning effectiveness may be affected, and they may find it burdensome, instead of enjoying the learning process.

Furthermore, a challenge experienced by teachers themselves in blended learning was teacher readiness. This was regarding their preference of traditional face-to-face instruction over online learning. A teacher’s comments were“…comfortable…enriches learning experiences…suit their current learning environment… convenient to learners who…hardly to use technologies throughout their learning”. Hence, this showed that the respondent still believed in the significance of having face-to-face interaction. Another teacher justified that“not all students had strong communication and collaboration skills in technology.” The same interviewee stood firm on the perception that “teachers and students in rural areas required the physical presence or face-to-face practices regarding the contents and delivery of lessons to enhance their knowledge and learning behaviour.” Based on the respondent’s experiences on teaching in rural areas, depending on technology alone would be insufficient and irrelevant.

http://ijlter.org/index.php/ijlter

168

5. Discussion 5.1. Teachers’ Experiences in the Implementation of Blended Learning

promotes autonomy and selfpaced learning

enhances learning environment and experience

minimal knowledge on blended learning

open to the idea of applying blended learning

enrich teacher’s knowledge and upgrade their skills

Teachers’ Experiences in the Implementation of Blended Learning

positive influence on students’ participation and engagement

promotes the use of ICT

a convenient means of searching and obtaining information

costeffective

could save time in English instruction

Figure 2. Teacher Experiences in Blended Learning

The major finding obtained from this study was that teachers have minimal knowledge on blended learning. This was illustrated by the mixed reactions teachers had on the application of blended learning in the primary ESL classroom. This could also be attributed to the limited exposure teachers had in blended learning. This finding was in congruence with the study of Ying and Yang (2016), in which academics probably provided vague feedback and expressed the shortcomings of blended learning as an indication of them being unfamiliar with the approach. Despite the lack of knowledge teachers had on blended learning, the answers provided suggested that they were still able to relate it to their teaching practices over the years within the traditional face-toface classroom, and their attempts in using ICT in their teaching and learning practices. On the subject of ‘perceived usefulness’ (PU) in TAM (Davis, 1989), teachers may be unaware and uninformed about the probability of blended learning in enhancing their teaching practice. This could explain their lack of recognition towards blended learning as a potential strategy to help improve English instruction. The mixed views could also be associated with the limited number of teachers who had never applied blended learning method and those who had never gone for blended learning training. As advocated by Bruggeman et al. (2020), for effective blended learning implementation, a detailed understanding and knowledge of the pedagogical concept was needed.

Focusing on the positive influences blended learning had, the results suggested that teachers were open to the idea of applying blended learning in the teaching and learning of English. Based on the questionnaire's feedback, many

http://ijlter.org/index.php/ijlter

169

respondents (40.7%) agreed that blended learning offered a good balance between online and classroom activities. In the words of a respondent, “…Blended learning was a style of learning that provided learners more opportunities to learn in two major ways…”. The same respondent described the traditional faceto-face interaction as ‘convenient to learners’ especially in a rural classroom setting where students hardly had the luxury of owning gadgets and online learning as ‘enhancement tool’ that complements face-to-face instruction. This could be an impetus for teachers to consider the use of blended learning and what affordances it had to offer in their classrooms.

Apart from that, respondents acknowledged that they put in efforts to integrate the two activities (traditional face-to-face and online learning), accounting for a total of 44.1% of those who agreed to the matter despite having the opinion that combining the two modes were difficult. This contrasts with the concept of 'perceived ease of use' (PEU) in TAM (Davis, 1989), which promoted the tendency of individuals to use a particular system when it was free of effort. In this case, the teachers chose to commit through the complexity of managing two modes of activities. This is in line with the findings of Ibrahim and Nat (2019), in which they discovered that the instructors' motivation to use blended learning was unaffected by their academic workload.

Blended learning had positively impacted the education landscape in various ways. The benefits of blended learning were as follows: (i) Enriched teacher’s knowledge and upgraded their skills in many aspects such as pedagogical knowledge and digital competency, (ii) Promoted the use of ICT with set of skills mandatory for the future generation to thrive in the future world, thus making it appropriate and relevant for learners to be exposed and trained to use technology competently, (iii) Saved teaching and learning time as teachers were more organised and prepared for class, (iv) Cost-effective as there were readily available technological tools such as showing lesson content, using PowerPoint slides ,or submitting tasks through email , and/or uploading to a learning management system that did not require the use of paper. (v) A convenient means of searching and obtaining information, (vi) Increased students’ participation and engagement, whereby it can be observed that students participated more actively and showed more interest during the classroom activities, (vii) Offered the prospect of communication and collaboration within and beyond the classroom walls where students can build stronger relationships among themselves, and together with their teachers, (viii) Enhanced learning environment and experience, which in turn helped to improve language achievement, and Promoted autonomy and self-paced learning as students were encouraged to play a more dominant role in their learning for the very reason that learning may take place according to their pace and ability.

http://ijlter.org/index.php/ijlter

170

6. Conclusion and Future Implications To a certain extent, this research's findings indicated that blended learning had a positive influence on the teaching and learning of English in rural primary ESL classrooms. Nonetheless, there were more to discover about blended learning adoption decisions among teachers. Teacher perceptions that implied otherwise were vindicated by the limited exposure and knowledge on blended learning.

The challenges which obstruct teachers from implementing the approach should steer the Ministry of Education, academicians, and education personnel to work closely with teachers and other stakeholders (students and parents). This was essential for the intensive and extensive preparation and planning prior to implementing the blended approach in English instruction. School administrators should take the initiative to create an environment that would foster open communication and collaboration, apart from promoting a respectful and supportive atmosphere to encourage teachers to have constructive discussions to address their concerns. It was critical to conduct training to up skill teachers' digital knowledge and skills. With a clear understanding of the blended learning modus operandi, teachers would be more confident and organised in the planning, delivery, and time management. Furthermore, the availability and usability of facilities in schools should be deliberated along with the consideration to expand internet access predominantly in rural areas. The limitations to this study were that the teachers had different academic qualifications, indicating that some did not have an English major, a small sample size, and that generalisation was obscure to experiences of teachers in other geographical locations with disparate demographics. Some suggestions for future work were put forward: • Diversified learning contexts in Malaysia, focusing on English as a Second Language (ESL) classrooms, • Examining the effect of blended learning implementation with primary school students with different levels of proficiency and age, • Blended experience in the teaching and learning of the four major English language skills in a primary ESL classroom, • Types of blended learning models and appropriate designs for the current CEFR-aligned curriculum in Malaysia, • Types of digital tools or applications preferred or widely used by teachers to teach English language skills and components.

7. References Albiladi, W. S., & Alshareef, K. K. (2019). Blended learning in English teaching and learning: A review of the current literature. Journal of Language Teaching and Research, 10(2), 232-238. https://doi.org/10.17507/jltr.1002.03 Al-Azawei, A., Parslow, P., & Lundquist, K. (2017). Investigating the effect of learning styles in a blended learning system: An extension of the technology acceptance model (TAM). Australasian Journal of Educational Technology, 33(2), 1-23. https://doi.org/10.14742/ajet.2741 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa Bruggeman, B., Tondeur, J., Struyven, K., Pynoo, B., Garone, A., & Vanslambrouck, S. (2020). Experts speaking: Crucial teacher attributes for implementing blended

http://ijlter.org/index.php/ijlter

171

learning in higher education. The Internet and Higher Education, 48. https://doi.org/10.1016/j.iheduc.2020.100772 Challob, A. I., Nadzrah, A. B., & Hafizah, L. (2016). Collaborative blended learning writing environment: Effects on EFL students' writing apprehension and writing performance.

English Language Teaching, 9(6), 229-241. https://doi.org/10.5539/elt.v9n6p229 Chandra, V., & Fisher, D.L. (2009). Students’ perceptions of a blended web-based learning environment. Learning Environment Research, 12, 31-44.

Chang, V., & Fisher, D. L. (2003). The validation and application of a new learning environment instrument for online learning in higher education. In M. S. Khine & D. L. Fisher (Eds.), Technology-rich learning environments: A future perspective (pp. 1-20). Singapore: World Scientific Publishing Co. Pte. Ltd. Cleveland-Innes, M., & Wilton, D. (2018). Guide to blended learning. Canada: Commonwealth of Learning.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Research Centre, 13(3), 319-340. https://doi.org/10.2307/249008 Fitriani, S. S., Weda, S., Samad, I. A., & Ananda, R. (2021). Genre-based visualisation through an online teaching platform: A strategy to engage with academic texts during the Covid-19 outbreak.

XLinguae, 14(1), 270-288. https://doi.org/10.18355/xl.2021.14.01.20 Ghazidadeh, T., & Fatemipour, H. (2017). The effect of blended learning on EFL learners’ reading proficiency. Journal of Language Teaching and Research, 8(3), 606-614. https://doi.org/10.17507/jltr.0803.21 Gough, E., DeJong, D., Grundmeyer, T., & Baron, M. (2017). K-12 teacher perceptions regarding the flipped classroom model for teaching and learning. Journal of Educational Technology Systems, 45(3), 390-423. https://doi.org/10.1177/0047239516658444 Hamilton, V. R. (2018). Blended learning and second language acquisition in the classroom [Master’s thesis, University of Northern Iowa]. https://scholarworks.uni.edu/grp/337 Hamouda, A. (2018). The effect of blended learning on developing Saudi English majors’ writing skills. International Journal of English and Education, 7(2), 40-83.

Hariharasudan, A., & Kot, S. (2018). A scoping review on digital English and education 4.0 for industry 4.0.

Social Sciences, 7(11), 1–13. https://doi.org/10.3390/socsci7110227 Huang, Y. (2019). The study on application of blended learning in college English listening and speaking teaching. Paper presented at the 4th Annual International Conference on Education Science and Education Management. https://doi.org/10.12783/dtssehs/esem2019/29758 Ibrahim, M. M., & Nat, M. (2019). Blended learning motivation model for instructors in higher education institutions. International Journal of Educational Technology in Higher Education, 16(12). https://doi.org/10.1186/s41239-019-0145-2 Jachin, N., & Usagawa, T. (2017). Potential impact of blended learning on Teacher Education in Mongolia.

Creative Education, 8(9), 1481-1494. https://doi.org/10.4236/ce.2017.89104 Johnson, A. M., Jacovina, M. E., Russell, D. E., & Soto, C. M. (2016). Challenges and solutions when using technologies in the classroom. In S. A. Crossley & D. S. McNamara (Eds.), Adaptive educational technologies for literacy instruction (pp. 1329). New York: Taylor & Francis. https://doi.org/10.4324/9781315647500-2

http://ijlter.org/index.php/ijlter

172

Kim, W. (2007). Towards a definition and methodology for blended learning. In J. Fong & F. L. Wang (Eds.), Proceedings of Workshop on Blended Learning (pp. 1- 8).

Pearson.

Kintu, M. J., Zhu, C., & Kagambe, E. (2017). Blended learning effectiveness: The relationship between student characteristics, design features and outcomes.

International Journal of Educational Technology in Higher Education, 14(7). https://doi.org/10.1186/s41239-017-0043-4 Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607-610. https://doi.org/10.1177/001316447003000308 Krishnan, P. D., & Yunus, M. M. (2019). Blended CEFR in enhancing vocabulary among low proficiency students. Arab World English Journal (AWEJ), 5, 141-153. https://doi.org/10.31235/osf.io/ujqma Larsen, L. J. E. (2012). Teacher and student perspectives on a blended learning intensive English program writing course [Doctoral dissertation, Iowa State University].

Murphy, K. (2019). Duquesne scholarship collection perceptions of blended learning in the high school classroom [Doctoral dissertation, Duquesne University]. https://dsc.duq.edu/etd/1864 Mustapha, S. M., Rahman, N. S. N. A., & Yunus, M. M. (2010). Factors influencing classroom participation: A case study of Malaysian undergraduate students.

Procedia Social and Behavioral Sciences, 9, 1079–1084. https://doi.org/10.1016/j.sbspro.2010.12.289 Oweis, T. I. (2018). Effects of using a blended learning method on students’ achievement and motivation to learn English in Jordan: A pilot case study. Hindawi, 2018, 1-7. https://doi.org/10.1155/2018/7425924 Padmadewi, N. N., Artini, L. P., & Agustini, D. A. E. (2020). Promoting autonomous learning and 21st century skills of English education students through empowering their pedagogical skills. Proceedings of the 4th Asian Education Symposium (AES 2019): Advances in Social Science, Education and Humanities Research. https://doi.org/10.2991/assehr.k.200513.024 Polushkina, T. A., & Tareva, E. G. (2021). Developing L2 prosodic competence online: Implications of the emergency remote teaching. XLinguae, 14(1), 38-48. https://doi.org/10.18355/xl.2021.14.01.03 Raymond, S. (2019). High tchool Teacher perceptions of blended learning [Doctoral Dissertation, Walden University]. https://scholarworks.waldenu.edu/dissertations/7725 Razali, N. N. F. M. (2016). The importance and efforts in using technology to improve language teaching and learning and the teacher’s readiness for integrating ICT in the classroom in Malaysian education. Journal of Education and Social Sciences, 5(2), 227–230.

Said, N. E. M., Yunus, M. M., Doring, L. K., Alfian, A., Aqilah, F., & Li, L. K. S. (2013).

Blogging to enhance writing skills: A survey of students' perception and attitude.

Asian Social Science, 9(16), 95–101. https://doi.org/10.5539/ass.v9n16p95 Shebansky, W. (2018). Blended learning adoption in an ESL context: Obstacles and guidelines.

TESL Canada Journal, 35(1), 52-77. https://doi.org/10.18806/tesl.v35i1.1284 Shih, R. C., Lee, C., & Cheng, T. (2015). Effects of English spelling learning experience through a mobile line app for college students. Procedia - Social and Behavioral Sciences, 174, 2634-2638. https://doi.org/10.1016/j.sbspro.2015.01.945 Sorbie, J. (2015). Exploring teacher perceptions of blended learning (Doctoral Dissertation,

http://ijlter.org/index.php/ijlter

173

Walden University] https://scholarworks.waldenu.edu/dissertations/1866 Wahab, N. A., Zain, A. M., & Yunus, M. M. (2018). Exploring the blended learning experience among 21st century language learners. Journal of Language and Communication, 5(1), 136-149.

Xu, D., Glick, D., Rodriguez, F., Cung, B., Li, Q., & Warschauer, M. (2019). Does blended instruction enhance English language learning in developing countries?

Evidence from Mexico. British Journal of Educational Technology, 51(1), 211-227. https://doi.org/10.1111/bjet.12797 Ying, A. N. L., & Yang, I. (2016). Academics and learners’ perceptions on blended learning as a strategic initiative to improve student learning experience [Conference presentation].

9th International Unimas Stem Engineering Conference (ENCON 2016) “Innovative Solutions for Engineering and Technology Challenges, MATEC Web Conf (Vol. 87). https://doi.org/10.1051/matecconf/20178704005

http://ijlter.org/index.php/ijlter

174

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 174-196, February 2021 https://doi.org/10.26803/ijlter.20.2.10

Developing Interaction in ESL Classes: An Investigation of Teacher-Student Interaction of Teacher Trainees in a Sri Lankan University Abdul Majeed Mohamed Navaz South Eastern University of Sri Lanka, Sri Lanka https://orcid.org/0000-0003-2042-652X

Abstract. This study examines the possibility of using of IRF (InitiationResponse-Follow-up) sequence of teacher-student interaction in Sri Lankan ESL (English as a Second Language) classes for developing longer interactional exchanges which are believed to be useful for language development. Usually, in Sri Lankan ESL classes, teachers ask more display questions and a few referential questions. As a result, teacher-student interaction occurs only occasionally and they follow the traditional IRF pattern with an evaluation at the third move. Teachers could develop longer interactional exchanges by giving follow-up questions or prompts at the third move of the IRF sequence so that students respond, elaborate, explain or prolong their responses. This study examines how the teacher trainees on their teaching practice of a TESL degree programme at a university interacted with their students in ESL classes and how they changed their pattern of interaction to sustain more student interaction. Using lecture discourse data as the basis of the analysis, this study evaluated the changes after an intervention that focussed on training the teacher trainees in developing longer interactional episodes. The results revealed that there was only a slight improvement in the way teacher trainees maintained interactions in the lessons after the intervention. Hence, this study enlightens the possibility of utilizing interaction for language development through intensive teacher training.

Keywords: questions; ESL classes; interaction; IRF; teacher trainees

1. Introduction English language proficiency of students at secondary and tertiary levels in Sri Lanka has been a concern for legislators and educational administrators. A tracer study conducted by the University Grants Commission in 2018 on graduate employment reports that only around 27% of the graduates from the Faculty of Arts of the university, where this study was conducted, were employed. Lack of employment has always been connected to lack of language proficiency of the graduates. It is believed that language proficiency of students, ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

175

especially the speaking skills, could be developed by giving opportunities to speak in the ESL classes. In other way, teachers could handle the classroom discourse to provide opportunities for students to speak.

It is generally claimed that students in Asian countries are reluctant to speak in the classroom. A previous study by the researcher (Navaz, 2012) in undergraduate content classes in the same university revealed that students rarely asked or answered questions. To improve the situation, based on the theory of output hypothesis proposed by Swain (1995), it is argued that students should be given opportunities to speak in the class. However, the tendency in Sri Lankan ESL classes at secondary and tertiary levels is that teacher talk occupies most of the class time. Recent peer observation by the researcher of a class conducted by an instructor at the faculty revealed that around 90% of his class time was occupied by teacher talk (Semester II, 2018/2019 academic year, Quality Assurance Process).

While limited interaction in ESL classes could be connected to various reasons, in this study, developing the classroom interaction has been considered as an alternative to developing the language proficiency of students. A few instances of teacher-student interactions in the study context, as exist in several other contexts, followed the traditional three-part exchange, IRF, which consists of a question, known as Initiation (I), followed by an answer, called Response (R) and the last step is a Follow-up by the teacher (F). Teachers initiate teacherstudent interaction by asking a question (Initiation – I) in a one-to-one situation (or sometimes, the first question is directed to the whole class), and the students respond to that (Response – R) question. This is followed by a follow-up (Follow-up – F) that can be a feedback. This pattern was identified by Sinclair and Coulthard (1975) as IRF pattern, whereas Mehan (1979) proposed it as IRE pattern, where the last move was ‘Evaluation’ instead of a ‘Feedback’. In this study, ‘IRF’ generally refers to a ‘Follow-up’ at the last move unless specified as feedback, while ‘IRE’ refers to an evaluation at the end position.

Despite the fact that IRF pattern has been a dominant type of discourse structure in ESL as well as content classes, which is also deemed to be limiting students’ participation in classroom discourse, there are possibilities that teachers could make use of the last move by providing feedback so that they can sustain longer meaningful interactions that are useful for language development of students in ESL classes (Walsh, 2011). That is, teachers should make use of the last move of a three-part exchange to give a follow-up question/comment (e.g. can you explain further?) to students instead of an evaluation (e.g good or OK). The follow-up questions as feedback help clarify learners’ contributions and pronunciation (Chang, 2009) so that the discourse could be prolonged. Recently, the focus has been shifted towards students’ construction of knowledge along with language development in content classes (e.g. Chin, 2007; Yip, 2004). Hence, several other research studies have examined the possibility of extending the teacher-student interaction beyond IRF sequence. For example, Cullen (2002), Lee (2007), Waring (2009), Skinner (2019), etc. have been discussed in the literature review section.

http://ijlter.org/index.php/ijlter

176

In the study context, the opportunities to learn English language for the students are confined to ESL classes only. The students in the faculty receive only a few hours of instruction in teacher-fronted classrooms. Hence, this study unearths the possibilities of developing students’ ability to interact in ESL classes in the faculty where students learn English as a second language while studying their academic studies in their mother tongue, Tamil. On the basis of the argument that teacher-student interaction which involves students in constructive dialogue in ESL classes would help develop language proficiency of the learners (Gibbons, 2003; Gibbons, 2015; Gupta & Lee; 2015; Hall & Verplaetse, 2000; Haneda, 2005; Haneda & Wells, 2010), this study attempts to examine the existing pattern of interaction in ESL classes and the possibilities of developing interaction by manipulating teacher controlled IRF sequences. In the absence of notable previous studies either in Sri Lanka or South East Asia on classroom discourse, this study could be considered important for trying to unearth an important area of classroom discourse which could be useful for developing the language proficiency of the students who are already deemed to be in a disadvantaged position in terms of employment opportunities for which lack of language proficiency is considered to be one of the reasons.

Teachers in Sri Lankan content classes rarely move beyond this IRF pattern of discourse for the reason that they are not aware of the techniques of moving beyond IRF nor are they concerned about the importance of extending the interaction, as was revealed in a previous study by the researcher in content classes (Navaz, 2012). Hence, studies in Sri Lankan ESL classroom discourse have been limited. Of the few studies conducted in Sri Lanka, Sanmuganathan (2017) claimed that most of the time ESL students and teachers use mother tongue to involve in interactions, while Samaranayake (2016) reported that the instructional method used by the ESL teachers did not provide opportunities for developing the speaking skills. His study was conducted in rural schools in Sri Lanka. Karunaratne (2003) also found that the methods used by the teachers in urban schools did not provide opportunities for language learning. Perera (2001) for her doctoral study investigated the role of interaction and use of textbooks for developing language proficiency in secondary level classes. Her study revealed that teacher-student interaction is not taking place at a satisfactory level.

The studies in Sri Lankan context reveal that teacher-student interaction does not take place in the ESL classrooms for various reasons. These studies focused on different factors like the teaching methods or textbooks used, etc. but no studies so far have looked into how the discourse could be handled by teachers to develop teacher-student interaction.

At the backdrop of the argument that the knowledge of IRF pattern and extending the IRF pattern is helpful for developing the interaction in classrooms, this study, involving the selected final year TESL (Teaching English as a Second Language) students on teaching practice (hereinafter referred to as Teacher Trainees -TTs) investigates what types of interaction the teacher trainees develop in ESL classes in the faculty and what impact an intervention focussing on training TTs to develop longer interactional exchanges has on classroom

http://ijlter.org/index.php/ijlter

177

discourse. All the students who follow the TESL degree have to undertake teaching practice as part of the requirements of the course. In addition, this study examines the underlying reasons for the patterns of classroom discourse.

In the absence of studies that deal with classroom interaction with a focus on classroom discourse in tertiary level ESL classes in Sri Lanka, the present study is considered important.

Hence, this study has the following research questions.

1.1 Research questions

RQ1. What types of interactional episodes do the teacher trainees develop before the intervention in the undergraduate ESL classes at the faculty and what are the underlying reasons?

RQ2: What impact does the intervention have on classroom discourse of ESL classes conducted by the TTs and what are the underlying reasons?

The next section describes the literature pertaining to the studies that dealt with classroom interaction in both content and ESL classes and the review looks into how the classroom discourse of traditional IRF/IRE pattern changes into longer meaningful interactional episodes with the use of ‘follow-up’.

2. Literature review 2.1 IRF in classroom discourse This study arises on the premise that teacher-student interaction which involves students in constructive dialogue in ESL classes would help develop language proficiency of the learners. It has long been established that teacher-student interaction is important in creating a suitable learning environment and for learners’ language development (Hall & Verplaetse, 2000). Hence, it helps develop the second language (Gibbons, 2003; Gibbons, 2015; Gupta & Lee; 2015, Haneda, 2005; Haneda & Wells, 2010). When the interaction is a recitation script with one–to–one question and answer sequences between teacher and students, the outcome of language development would be a question. When the ESL or content classroom talk is dialogic with mutual contributions from both students and teacher, it can be a good platform for content and language learning (Swain & Lapkin, 1998). Swain (1995) also argues that students should get opportunities to practise the language to become fluent in the language. She proposed this argument through her ‘output hypothesis’ which regards conversational interaction as an ‘excellent opportunity for developing speaking’ (p. 248).

It is a widely accepted fact that classroom discourse of teacher-students in Sri Lanka as well as in other parts of the world follows the traditional IRF pattern which is also called IRE when the third move becomes an evaluation instead of a feedback (Hall, 1998). The three-part exchange structure or the ‘triadic dialogue’ (I-R-F) (Lemke, 1985; 1990) is the most occurring pattern as well as a popular tool for analyzing the classroom discourse structure. Wells (1993) regards that around 70% of the classroom discourse could be accounted by this IRF pattern.

The IRF sequence, being the general pattern of classroom interaction in ESL as well as content classes, is being criticized as teacher-centred. Recently, attention

http://ijlter.org/index.php/ijlter

178

has been paid to utilize the IRF sequence to enhance student participation.

Despite the fact that the IRF sequence is highly teacher controlled, the discussions are underway on how the teachers could make use the IRF to sustain interaction in classes (Davidson & Edwards-Groves, 2020; Hall & Walsh, 2002; Musumeci, 1996). In the absence of studies connected to ESL classroom discourse in Sri Lanka, this study, along this argument, tries to identify the existing pattern of discourse in undergraduate ESL classes and examines the possibilities of training the teachers to make use of the IRF sequence to develop interaction. In this section, I review the studies that attempted to use the IRF sequence for developing interaction from different contexts.

The general argument is that the IRF pattern of exchanges could be manipulated by the teachers to get more student participation. That is, the three-part structure is an effective tool for guiding the students for learning (Mercer, 1995) on the premises that the last move could be handled by the teacher to bring in more student involvement and participation (Wells, 1993).

2.2 Follow-up at the third move and its benefits Hall (1998) as well as Lee (2007) stated that the variation in the IRF pattern could create different abilities to participate, different learning opportunities, and different outcomes. Two moves become important here; one is the initiation or the questions teachers ask that decide the nature of the flow of interaction. The other one is the third move which is equally important because it decides whether the interaction continues or is brought to a halt (Lee, 2007). The third move could play different functions such as offering evaluation, feedback or follow-up on the students’ second move. In the third move, teachers not only respond to whether the students’ answers are correct, adequate or relevant but also, even for correct answers, teachers could ask students to elaborate, reformulate or defend their answers (ibid).

Lee (2007) analysed the three-turn exchanges of the discourse of college ESL courses produced by international and immigrant students in North American universities to identify how the third turns (F) were used to sustain interaction with students. Lee considers that the teachers should make use of the third move, without limiting to evaluate students’ responses, for different discourse purposes. Some of them could be “estimating what students know and what they do not know, discovering particular identities of their students and their problems, finding and repairing what becomes problematic in the second turns” (ibid: 1226).

The findings of Lee (2007) agree with Wells and Arouz (2006) as well as Nassaji and Wells (2000) who argue that it is important how the teacher handles the last move, the ‘follow-up’, which is believed to have the ability of generating sufficient interaction in the classroom. Their research was based on school level science content classes in Toronto. Nassaji and Wells identified six functional moves that the third turn displays: evaluation, justifications, counter-argument, clarification, meta-talk and action. They advise that teachers should avoid giving evaluation in the follow-up move that would possibly suppress students’ participation; instead, they demonstrated, teachers could request, in the follow-

http://ijlter.org/index.php/ijlter

179

up move, for “justifications, connections, or counter arguments and allow students to self-select in making their contributions” (p. 401).

In a study conducted in a community English programme in the United States, Waring (2009) demonstrated using the discourse collected from a single student during the homework feedback session that how moving away from the traditional IRF sequence could bring in more in-depth discussions. Similarly, Wong and Waring (2009) suggest that giving a feedback like ‘very good’ is not conducive for learning because the feedback brings the students’ contribution to a halt. Instead, they advise the teachers to use ‘pursuit’ questions (e.g. why do you say that?) so that students could engage in the activity more. Here Wong and Waring use the name feedback for evaluation and pursuit questions for what others call follow-up questions.

Liu (2008) differentiated the discourse between two teachers and demonstrated the opportunity to develop language through the handling of IRF moves, especially the follow-up move. In his study conducted in Chinese language classrooms, one of the teachers valued student contributions by providing positive acknowledgements and reformulations. Liu claimed that this has promoted students’ involvement and created effective language learning opportunities (ibid). Similarly, Hellermann (2003) showed that the effective handling of the ‘F move’ in the IRF pattern had a number of positive consequences for student participation and learning. His study was based on urban high school content classes.

Cullen (2002) considered the teachers’ handling of follow-up move in the IRF sequence in a secondary school EFL classroom in Tanzania. He invented two kinds of F moves: Evaluative and Discoursal. Cullen regards that the latter type of F move could be useful for language development as it reformulates and elaborates on student contributions, and derives further initiating moves from them. This study reveals that teacher could handle the last move in the IRF sequence to prolong the teacher–student interaction and obtain more student participation.

Skinner (2019) in a study conducted with nine teacher trainees on their teaching practice for their TESOL degree programme found that when the trainee teachers asked display questions for which answer is already known, the classroom discussions were confined to IRF sequence. As a result of the study, Skinner suggests that teachers should use referential questions which require genuine answer from the students.

As Boyd and Markarian (2015) claim, to develop longer interaction, the teachers can ask for clarification or elaborations in the third move instead of an evaluation which usually brings the interactional exchange to a halt. This could occur through requests for clarifications and elaborations in the third move (Edwards-Groves, 2014).

One way the feedback could develop language development is it helps clarify learners’ contributions and pronunciation. This could occur through repairs producing reformulations (Chang, 2009) that are corrections made by learners

http://ijlter.org/index.php/ijlter

180

themselves when teacher makes them realize that their utterance is not understood by the teacher. It is already established that negotiated interaction between the learners and the teacher can enhance comprehension of the input and in turn lead to language development (Long, 1985). Further, Van Lier (2001) claims that when the initiation-response-feedback exchange moves beyond mere recitation and display, it helps scaffolding instruction, develops cognitive structures in the zone of proximal development, or assists learners to express themselves with maximum clarity.

Despite the different opinions, researchers generally agree that when the third move of the IRF is a follow-up instead of evaluation, the teacher could develop longer meaningful interactions in the classroom. That is, the IRF pattern could be used for better student contribution by handling the last move ‘F” effectively (e.g. Hall, 1998; Lee, 2007; Nassaji & Wells, 2000). In this way, teachers could maintain dialogic interaction in their classrooms too.

As indicated in Lemke’s triadic dialogue, usually in classrooms a teacher’s question receives a response from the students, and the teacher evaluates the student’s response to end the exchange. This could be followed by another initiation by the teacher. Even though researchers (e.g. Wood, 1992) have earlier questioned the usefulness of this triadic dialogue, now it has been realised that these triadic dialogues could be used in a fruitful way by giving follow-up (e.g. feedback) instead of an evaluation (E) to make meaningful, connected interactional episodes between teacher and students (e.g. Nassaji & Wells, 2000).

This study also looks into the possibility of extending beyond the IRF pattern by asking follow-up questions at the last move. The conceptual framework of the study is given in Figure 1 below.

Figure 1: Conceptual framework of the study

http://ijlter.org/index.php/ijlter

181

This conceptual framework explains that classroom interaction can move in two directions, initiated with teachers’ questions. When the teacher gives an evaluation for the student’s response, the interactional exchange is brought to an end. Instead, if the response is followed by a follow-up question, the exchange continues. Giving follow-up, in this study, was facilitated by an intervention.

Hence, the following methodology was used to investigate the classroom discourse and answer the research questions.

3. Methods This study follows a classroom action research method and the data were collected mainly through qualitative methods. Classroom discourse was the basis for data collection. The observed lessons were recorded and some selected lessons were transcribed. In addition, classroom observation, and teacher trainees’ reflections obtained through individual discussions and their written answers for structural questions were used in this study.

3.1 Participants The participants of this study were the final year TESL students who follow the TESL (Teaching English as a Second Language) degree with the Department of English Language Teaching (DELT) at the Faculty of Arts (herein after referred to as faculty). As part of the requirements of the course, the students have to undertake teaching practice at the faculty for 15 weeks in semester I of the fourth year of the degree programme. They are called Teacher Trainees (TTs) in this study. Of the twelve TTs in the academic year 2019/2020, four TTs were selected based on the convenience of the researcher for observation. These teacher trainees are in their early 20s and have studied in their mother tongue, Tamil, up to their GCE A/L1 (General Certificate in Education Advanced Level) and have studied English as a subject in their school from grade 3. Only those students with a good language proficiency are selected to the TESL degree each year. For three years, they have followed different courses in English language, linguistics and methodology in the medium of English. They have learnt the theoretical and practical aspects of teaching during their study.

For the teaching practice, the TTs are assigned to teach classes along with an instructor and observed by a senior lecturer. In this study, the researcher was the observer. Four TTs were selected based on the convenience of the researcher for observation and all of them were female students. In the batch there are eleven female students and only a male student and therefore the sample consisted of only female students. The TTs were assigned to teach a course in Elementary English to the first year Arts students. These first year students were just starting their academic studies in the university after studying English at their primary and secondary levels for 10 years. The students were grouped according to their language proficiency level from weaker to the best groups and these TTs were working with the top four groups out of eight.

1

GCE A/L is the qualifying examination for university entrance

http://ijlter.org/index.php/ijlter

182

3.2 Observation A semester consists of 15 weeks and the observation took place at two stages, the first five weeks, followed by an intervention that lasted for four weeks and again the classes were observed for the last 5 weeks. Each teacher trainee was expected to teach around 30 hours of lessons for the semester, 2 hours a week. The ESL classes were held on Mondays and Tuesdays, and therefore the researcher was able to observe two TTs a week. Altogether four lessons were observed of each TT. In addition, the researcher maintained a field notebook to take down vital activities in the observed classroom for individual discussion with the TTs. 3.3 Recordings and transcriptions All the observed lessons were recorded using smartphones which the TTs carried with them. The first lesson delivered by TTs was not taken into analysis for the reason that students would have behaved differently because of the presence of the researcher. From rest of the lessons, longest ones were selected.

Two of the recorded lessons of each TT, before and after the intervention, one each, were transcribed using usual transcribing conventions by a colleague of the researcher. That is, altogether eight lectures were transcribed and the transcripts were compared with the recordings by the researcher.

The four TTs are identified as TTA, TTM, TTS, TTZ based on the first letter of their names along with TT. In addition, each of their two transcribed lessons were identified as 1 and 2 with their code names. No. 1 refers to the ESL lesson conducted before the intervention, while ‘2’ indicates the lesson after the intervention. For example, TTZ 1 refers to the lesson conducted by TTZ before the intervention and TTZ 2 means the lesson after the intervention.

3.4 Ethics At the faculty, where the study was conducted, the ethical guidelines are yet to be introduced. However, the informed consents were obtained from the four TTs and the researcher explained the purpose of the study to the students in the four groups. In addition, the dean of the faculty was informed about the study.

3.5 Intervention An intervention was administered following the first part of the observation.

The intervention focused on training the TTs to ask different types of questions that are useful for language development, mainly referential, as well as openended and closed questions during lessons. It also focused on interactional patterns that are developed by asking different types of questions. TTs were exposed to the discourse obtained from their own lessons conducted at the beginning (pre-intervention period) and exemplified how they maintained interactions and how they could have developed the interactions better in their lessons. Based on these reflections of their own lessons, changes were suggested.

Hence, this process was participatory since the TTs themselves suggested how they could have progressed in the lessons, despite the fact that there were other constraints.

http://ijlter.org/index.php/ijlter

183

The intervention was held for two hours a week for four weeks, followed by one week ‘no observation week’ for the TTs to get oriented themselves with the intervention and during the last five weeks they were observed again.

3.6 Analysing the transcript The transcripts were analysed for questioning patterns, and interactional exchanges. The focus was on the IRE/F (follow-up) patterns developed before and after the intervention in each lecture. Student-initiated interactions were also checked for their availability but there was not any in any lesson discourse.

In this study, the teacher-student interactional exchanges were the important analytical components. Therefore, in analysing the lecture discourse, the following two components were paid attention. 1. Questions asked by the TTs: they are important because they make the initiation move. 2. Interactional exchanges: they are the key component of analysis. Following Hu and Li (2017) and Hu and Duan (2018), any utterance identified as interrogative, imperative, or declarative which elicited a verbal response was considered a question (Navaz, 2020).

Then at the next stage, those questions that built into interactional exchanges2 or episodes3 were considered irrespective of the length of the exchanges and their numbers were counted. A colleague of the researcher assisted in the identification of questions. These questions that initiated interactions were categorised into two types: display and referential questions. Display questions require answers which are already known to the teacher, while referential questions request information from the students which is not known to the teacher (Brock, 1986, as cited in Navaz, 2020). All the others were put into the category of others. In addition, the TTs were requested to reflect on their own lecture delivery.

3.7 Reflection of TTs The TTs were asked to comment on their own lesson delivery, important activities, etc. The researcher picked the vital points for clarification with the TTs from the field notebook. Some of the things that were clarified were the reasons for teacher questioning and their classroom discourse patterns at each stage of the lesson delivery. That is, their concerns and constraints in making classes interactive were also found out. The reasons for TTs and students’ behaviours in lessons were also focussed on. In addition, the TTs were given some structured questions to write their views. For example, they were asked about the reasons for lack of classroom interaction.

4. Findings In this section, mainly the following research questions are answered: RQ1. What types of interactional episodes do the teacher trainees develop before the intervention in the undergraduate ESL classes at the faculty and what are the underlying reasons?

2

Question-answer-feedback/evaluation is known as an exchange. Usually a teacher question, student response and teacher feedback (e.g. can you explain further) or evaluation (e.g good) 3 One or several exchanges that occur at one point in a lesson make an episode.

http://ijlter.org/index.php/ijlter

184

RQ2: What impact does the intervention have on classroom discourse of ESL classes conducted by the TTs and what are the underlying reasons?

Before answering the first research question, some background details that are connected with those research questions are presented.

4.1 Summary of the lessons The lessons at the pre-intervention stage focused on parts of speech such as nouns, adjectives, adverbs, etc., while at the post-intervention stage they focused on present simple and continuous tenses.

4.2 Identification of lessons transcribed before and after the intervention The table below (Table 1) indicates the lessons transcribed before and after the intervention and they are identified using TTs’ code. Altogether eight lessons were transcribed and taken into analysis. Those lessons are TTA 1, TTA 2, TTM 1 and TTM 2, which are, for example, delivered by TTA and TTM respectively.

Table 1: Identification of lessons transcribed TTs

Transcribed lessons before the intervention

Transcribed lessons after the intervention

TTA TTM TTS TTZ

TTA 1 TTM 1 TTS 1 TTZ 1

TTA 2 TTM 2 TTS 2 TTZ 2

Following this description, the first research question has been answered.

4.3 Types of interactional episodes All kinds of interactional episodes were counted before and after the intervention. A series of interactional exchanges that occurred at a time were counted as a single episode. To say simply, a dialogue between teacher and students at a point of time is considered an episode. An episode can contain either single or a series of exchanges with IRE/IRF sequences. When the teacher returns to monologic talk or asks students to do some activity, the interactional exchange/episode ends. In this study, the interactional exchanges/episodes are called episodes and IRE refers to shorter single episode which has three moves and ends with evaluation, while extended IRF denotes longer episodes with several moves extended by follow-up questions or prompts.

Table 2: Types of interactional episodes and their number before the intervention TTs

No. of IRE interactional episodes

TTA TTM TTS TTZ Total

http://ijlter.org/index.php/ijlter

15 09 17 18 59

No. of extended IRF interactional episodes 00 00 01 02 03

185

The number of interactional episodes found in the analysed lecture discourse of the four TTs are given in Table 2. They were limited in number and altogether there were 62 of them. Further, 59 of the total interactional episodes belonged to the IRE pattern. An example of an IRE sequence is given in Extract 1. The evaluation usually given in the lessons are ‘very good’, ‘good’, ‘OK’, ‘fine’, etc.

Across the lessons, OK was more prevalent. We have already seen that when the evaluation is given in the third move, the interactional episodes have been brought to a stop. This makes a typical IRE episode. Then, the TTs asked another question or got on with explaining the lesson.

Extract 1: Example of IRE pattern of discourse (from TTA’s lesson) T: The park has a big pond. In this sentence, what are the nouns? - I S: Park and pond. - R T: Park and pond. OK. – E [[Transcribing conventions used: T- Teacher, S, S1, S2 … students]]

In addition, on three occasions the extended pattern of IRF sequences were produced by TTS and TTZ. For example, in a lesson conducted by TTS, the TT used follow-up questions to sustain the interaction. She avoided giving evaluation, instead used a follow-up question to continue the interaction, as shown in Extract 2 below.

Extract 2: Example of IRF pattern of discourse (From TTS’s lesson) T: Today we are going to study about TV programs and shows. Do you have a habit of watching TV in your life? Everyone please respond. Do you have a habit of watching television or dramas or movies?

I S1: Yes miss.

R T: Now tell me what are the TV programs that are being telecasted on TV? There are many varieties of programs telecasted there no? Ok fine. Let me explain it.

F– Follow-up T: Have you all understood what is documentary?

I S1: Yes miss.

R T: Have you all understood what is sitcom?

F S: A situational comedy. [[many students answer]] R T: Yeah comedy program. The characters are real life characters or fictional characters?

F S2: Fictional characters.

S T: Do you all like any dramas? or you used to watch any dramas in your life? What is your favourite drama? Have you ever watched any serials?

I [[the dialogue continues…]]

The reasons for generally producing limited number of interactional as well as shorter episodes were revealed by the TTs. One is that the TTs tried to push through the lesson and wanted to cover the whole materials in the lesson within the allocated time. Other one is that they were anxious in the class because the supervisor and an instructor were observing their lessons so they were hesitant to get involved in a dialogue with the students because they feared they might get stuck or use wrong forms of language while getting involved in interaction.

http://ijlter.org/index.php/ijlter

186

TTA said, “we are afraid that we need to finish the lesson on time and have to cover all the contents of the allocated modules. Sometimes, supervisors think that we are idling in the class by asking questions.” The tendency to getting through and moving on with the lesson was reported in the Skinner (2019) where the teacher trainees did not want to prolong the interaction but they wanted to continue with the lesson.

Moreover, the TTs did not have an idea of the pattern of interaction they produced in the lessons. Mostly, the TTs produced IRE pattern because for them interaction was a question-answer-appreciation (evaluation) sequence.

It emerged from the study that another reason for developing IRE pattern could be connected to the types of questions. When TTs initiated the interaction with the display questions, the interaction usually followed the IRE pattern. As shown in Table 3, more than 75% of the questions produced in the lessons are display questions. In Skinner’s study (2019), when the teacher trainees initiated display questions they could not develop longer interactional episodes. Skinner describes, “display questions are less likely to open up interaction than referential questions and as with the IRF move, can be used by teachers to constrain interaction, and, in effect, confine learner contribution” (ibid: p. 4). This situation is similar in this faculty too. The reasons for triadic dialogue pattern of IRE may be because of the display questions asked by the TTs. Table 3: Types of questions before the intervention Types of Questions in the ‘I’ move / TTs Display Referential Others Total

Before the intervention (%) (found in a single lesson of each TT) TTA TTM TTS TTZ 61 (86%) 30 (77%) 97 (79%) 67 (81%) 00 00 03 (02%) 05 (06%) 10 (14%) 09 (23%) 23 (19%) 11 (13%) 71 39 123 83

The TTs, especially TTZ and TTS who produced longer interactional episodes were asked about how they could produce extended sequences of interaction using IRF pattern but they described it as incidental. That is, they did not plan to have longer interactional episodes but it occurred during teaching.

With these details of the pattern of interaction found in the lessons before the intervention and discussing the underlying reasons, mainly for the pattern of discourse, the second research question is answered.

RQ2. What impact does the intervention have on classroom discourse of ESL classes conducted by the TTs and what are the underlying reasons for the patterns of interactions?

In this part of the findings, I compare the situation before and after the intervention under the title “outcome of the intervention.“

http://ijlter.org/index.php/ijlter

187

4.4 Outcome of the intervention As a result of the intervention, two things were envisaged to occur in the lecture discourse of TTs. One is that the TTs needed to ask more number of referential questions and more importantly, they needed to maintain lengthy interactional exchanges that used IRF sequence with follow-up moves.

4.4.1 Types of questions It was obvious from the findings that most of the questions asked by the TTs were display type questions. However, there was a slight increase in the number of referential questions asked by a TT (TTZ) after the intervention, as shown in Table 4. It was shown already that display questions warrant an answer which is already known to the questioner, usually the teacher, while referential questions request information from the respondents which is not known to the teacher.

Teo (2016) found that teachers asked more display type questions in a preuniversity programme in Singapore that prevent them from developing longer interaction. Teo (2016) further argued that display questions seem to extract predetermined answers that the teacher is looking for, thereby constricting the space for reflection, question and dialogue, leading to confined interactional exchanges. Cullen (2002) and Skinner (2019) exemplified that display questions are less likely to open up interaction than referential questions.

It was further revealed that only after the intervention the TTs had got a clear understanding of the fact that referential questions are useful for language development compared to the display questions. Moreover, the reflection of their own lessons during the intervention stage was more useful for them to get an awareness of their own questioning pattern and the desirable types of questions.

Table 4: Types of questions before and after the intervention (parenthesis indicates %) Types of Questions in the ‘I’ move // TTs

Before intervention (%) TTA TTM TTS

Display

61 (86%) 00

30 (77%) 00

10 (14%) 71

09 (23%) 39

Referential Others Total

97 (79%) 03 (02%) 23 (19%) 123

TTZ

After intervention (%) TTA TTM TTS

TTZ

67 (81%) 05 (06%) 11 (13%) 83

60 (92%) 01 (02%) 04 (06%) 65

90 (84%) 11 (10%) 06 (06%) 107

28 (82%) 00 06 (18%) 34

90 (80%) 04 (04%) 18 (16%) 112

Of the questions asked by the TTs, some were one word questions/prompts like next?, then?, why?, etc. In addition, there were some classroom procedural questions too. E.g. Did you attend the last class? and clarification requests. A clarification request is a direct form of question or request to clarify the preceding utterance of the other speaker (e.g. ‘Beg your pardon’ in TTS1; ‘Miss, repeat the answers’ in TTZ1), while a confirmation check is made to ensure that what is heard by the other speaker is correct. The latter is usually made by repetition of all or part of the other's preceding utterance through rising

http://ijlter.org/index.php/ijlter

188

intonation questions (Long, 1981). These types of questions are put under the category ‘others.’

4.4.2 Number of interactional episodes developed All kinds of interactional episodes that occurred before and after the intervention are given in Table 5.

Table 5: Number of interactional episodes developed before and after the intervention TTs

TTA TTM TTS TTZ

No. of interactional episodes before the intervention 15 09 18 20

No. of interactional episodes after the intervention 19 11 23 21

According to this table, the number of interactional episodes, developed in the observed two lessons, are limited in number. However, the higher number of interactional episodes does not mean that the interactional episodes were longer and useful for language development. The pattern of interactional exchanges only would tell us if the episodes are useful for language development. In the post intervention lessons conducted by TTS and TTZ, many of the questions were not answered by the students and they passed as rhetorical questions which expect no answer from the students or were answered by the TTs themselves. The reasons for not answering questions by the students could vary.

TTs after asking questions, had to give enough wait-time, repeat the questions or nominate a student to answer the questions. These measures made the students answer the questions to a certain extent. In the observed classes, TTZ had used some of these strategies to make the students answer. In addition, students’ shyness, anxiety or lack of language proficiency would also prevent students from answering the questions. Further discussion on this is given under the discussion section.

4.4.3 Patterns of interaction Two patterns of interactions were looked into, shorter IRE pattern with evaluation and or an extended IRF pattern with follow-up moves. Earlier, Table 2 indicated the patterns of interaction before the intervention and they are given here again for a comparison. As shown in Table 6 below, even after the intervention most of the interactional exchanges followed the IRE pattern. That means the TTs usually gave an evaluation at the third move and brought the interaction to an end.

Table 6: Patterns of interaction before and after the intervention TTs

TTA

Before the intervention No. of IRE No. of extended IRF interactional (follow-up) episodes before interactional the intervention episodes before the intervention 15 00

http://ijlter.org/index.php/ijlter

After the intervention No. of IRE No. of extended IRF interactional (follow-up) episodes after interactional the intervention episodes after the intervention 19 00

189

TTM TTS TTZ

09 17 18

00 01 02

11 21 14

00 02 07

Here, the expectation was that after the intervention there would be more extended IRF patterns and lower IRE patterns. But, there wasn’t much improvement in the way TTs handled interactions. Except TTZ and TTS, the other TTs did not use the extended pattern. In the case of TTA and TTM, none of the episodes followed the IRF pattern. TTZ showed some improvement in her delivery. An example from the lesson TTZ 2 (Extract 3) is given below where TTZ and students involved in a dialogue which used longer students’ responses.

Extract 3: From TTZ 2 (T: TT; S1, S2 – students) T: Can anyone say (tell) the summary of today's lesson? What did we study today? I S1: About used to.

R T: yeah we studied about ‘used to’ and ‘didn't use to’ ok? When do we use used to? E + I S1: We use ‘used to’ to describe something happened in the past. But not now.

R T: Yeah very good. To describe something which happened regularly in the past but it doesn’t happen now. ok then the structure? How do we use used to (in) affirmative sentence? E + F (follow-up) S2: Sub+ used to+ didn’t use to + verb.

R T: Ok. Actually, the subject + used to + verb. Ok then how do we use used to in negative sentences? can you all tell me the structure?

E + I S2: Sub+ didn’t use to + verb. R T: Very good. Then?

E + F S1: Question form is, did +sub+ used to+ verb. R T: Very good ok fine. E

In this extract, TTZ by asking follow-up questions in addition to evaluation continues the interaction. She initiates connected questions or uses a prompt like ‘then’ to signal the students to continue their utterances. When she wants to stop the interaction she uses ‘ok’ and ‘fine’, which are treated to be evaluations and the exchange is brought to an end. On many occasions, the TTs asked display questions and the interactional exchanges were shorter, where the third moves are not available, as shown below in extract 4.

Extract 4: from TTA 1 T: What is the last month of the year?

S1: December.

T: What is the month after June?

S2: July.

I R I R

Similarly, in other lessons from TTA1 and TTS 2 students gave short answers as given in extract 5. In these examples, TTS asks display type questions too for which students tend to answer in a single word.

Extract 5: from TTS 2 T: S: T:

What is there on the screen?

Headline news.

What is news?

http://ijlter.org/index.php/ijlter

I R I

190

S: T: S: T: S: T: S:

Information.

R Do you have the habit of reading newspapers?

Yes miss?

R Do you like to read newspaper?

I No. R Who likes news?

I I like.

R

I

It is also alleged that when the teachers use open-ended questions, the interaction would be longer. For example, in another lesson when TTZ asked an open-ended question which required a lengthy answer, student tried to use longer utterances. In TTZ’s move, an evaluation for the previous utterance precedes an initiation, as shown in Extract 6.

Extract 6: from TTZ 2 TTZ: Can you tell me all the pictures in order?

S: First one is the welcome poster. Second one is about the music…….

I R

These findings indicate that when the TTs used referential or open-ended questions, the students attempted to answer in longer utterances, whereas when the TTs used display questions, students answered in a single word. Brock (1986) and Musumeci (1996) consider that it is important to have open-ended questions to generate a greater interaction, and Morell (2004) in a similar vein argues that referential questions, which are open-ended, bring more contributions from students.

4.4.4 The underlying reasons for the pattern of interaction Several reasons were discussed and of them some important ones are given below such as TTs’ lack of knowledge on question types and interactional patterns, students’ cooperation, students’ language proficiency, TTs’ time constraint and anxiety to develop interaction. Some of these reasons were discussed earlier in this section.

The TTs while acknowledging the fact that they lacked awareness on question types and the extended interactional exchanges are limited in their lessons, stressed that students’ cooperation was important.

TTZ mentioned: “The main thing we have to understand in order to maintain the interaction is to identify their capacity and needs” “Sometimes it's our unawareness and in few instances students’ support. Also, in some situations we don't know how to interact quickly to (with) the students.”

TTS reflected that she was happy with the way the interaction occurred in her class. She stated: “in my class I am trying my level best to maintain a good rapport in order to develop interaction with the students.” Nevertheless, there were plenty of IRE patterns across her lectures. When she was indicated it, she agreed that she needed to improve the pattern of interaction in the future.

http://ijlter.org/index.php/ijlter

191

TTA’s claim was that the students did not cooperate to maintain interaction in lessons. She mentioned: “I felt that even though we named students in asking questions, they keep silent (quiet) and also in some instances they acted like they didn’t hear the teacher.”

TTZ who had a few longer interactional episodes in her discourse mentioned that only after the intervention she realized the importance of maintaining longer interactional exchanges. She too had the similar opinion about the students that they did not cooperate. She states: “In my point of view, I felt that student's support is more important for the successful interaction in the classroom. As I asked feedback from my students at last, they told me that the lecture was interesting. At the same time some of the students remained silent when asked some questions. Only some of the students who regularly talk answer the questions. According to it, it’s very hard for us to conduct the lectures in an interactive manner.”

As a whole, the TTs are of the opinion that students’ support is important for developing interaction, albeit they agreed that they have to put more effort in the future with more awareness of the importance of developing interaction and the know-how of doing it.

5. Discussion The discussion section should be considered with the limitations of the study when it comes to interpreting the results of the study. The participants in the study were teacher trainees in their final year of their study and should be treated as novice teachers and only four of them were involved in the study. In addition, only two lessons of each TT were taken into analysis, one before the intervention and one after the intervention.

The findings of this study confirmed that before the intervention the interactional exchanges developed were similar to the recitation scripts (Tharp & Gallimore, 1988) in which one-to-one question, answer, evaluation or feedback patterns existed. More importantly longer interactional exchanges were absent throughout the majority of the lesson discourse. The TTs asked more display questions and most of the time students answered in one or two word utterances. Skinner (2019) explains that IRF pattern (or the three-part exchange) is preferred by teacher trainees because they think it is a safer option that it gives less interactional space to the real learners, so that trainees need not answer any unexpected questions from learners. Similarly, in the findings of this study it was mentioned that the TTs did not spend much time in developing interactions so that they wanted to teach the whole content of the lesson. These findings were similar to Skinner’s (2019) findings.

It could be noticed that after the intervention there was a slight change in the way interactional exchanges were handled, especially two TTs have shown slight improvement in maintaining longer interactional exchanges where students’ responses were also longer. Based on the reflection of the TTs, we can maintain that the presence of longer interactional exchanges was the results of the intervention even though individual abilities of the TTs could also influence

http://ijlter.org/index.php/ijlter

192

as discussed below. The TTs also confirmed that only after the intervention they came to know the importance of maintaining interactional exchanges, especially longer and meaningful interactional exchanges. However, substantive evidence should be collected via future research with more samples to prove that intervention has an impact on the classroom discourse pattern. It is also noteworthy that longer interactional episodes would not always help language development.

The reasons for the present pattern of interaction were revealed by the TTs that they were unaware of the importance of interaction, and also they reported that students’ cooperation with classroom interaction was poor. In addition, TTs, being novice teachers, were unable to control and develop interaction successfully. The language proficiency of the students and their usual reluctance in classroom participation including shyness would also hamper their active participation in involving in teacher-student interaction in ESL classes.

Generally, students in the Asian region are shy to talk in classes irrespective of their language proficiency. A previous study by the researcher in tertiary level content classes (Navaz, 2012; 2013) revealed that students felt asking or answering questions in classes was not appropriate. They would rather ask any questions in private if they had.

Of the four TTs, only one (TTZ) had shown a slight improvement in maintaining the pattern of interaction towards extended patterns. The reasons for this nature could be connected to the individual abilities of the TTs too. TTZ was a high performer in the class and her language ability and the confidence she handled the classes with seem to be relatively higher. Previous studies have established that the success of developing longer interactional episodes depends on the personality of the teachers and their language proficiency. For example, Samaranayake (2016) found that the teachers’ ability to develop and sustain oral activities influenced the classroom interaction. Similarly, teachers’ language proficiency is important for effective teaching. Even though TTZ whose language proficiency is higher compared to others had developed longer interactional exchanges using referential and open-ended questions, further evidence may be required through future research studies to make the claim that teachers’ personality and language proficiency influence the classroom interaction.

This study also informs us that teachers should be aware of the importance of asking questions in ESL classes and gain the expertise needed for asking openended referential questions and developing longer interactional episodes. More importantly, the teacher should be trained to handle to sustaining longer interactional episodes in classes. Smith and Higgins (2006) explain that it may not be the quality of the original question which determines subsequent students’ participation, rather the quality of the teachers’ reaction to pupils’ responses (the F move in the I–R–F exchange). This view endorses the view expressed by Nassaji and Wells (2000) who argued that if the teacher used a follow-up move instead of evaluation, the discourse develops further. The same view was previously expressed by Wells (1999) that follow-up move can give opportunity to extend student’s answer.

http://ijlter.org/index.php/ijlter

193

Despite its limitations, this study could be considered a precursor to the future studies that could deal with classroom discourse as a platform to develop the language proficiency of the students. In the absence of studies in South East Asia on using classroom discourse for developing language proficiency, the findings of this study can be important for teacher training programmes including the TESL degree programmes. Any such training programmes should incorporate the knowledge and practice of how to handle classroom discourse for language development successfully into their curricula.

6. Conclusion This study investigated how the teacher trainees could develop longer and meaningful interactional episodes in their ESL lessons by manipulating the IRF sequence of interaction. Just like ESL classes in other contexts and countries, Sri Lankan tertiary level ESL classes also follow the IRE pattern of classroom discourse. Hence, an attempt made to bring changes into the way the teacher trainees developed and maintained interactional exchanges showed a slight change as a result of the intervention. This sheds light for future development in classroom discourse in terms of interactivity by training the ESL teachers to gain more experience and exposure to the techniques of controlling classroom discourse. This was found by Lee (2007) and Liu (2008) that teachers could be trained to use the IRF sequence productively in ESL classes. The TTs involved in the study were going to begin teaching as their career and were exposed to a limited amount of intervention and therefore drastic changes cannot be expected. Hence, they need to gain more experience to be able to control the classroom discourse. The findings of the study inform that more attention should be paid by ESL teachers for enhancing their knowledge of asking questions and maintaining interactions in ESL classes. In addition, future research is needed to find how the experienced teachers control the classroom discourse to sustain interaction. The proposed study should focus on providing more in-depth training to teachers in handling classroom discourse for language development and eliciting more data from lesson discourse.

7. References Brock, C. A. (1986). The effects of referential questions on ESL classroom discourse.

TESOL Quarterly, 20(1), 47–58. https://doi.org/10.2307/3586388 Boyd, M. P., & Markarian, W. C. (2015). Dialogic teaching and dialogic stance: Moving beyond interactional form. Research in the Teaching of English, 49(3), 272–295. https://doi:10.1080/09500782.2011.597861 Chang, F. (2009). How teacher questioning behaviours assist and affect language teaching and learning in EFL classrooms in Taiwan [PhD Thesis, University of Warwick]. http://webcat.warwick.ac.uk/record=b2282533~S9 Chin, C. (2007). Teacher questioning in science classrooms: Approaches that stimulate productive thinking. Journal of Research in Science Teaching, 44(6), 815–843. https://doi.org/10.1002/tea.20171 Cullen, R. (2002). Supportive teacher talk: the importance of the F move. ELT Journal, 56(2), 117–127. https://doi.org/10.1093/elt/56.2.117 Davidson, C., & Edwards-Groves, C. (2020). Producing and closing down multipleresponse sequences during whole-class talk in an early years classroom.

Language and Education, 34(3), 193-211. https://doi.org/10.1080/09500782.2019.1689995

http://ijlter.org/index.php/ijlter

194

Edwards-Groves, C. (2014). Talk moves: A Repertoire of practices for productive classroom dialogue. PETAA Paper, 195, 1–12.

Gibbons, P. (2003). Mediating language learning: Teacher interactions with ESL students in a content-based classroom. TESOL Quarterly, 37(2), 247–273. https://doi.org/ 10.2307/3588504 Gibbons, P. (2015). Scaffolding language, scaffolding learning: Teaching English Language Learners in the mainstream classroom (2nd ed.). Portsmouth, NH: Heinemann.

Gupta, A., & Lee, G. L. (2015). Dialogic teaching approach with English language learners to enhance oral language skills in the content areas. International Journal of Language and Linguistics, 2(5), 10-17. https://digitalcommons.odu.edu/teaching-learning_fac_pubs/19 Hall, J. K. (1998). Differential teacher attention to learner utterances: The construction of different opportunities for learning in the IRF. Linguistics and Education, 9(3), 287–311. https://doi.org/10.1016/S0898-5898(97)90003-6 Hall, J. K., & Verplaetse, L. S. (Eds.) (2000). Second and foreign language learning through classroom interaction. Mahwah, NJ: Lawrence Erlbaum.

Hall, J. K., & Walsh, M. (2002). Teacher-student interaction and language learning.

Annual Review of Applied Linguistics, 22, 186–203. https://doi.org/10.1017/S0267190502000107 Haneda, M. (2005). Some functions of triadic dialogue in the classroom: Examples from L2 research. The Canadian Modern Language Review/La Revue canadienne des langues vivantes, 62(2), 313–333. https://doi.org/10.1353/cml.2006.0003 Haneda, M., & Wells, G. (2010). Learning science through dialogic inquiry: Is it beneficial for English-as-additional-language students? International Journal of Educational Research, 49(1), 10–21. https://doi.org/10.1016/j.ijer.2010.05.003 Hellermann, J. (2003). The interactive work of prosody in the IRF exchange: teacher repetition in feedback moves. Language in Society, 32(1), 79—104. https://doi.org/ 10.10170S0047404503321049 Hu, G., & Duan, Y. (2018). Questioning and responding in the classroom: a crossdisciplinary study of the effects of instructional mediums in academic subjects at a Chinese university. International Journal of Bilingual Education and Bilingualism, 22(3), 303-321. https://doi.org/10.1080/13670050.2018.1493084 Hu, G., & Li, X. (2017). Asking and answering questions in English-medium instruction classrooms: What is the cognitive and syntactic complexity level? In J. Zhao & L. Q. Dixon (Eds.), English medium instruction in Chinese universities: Perspectives, discourse and evaluation. (pp. 184-203). New York: Routledge.

Karunaratne, I. M. (2003, November 28–30). Teaching English in urban Sri Lanka: Some pedagogical issues. 9th International Conference on Sri Lankan Studies, Matara, Sri Lanka.

Lee, Y. (2007). Third turn position in teacher talk: Contingency and the work of teaching.

Journal of Pragmatics, 39(6), 1204–1230. https://doi.org/10.1016/j.pragma.2006.11.003 Lemke, J. L. (1985). Using language in the classroom. Deakin University Press.

Lemke, J. L. (1990). Talking science: language, learning and values. Norwood, NJ: Ablex.

Liu, Y. (2008). Teacher–student talk in Singapore Chinese language classrooms: A case study of initiation/response/follow-up (IRF). Asia Pacific Journal of Education, 28(1), 87-102, https://doi.org/10.1080/02188790701850071 Long, M. H. (1981) Input, interaction, and second language acquisition. In H. Winitz, (Ed.) Native Language and Foreign Language Acquisition (pp. 259-278). New York Academy of Sciences.

http://ijlter.org/index.php/ijlter

195

Long, M. H. (1985). Input and second language acquisition theory. In S. M. Gass & C. Madden, (Eds.) Input in Second Language Acquisition (pp. 377-393). Rowley, MA: Newbury House.

Mehan, H. (1979). Learning lessons: Social organization in the classroom. Cambridge, MA: Harvard University Press.

Mercer, N. (1995). The guided construction of knowledge: Talk amongst teachers and learners.

Clevedon: Multilingual Matters.

Morell, T. (2004). Interactive lecture discourse for university EFL students. English for Specific Purposes, 23(3), 325–338. https://doi.org/10.1016/s0889-4906(03)00029-2 Musumeci, D. (1996). Teacher-learner negotiation in content-based instruction: Communication at cross-purposes? Applied Linguistics, 17(3), 286–325. https://doi.org/10.1093/applin/17.3.286 Nassaji, H., & Wells, G. (2000). What’s the use of triadic dialogue. Applied Linguistics, 21(3), 376–406. https://doi.org/10.1093/applin/21.3.376 Navaz, A. M. M. (2012). lecturer-student interaction in English-medium science lectures: an investigation of perceptions and practice at a Sri Lankan university where English is a second language [PhD Thesis, University of Nottingham]. http://eprints.nottingham.ac.uk/12995/ Navaz, A. M. M. (2013). A study on perception of lecturer – student interaction in english medium science lectures. Novitas – ROYAL (Research on Youth and Language), 7(2), 117-136. https://novitasroyal.org/download/a-study-onperception-of-lecturer-student-interaction-in-english-medium-science-lectures/ Navaz, A. M. M. (2020). Developing a framework for understanding lecturer-student interaction in English-medium undergraduate lectures in Sri Lanka: First step towards dialogic teaching. International Journal of English Linguistics, 10(6), 208229. https://doi.org/10.5539/ijel.v10n6p395 Perera, M. E. (2001). The role of classroom interaction in second language acquisition in Sri Lanka [Doctoral thesis, University of Wollongong, Australia]. http://ro.uow.edu.au/theses/1805 Quality Assurance Process - Peer Observation. (2020). Observation of ESL classes conducted by an Instructor.

Samaranayake, S. W. (2016). Oral Competency of ESL/EFL Learners in Sri Lankan Rural School Context.

SAGE Open, April-June, 1–10. https://doi.org/10.1177/2158244016654202 Sanmuganathan, K. (2017). Historical perspective of English and some pedagogical problems and solutions in teaching English in Sri Lankan schools. International Journal of Scientific and Research Publications, 7(6), 623-629. http://www.ijsrp.org/research-paper-0617/ijsrp-p6678.pdf Sinclair, J. M., & Coulthard, M. (1975). Towards an analysis of discourse: The English used by teachers and pupils. Oxford University Press.

Skinner, B. (2019). ‘Let’s move on’: second language trainee teachers’ talk and its impact on learner interaction.

The language learning journal. https://doi.org/10.1080/09571736.2019.1642371 Smith, H., & Higgins, S. (2006). Opening classroom interaction: the importance of feedback.

Cambridge Journal of Education, 36(4), 485-502. https://doi.org/10.1080/03057640601048357 Swain, M. (1995). Three functions of output in second language learning. In G. Cook & B. Seidlhofer (Eds.) Principle and Practice in Applied Linguistics (125-144). Oxford University Press.

http://ijlter.org/index.php/ijlter

196

Swain, M., & Lapkin, S. (1998). Interaction and second language learning: Two adolescent French immersion students working together. The Modern Language Journal, 82(iii), 320–337. https://doi.org/10.1111/j.1540-4781.1998.tb01209.x Teo, P. (2016). Exploring the dialogic space in teaching: A study of teacher talk in the preuniversity classroom in Singapore. Teaching and Teacher Education. 56, 47-60. https://doi.org/10.1016/j.tate.2016.01.019 Tharp, R. G., & Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and schooling in social context. Cambridge University Press.

University Grants Commission. (2018). Tracer study of graduates in Sri Lankan universities.

Colombo: University Grants Commission.

Van Lier, L. (2001). Constraints and resources in classroom talk: Issues of equality and symmetry. In C. Candlin & N. Mercer (Eds.) English language teaching in its social context: A reader (90-107). Routledge.

Walsh, S. (2011). Exploring classroom discourse language in action. Routledge.

Waring, H. (2009). Moving out of IRF (Initiation-Response-Feedback): A single case analysis.

Language Learning, 59(4), 796–824. https://doi.org/10.1111/lang.2009.59.issue-4 Wells, G. (1993). Reevaluating the IRF sequence: A proposal for the articulation of theories of activity and discourse for the analysis of teaching and learning in the classroom. Linguistics and Education, 5(1), 1–37. https://doi.org/10.1016/S08985898(05)80001-4 Wells, G. (1999). Dialogic inquiry: Towards a sociocultural practice and theory of education.

Cambridge University Press.

Wells, G., & Arauz, R. M. (2006). Dialogue in the classroom. Journal of the Learning Sciences, 15(3), 379–428. https://doi.org/10.1207/s15327809jls1503_3 Wood, D. (1992). Teaching talk. In K. Norman (Ed.), Thinking voices: The work of the national oracy project (203-214). Hodder & Stoughton (for the National Curriculum Council).

Wong, J., & Waring, H. Z. (2009). ‘Very good’ as a teacher response. ELT Journal, 63(3), 195-203. https://doi.org/10.1093/elt/ccn042 Yip, D. Y. (2004). Questioning skills for conceptual change in science instruction. Journal of Biological Education, 38(2), 76–83. https://doi.org/10.1080/00219266.20

http://ijlter.org/index.php/ijlter

197

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 197-217, February 2021 https://doi.org/10.26803/ijlter.20.2.11

Teaching with Relevance: Saudi Students’ Perceptions of a Foundation Course in Communication Skills Philline M. Deraney Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia https://orcid.org/0000-0001-8392-293X

Abstract. This study explores undergraduate students' perceptions of a foundation-year communication skills course at a large public university in Saudi Arabia from the aspects of perceived course effectiveness and relevance using a cross-sectional survey research design. The survey yielded scaled and short-answer responses, which were then analyzed using statistical and thematic analyses. The results revealed that firstyear undergraduate students (n=209) in this study perceive communication skills as important for their future career and in major areas of their lives, and consider being a ’good’ communicator as part of their cultural and religious heritage. Participants further define communication skills as predominantly an oral, information transfer that builds relationships with others from interpersonal and intercultural aspects. The findings also revealed disagreement and uncertainty about the communication course focus and requirement, relevance of course assignments and materials, and language of instruction.

Recommendations for teaching communication skills with relevance in this context include explicit, focused communication instruction, studentcentered practical activities with cultural relevance, and, moving forward, coursework that integrates the communication discipline with the needs of the students’ academic fields. Future research in this area could enhance teaching communication skills in the Saudi context and lead to more relevant instruction that could positively impact students in their professional lives.

Keywords: communication foundation year; Saudi Arabia

skills;

culturally-relevant

teaching;

1. Introduction Human communication in its varied forms is now omnipresent in undergraduate degree programs across the world. Communication has been offered as a social science discipline for decades and more recently as a skillset necessary for the 21st Century that promotes success in higher education study, enhances professionalism, and advances graduates’ careers (Arkoudis & Doughney, 2014; ©Author This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

198

McLean, Shaban & Murdoch-Eaten, 2011; Wyatt & Nunn, 2019). Universities often offer communication skills in relation to other transferable skills that can be adapted to various disciplines and professional contexts (Arkoudis & Kelly, 2016) such as teamwork, problem solving, leadership, and critical thinking.

Communication skills are now commonly integrated throughout program curricula or offered as a stand-alone course or series of courses (Arkoudis & Kelly, 2016; Rüttermann et al., 2017).

In the Kingdom of Saudi Arabia, communication skills are integrated within learning domains in the National Qualifications Framework (Education and Training National Evaluation Commission, 2020) and included as a graduate attribute at several national universities. Yet, as discussed in the literature and common to the teaching of communication, “communication skills have traditionally been situated as marginal to core teaching and learning practices in disciplinary contexts” (Arkoudis & Kelly, 2016, p. 3). Regional studies have also shown that undergraduate and even graduate students lack experience and confidence in communication skills (Alofisan et al., 2016; McLean, Shaban & Murdock-Eaton, 2011). Adding to the issue of confidence, students who are not proficient in English, the language of instruction in the Saudi and regional context in several fields including health, engineering, computer and administrative sciences (Nouraldeen & Elyas, 2014), often struggle with communication skills and intercultural competencies in the language (Karolak & Guta, 2015; McLean, Shaban, & Murdock-Eaton, 2011).

Communication skills, as a course and content area, is still fairly new to Saudi Arabia, and research on the perceptions or effectiveness of communication instruction has been focused on individual teaching strategies or interventions (Arkoudis & Kelly, 2016) or communication skills as a byproduct of English language teaching (Alrashidi & Phan, 2015). It is unclear from the research in this context how students perceive communication skills instruction. Such research could shed more light on the subject through a focused lens, apart from other transferable skills, and possibly support improved instructional practice in communication skills for both instructors and students. In this study, the communication skills course was offered in the first-year foundation program of a large public university in Saudi Arabia. Although communication skills are integrated throughout program curricula, the course serves as an introduction to communication skills. This study aims to contribute to the limited research on the subject and bring communication skills as a content and instructional area into the forefront through students’ perceptions (Ardoukis & Kelly, 2016; Alofisan et al., 2016; Alotaibi & Alsaeedi, 2016; Havril, 2015; Karolak & Guta, 2015; McLean, Shaban & Murdoch-Eaten, 2011).

Research Question: Thus, this study uses a survey research design to explore one main question from two aspects: How do first-year undergraduate students perceive a communication skills course offered at a large, public Saudi university with regards to: 1) Course effectiveness on students’ communication skills?

2) Course relevance to students as individuals?

http://ijlter.org/index.php/ijlter

199

2. Related Literature To frame the study, communication skills in higher education, relevance in relation to teaching communication skills, and the study context are discussed through literature related to the Gulf region in the Middle East.

2.1. Communication skills in higher education Communication skills in higher education, defined as “oral and written communication skills aligned with professional, academic and community standards” (Ardoukis & Kelly, 2016, p. 3), are recognized as a highly-desired skillset for 21st Century university graduates. As shown in various research studies, there is no definitive approach to teaching or assessing communication skills either globally or regionally (Alofisan et al., 2016; Ardoukis & Kelly, 2016; Wyatt & Nunn, 2019). Some academic programs include elements of interpersonal, business, and intercultural communication within one course. In other instances, communication skills are woven throughout the curriculum as a part of the discipline or as a series of courses (Karolak & Guta, 2015). In some programs, a basic communication skills course is offered as an introductory course in students’ first year (Arkoudis & Kelly, 2016; Munz & Colvin, 2018) with communication skills then integrated throughout the remainder of the students’ curriculum. The latter represents the communication course approach in this study’s context.

Beyond the basic definition of ‘oral and written communication’ as desired competencies, there are various communication aspects to be considered under the scope of communication skills that include communicating with confidence, non-verbal “gestures, facial expressions, body posture, and use of space” (Bonvillain, 2003, p. 35), and intercultural communication considerations. Munz and Colvin’s (2018) study in a basic communication course found that students had communication apprehension or anxiety in situations of “real or anticipated communication” (McCroskey, 2001, p. 40) related to cultural identity factors including perceived language proficiency, possible judgment by peers, and religious affiliation. The authors concluded that the diversity of the students should be supported pedagogically through more informal interactions and activities related to their identities in the communication classroom.

Regionally, McLean, Shaban and Murdoch-Eaten (2011) studied the development of general skills over a one-year period in a first-year medical program in United Arab Emirates (UAE). The authors found that students’ experiences with communication skills improved over the year though they were still not sufficiently confident in their skills, partially due to their perceived English language proficiency and lack of practice. From a pedagogical perspective, the authors suggested that “opportunities for practice of transferable skills should be identified in the curriculum” (p. e297) to improve students’ experience and confidence, and the purpose of such courses should be “articulated as important for future practice and should be assessed” (p. e297). Similarly, Wyatt and Nunn (2019) studied a community of practice in teaching communication skills to engineering students through project-based learning in UAE and concur with regional and international studies (Noblitt, Vance & Depoy Smith, 2010) that

http://ijlter.org/index.php/ijlter

200

found that student-centered, practical approaches, such as project-based learning and case studies, improve team, oral and non-verbal communication skills.

English-language scholar Al-Seghayer (2015) argued that communicative competence is being taught only minimally in the Saudi context, and urged instructors to use student-centered techniques such as discussions, interviews, and role plays to promote communication skills and the ability to interpret meaning in discourse. The author suggested that educators should “equip their students with techniques they could employ in the case of a communication breakdown” (p. 90) to support more meaningful communication instruction.

Alofisan et al. (2016) found that the vast majority (95%) of Saudi pediatric residents in hospitals across the Kingdom believed that learning communication skills was crucial and that medical students should be taught effective communication skills for strong patient-doctor relationships (93%). Yet, only 39% reported having communication skill training available to them and even fewer felt highly confident in their communication skills, similar to other regional studies mentioned. The authors concluded that communication skills training should be integrated into the residents’ curriculum.

Similarly, Alotaibi and Alsaeedi (2016) studied medical students’ attitudes toward communication skills in Taif University (western Saudi Arabia). The researchers found that medical students believed that communication skills were important for future doctors who would need to communicate well and respectfully with patients. The study also revealed that more advanced students (level 5) had a more positive attitude than students earlier in their academic career (level 2).

2.2. Communication skills and teaching with relevance Building on Ladson-Billings (2013) concept of culturally-relevant pedagogy, Gay (2010) defines culturally responsive teaching as “using the cultural knowledge, prior experiences, frames of reference, and performance styles of ethnically diverse students to make learning encounters more relevant and effective” (p. 31) including examples, materials, and other references that are relevant to the students as individuals. Bassey (2016) also posits that educators who employ “a culturally responsive method in their teaching see culture as a strength which can be used effectively to enhance academic and social achievement” (p. 1) and promote equitable education, in which culture is a valued asset. Thus, cultural relevance in relation to communication skills is important as effective and meaningful communication skills rely on implicit and explicit cultural knowledge and experiences through different channels (e.g., oral, written, non-verbal).

Further emphasizing the importance of communication and teaching with relevance in the Saudi context, Nouraldeen and Elyas (2014), in support of Kramsch’s (1998) work, posit that culture, language and communication are related from various aspects as people use “spoken, written or visual means to create meaning…reflect their attitudes, beliefs, and point of view” and as a “symbol of their social identity” (Nouraldeen & Elyas, 2014, p. 64). As English is considered a global language (Crystal, 2003) and the language of communication

http://ijlter.org/index.php/ijlter

201

in many scientific disciplines in the Arab world (Zughoul, 2003), communication skills in English are relevant to students in this context. Further, these skills in English are viewed as instrumental for students’ future, personally and professionally in particular (Alrashidi & Khan, 2015; Elyas, 2008). In AlSeghayer’s (2011) seminal work on teaching English in Saudi Arabia, the author emphasizes how the English language has been and still is an important tool for obtaining and advancing careers, international business activities, flexible access to global knowledge, and understanding of and between as a “link language” (p.

13) to other cultures, i.e., a means of intercultural communication.

Yet, as recognized by Nouraldeen and Elyas (2014) and international scholars (McLean, Shaban, & Murdoch-Eaton, 2011; Munz & Colvin, 2018), there are other factors to be considered when teaching communication skills, not only the language of instruction. Nouraldeen and Elyas (2014), in their study on cultural factors that affect learning English, discuss communication and its relationship to the Islamic culture, which, the authors argue, is synonymous with Saudi culture.

In agreement with Al Haq and Smadi’s (1996) earlier work, the authors posit that Islam encourages communication through different languages to spread the faith, increase knowledge, and enhance communication with people from other cultures. The research also found that while Saudi students communicate in the English language for instrumental purposes (i.e., as a function to work/study in international/intercultural contexts), Arabic is often preferred to communicate within conversations, within the culture, at the interpersonal level (both oral and written), as part of cultural identity, i.e., Arabic is their ‘mother tongue’ that offers fluent conversation and understanding.

As many faculty members in Saudi higher education are expatriates and students study/travel abroad (Al-Seghayer, 2011), intercultural communication patterns for Saudi students are increasingly dynamic. Havril (2015), who studied Saudi university students’ intercultural competence at Jazan University (southwest Saudi Arabia), concluded that intercultural competence, both in positive and negative experiences, is needed in progressively multicultural environments. “If the participants of intercultural situations experience more positive and negative, upward and downward dimensions, in the end, they can really become interculturally competent in our culture-convergent world” (Havril, 2015, p. 565).

Karolak and Guta (2015) also studied communication patterns between students and their international instructors from the students’ perspective in Saudi higher education. The authors discussed the complexities of understanding nonverbal cues, such as head nodding or lack of response, not necessarily as unwillingness to communicate or passive participation but as not fully understanding the utterance.

These complexities can cause miscommunication and/or communication avoidance and, ultimately, the authors recommended teaching considerations and approaches that are culturally-relevant. The authors recommended explicit instruction to deal with communication uncertainty, clear focused objectives and assignments, and debates to ensure students know how to represent their points of view. In concurrence with studies previously mentioned, Karolak and Guta (2015) concluded that intercultural competences such as

http://ijlter.org/index.php/ijlter

202

empathy, tolerance, “patience, flexibility, suspending judgment” (p. 52) are needed for Saudi students’ futures in an increasingly globalized workforce.

2.3. Communication skills course in the study context The context for this study was a communication skills course offered in the firstyear program of a large public university in Saudi Arabia. As per the current practice in Saudi higher education, students take a foundation year to prepare for study in their discipline. The communication skills course is part of a series of general courses including English as a Foreign Language (EFL), mathematics, natural sciences, computer education, and related supporting studies. Although communication skills are integrated throughout the curriculum as required by the National Qualifications Framework, this course serves as an introductory, standalone course in the foundation year. The course medium of instruction, the lecture notes/booklet, presentations, and the vast majority of resources, including videos and course links, are in the English language, and the course is gender-segregated as per the Saudi cultural norm for undergraduate students. Course lecture notes and presentations include a few scattered references to the Saudi culture when referring to non-verbal communication such as eye contact/gestures and the mention of communication as positive speech and good manners in the Islamic tradition. This course, a supporting course to prepare students for better communication in their prospective colleges, has been offered since 2015 and averages over 20 sections with 20-25 students (average) in each section.

The course focuses on nine communication skills topics: communication principles, channels of communication distinguishing verbal and non-verbal communication, intro to field-specific communication (health, computer science, etc.), barriers to communication, communication self-awareness, presentation skills (including basic paralinguistic elements of vocal variety and prosody), professional/work communication, and teamwork. Topics are covered weekly, a total of 28 instructional hours, and the main assignments associated with these topics are graded role plays, a group project consisting of a multi-media presentation (speeches with visual), and two written assignments (resume and skills assessment) along with quizzes/final as assessments. Throughout the weekly interactive lectures, several activities such as case studies for communication, spontaneous role plays, and discussion prompts are included.

The weekly lectures are also standardized for the faculty members through shared presentations prepared by the department.

3. Methodology 3.1. Research Design To explore the research question of the communication course’s perceived effect on communication skills and relevance, this study employed a cross-sectional, online survey design in which data were collected from first-year students enrolled in the introductory communication skills course. This design was chosen as surveys provide a “description of trends, attitudes, or opinions of population by studying a sample” (Creswell, 2014, p. 155) and for ease of participation, particularly during the COVID-19 pandemic and consequent online instruction.

The survey was distributed in April 2020 (primarily via social media applications and learning platforms) and completed anonymously in April-May by first-year

http://ijlter.org/index.php/ijlter

203

students who were enrolled in the communication skills course in the 2019-2020 academic year. The two-part survey design measured students’ perceptions trends in scaled items (quantitative) with regards to course effectiveness and course relevance and explored their perceptions further in open-ended shortanswer questions (qualitative).

3.2. Participants Of the 225 participants,16 responses were removed from the data due to incomplete or unrelated responses. The final sample analyzed consisted of 209 Saudi nationals (ages 17-20) in the first year of a foundation year program, in a large, public university as shown in Table 1. Although a convenience sample, participants represented seven different sections of communication skills taught by six different instructors.

Table 1. Participant demographics (n = 209).

Gender

Prospective College

Male Female Health Computer Science/Information Technology Business No response/other

Frequency 82 127 120

% 39.2 60.8 57.4

42

20.1

35

16.7

12

5.7

3.3. Instrument The online instrument was created and piloted specifically for this study to answer the question not only of perceived course effectiveness, but also the issue of cultural relevance in this context. To ensure content validity, five bilingual experts with doctorates and extensive experience in teaching in Saudi Arabia and the Gulf region reviewed and offered feedback on the survey. An expert in educational evaluation and measurement provided feedback on the scale and direction of the items for clarity and ease of completion. A professor in curriculum and pedagogy revised the curriculum/course materials survey items while a third expert in educational psychology offered perspective on varying the survey into scaled and short-answer questions to gather more meaningful data. A fourth expert with a doctorate in English reviewed the language for enhanced meaning, and, finally, a communication professor examined the scope of the items in relation to the communication skills course and concept. Further, the researcher has academic background in the communication discipline and has taught various communication skills courses in Saudi Arabia for several years.

After the initial review and feedback to ensure content validity (Creswell, 2014), three items were removed for lack of clarity, leading or repetitive questions, and six of the questions were rewritten. The survey was then piloted further with four current undergraduate students all who had taken communication skills within the past two years to review content and clarity of questions and language. Based on the student feedback, the visual format was changed for easy use on various

http://ijlter.org/index.php/ijlter

204

devices and one question was slightly changed to provide further clarity. Within one week of the survey distribution, the researcher also discussed the survey (questions and format) with another bilingual instructor and two students who had completed the survey for further feedback. At that point, after review of the instrument by five experts, one colleague, and six students, no changes or revisions were suggested.

The resultant 19-item survey (see Appendix 1) consisted of 12 scaled items on a 4point Likert scale from strongly disagree (1) to strongly agree (4) and seven shortanswer questions to allow participants to express their perceptions and add more explanatory data to the research design (Creswell, 2013). Cronbach's Alpha was used to measure instrument reliability on the scaled items and found to be .896 showing high internal consistency (Nunnally & Bernstein, 1994). The validity of the internal consistency of the scaled items was verified by calculating the Pearson correlation coefficient between the scores for each item and the aspect studied (course effectiveness and relevance) and found to be internally consistent as shown in Appendix 1.

The data of the scaled-item responses were found to have a moderately leftskewed, non-normal distribution with a skewness of -.506 (SE = .168) and kurtosis of .015 (SE = .335); Shapiro-Wilk test: W (209) = .972, p = .00). The central limit theorem in statistics states that in large samples (> 30), data distribution tends to be normal, regardless of the shape of the data, and parametric tests can be used (Field, 2013; LaMorte, 2016; Zahediasl & Ghasemi, 2012). In addition, the seven qualitative short-answer questions further supported and expanded upon the scaled items. For the descriptive data, mean ranges were categorized to interpret and analyze the data trends at equal intervals (.75): 4.00-3.28 (Strongly Agree); 3.27-2.52 (Agree); 2.51-1.76 (Disagree); 1.75-1.00 (Strongly Disagree).

3.4. Data analysis As the survey consisted of two forms of data, scaled survey and short-answer responses, two analyses, both quantitative and qualitative, were completed. For the scaled-survey items, descriptive and inferential statistical analyses were conducted considering frequency, percentages, and the variables of gender and prospective college via an independent sample t-test and one-way ANOVA. For the short-answer responses, qualitative analysis, including coding responses and identifying recurring patterns in the data (Creswell, 2013), was performed to inductively explore participants’ perceptions on the communication course and its relevance to the students. Based on the recurring patterns, common themes emerged, which were then rechecked with the data and aligned with the scaled items to answer the research questions, as shown in the results. Thus, the scaled survey items measuring effectiveness and relevance were corroborated and supported with qualitative short-answer responses. In reporting the qualitative responses, the participants’ exact quotes were used for authenticity of data.

4. Results Overall, first-year students enrolled in an introductory communication skills course perceived that the course had positive effects on their communication skills and relevance to them as individuals (M = 2.81; SD = .579). As further shown in

http://ijlter.org/index.php/ijlter

205

Table 2 and the students’ short-answer responses, students ranked their understanding of the concepts of communication/communication skills higher than the relevance of the course to them as students. Using statistical analyses to explore key variables of participants’ gender and prospective college (α = 0.05), an independent sample t-test revealed that there was no statistically significant difference in student perception of the communication course based on gender (female M = 2.89; SD =. 55; male M = 2.74; SD = .64; T = 1.875; α = .063). A oneway ANOVA analysis on the variable of prospective college also indicated no statistical significance (F = 1.294; α = .274) on students’ perceptions of the communication course. It was shown, however, that IT and business-track students (M = 2.86; SD = .52 and M = 2.84; SD = .53 respectively) perceived the communication course slightly more positively, followed by the health-track students (M = 2.79; SD =. 60).

Table 2. Students’ perceptions of the communication skills course (n = 209).

Mean

SD

General trend

Course Effect on Communication Skills

2.91

0.590

Agree

Course Relevance to Participants

2.60

0.695

Agree

Overall

2.81

0.579

Rank 1 2

Agree

4.1. Communication course effectiveness Of the 10 survey items (eight scaled and two short-answer) directly concerned with the aspect of course effectiveness on communication skills, participants agreed that communication was important in their lives and that the course had a positive effect on their communication skills (M =2.91; SD = .590). As shown in Table 3, the majority of participants (85%) perceived communication skills as important ‘for my future career’ (M = 3.29; SD =.891). Similarly, nearly 87% of participants responded that ‘communication is used in all major areas of life’ (M = 3.25; SD = .794).

At the lowest end of the item rankings, there was division on participants’ responses about whether the course should be required (M = 2.46; SD = .938); half of participants (50.2%) responded that the course should not be required and nearly the same number (49.8%) agreed it should be a required course.

Table 3. Descriptive statistics of communication skills course effectiveness.

Scale Items Communication skills are important for my future career.

Strongly disagree

Disagree

Agree

Strongly agree

N

15

16

71

107

%

7.2

7.7

34.0

51.2

http://ijlter.org/index.php/ijlter

M

SD

3.29

.891

Rank

Response Trend

1

Strongly agree

206

Communication is used in all major areas of life.

The skills learned in this course will help me in my other courses.

N

9

19

92

89

%

4.3

9.1

44.0

42.6

N

17

37

110

45

%

8.1

17.7

52.6

21.5

Communication skills should be a required course.

N

37

68

75

29

%

17.7

32.5

35.9

13.9

N

22

62

97

28

%

10.5

29.7

46.4

13.4

N

23

75

81

30

%

11.0

35.9

38.8

14.4

N

9

23

107

70

%

4.3

11.0

51.2

33.5

N

10

34

103

62

%

4.8

16.3

49.3

29.7

This course helped me understand myself more as a communicator.

I communicate better with my classmates/peers after this course.

If I apply good communication skills, I will be a better person for myself and for others.

Diversity (different viewpoints) improves communication skills.

Course Effect on Communication Skills

3.25

.794

2.88

.840

2.46

.938

2.63

.846

2.56

.870

3.14

.755

3.04

.808

2.91

0.590

2

Agree

5

Agree

8

Disagree

6

Agree

7

Agree

3

Agree

4

Agree

Agree

To further explore perceptions of the communication skills course, participants were also asked ‘what communication skills were learned or improved in the course’ and ‘after taking the course, define communication skills’ in short-answer responses (See Appendix 1). Two consistent themes emerged from the shortanswer data in this area: 1) communication as a skill for information transfer, and 2) communication skills to build relationships and understand others.

4.1.1. Communication skills as information transfer The majority of student participants (about 69%) defined communication as a direct transfer of information from sender to receiver or between people in one form or another for a direct purpose; communication as a ‘way’ or ‘process to send and receive messages’, to ‘give and take messages,’ or to communicate for a purpose such as to inform: Sending and receiving signals from a sender to a receptor for a certain reason or goal. (health-track student) Communication is sending information, feelings, thoughts, ideas and work through some channels like social media or talking or gesturing to someone. (computer science student)

http://ijlter.org/index.php/ijlter

207

In congruence with the definition of communication for a direct, focused purpose or as information transmission, a large percentage of the participants, about 60%, mentioned presentation, sometimes referred to as ‘talking’ or ‘speaking,’ and nonverbal communication as the skills most improved or learned in the course. These skills, as reported by students, focus on the ‘how’ of direct communication: When someone speaks to you, you should see him [eye contact]. And don’t give him your back. (health-track student) Maintaining better face [facial] gestures. (computer science student) Speaking, presentation, and non-verbal communication skills. (several similar responses) 4.1.2. Communication skills to build relationships and understand others The second major theme in students’ responses when defining communication and learned/improved communication skills was communication skills to build relationships and understand others. Students discussed elements of empathy (rather than sympathy) and listening to other people’s opinion for better understanding: Communication is very important to understand the world, people, ideas, cultures, and everything; establish rapport and maintaining it. (several similar responses) Set of skills that enable and facilitate communication with different types of people, from different cultures and communities. (health-track student) When responding about gained or improved communication skills, students (40%) noted that the most important communication skills improved were related to building relationships and understanding others: Empathy + accepting other people's opinion. (health-track student) Reaching out [to others] to communicate/active listening. (business student) 4.2. Communication course relevance To consider the communication skills course relevance to participants as individuals, the survey questions, four scaled and four short-answer, focused on students’ perceptions of the language of instruction, curriculum/materials relevance including from an Arab/Islamic perspective, and practical applications (i.e., assignments and activities).

As illustrated in Table 4, students, as a general trend, slightly agreed that the course book and materials were relevant to them as individuals, deepened their understanding of the concept, and improved their experience (M =2.60; SD = .695).

The highest ranking in this aspect was related to the course material improving students’ learning experience (M = 2.70; SD = .837). However, participants disagreed that the assignments helped them ‘become a better communicator’ (M = 2.45; SD = .935) with 19% of students strongly disagreeing on this item.

http://ijlter.org/index.php/ijlter

208

Table 4. Descriptive statistics of communication course relevance.

Scale Strongly disagree

Disagree

Agree

Strongly agree

27

52

104

26

%

12.9

24.9

49.8

12.4

N

20

53

105

31

%

9.6

25.4

50.2

14.8

N

15

67

103

24

%

7.2

32.1

49.3

11.5

N

40

60

84

25

%

19.1

28.7

40.2

12.0

Related Items The course book and materials deepened my understanding of communication.

The course material improved my learning experience.

The course material was related to me as a person; I could identify with it.

Overall, the assignments helped me become a better communicator.

N

Course Relevance

M

SD

2.62

.864

2.70

.837

2.65

.777

2.45

.935

2.60

.695

Rank

Response Trend

3

Agree

1

Agree

2

Agree

4

Disagree

Agree

Participants were also asked short-answer questions to explore their perceptions of the course relevance to them as individuals: ‘Should the course be taught in English or Arabic,’ ‘Were the course book, materials, and examples related to the Saudi/Arab or Islamic culture?’ ‘Which assignments or activities improved your communication skills the most and the least’ (See Appendix 1). For each question, students were asked to briefly explain their responses. Three main themes emerged from the short-answer data: 1) relevance of language of instruction; 2) communication skills linked to religious/cultural heritage and intercultural communication; and, 3) relevance of oral communication activities.

4.2.1. Relevance of language of instruction Students’ short-answer responses showed uncertainty about the language of instruction—whether communication skills should be taught in their professional language of English or native language of Arabic. A slight majority of the students indicated that the course should be taught in English (53%), others recommended Arabic (25%), and a third group suggested both languages to offer students a choice (22%). Noteworthy to mention, the question did not suggest instruction in both languages as an option; nevertheless, several students added the response.

The majority of explanations for those who preferred the course in English focused on English as a global or professional language needed for their coursework:

http://ijlter.org/index.php/ijlter

209

English, it is the universal language (used by most countries); In English because almost all of our future study depends on [the] language. (several similar responses) English, because English has become the main language. But that doesn’t mean that people should ignore Arabic communication skills. (healthtrack student) Arabic was mentioned as the ‘mother tongue’ to be used within the culture (intracultural) and in interpersonal situations for better expression and also considering issues of language proficiency; students take a separate English language course simultaneously with the communication skills course. A sizeable third group of participants suggested that the course should be offered in both languages, perhaps bilingually, to further benefit students: Arabic, firstly because it's my native language. Secondly there are more Arabic speakers than English one in K.S.A. [Saudi Arabia]. Thirdly, Arabic is nearer to our hearts than English. There is a variety of vocabular in Arabic that lets us express our emotions better than English. (healthtrack student) In my opinion I think it should be taught in both languages because our first language is Arabic, which will be used more in this country, but, also, we need to learn this course using English language because it is the global language. (computer science student) 4.2.2. Relevance to students’ heritage and intercultural communication The short-answer question regarding course materials/examples used and their relevance to Saudi culture (i.e., culturally-relevant pedagogy) offered the most varied responses in that students often replied with both yes/no or hesitant responses of “I think so” or “I don’t know.” Considering the diversity of responses to this question, the overarching pattern of the data suggested that communication skills are essential, no matter the course materials or examples, as part of the students’ cultural and religious (Islamic) belief structure. In more detail, a slight majority of participants (55%) perceived the course as a relevant because, as shown in their responses, being a good communicator is related to being a good Saudi Muslim.

Yes, because Islam wants people to be good communicators. (health-track student) Yes, it is, because Islam religion / our culture encourages people to behave and communicate well with others. (several similar responses) Another substantial number of participants (45%) responded that the course materials and examples were not directly related to Saudi culture. However, several viewed the lack of relevance as an opportunity to improve their intercultural communication skills. A few perceived the lack of culturally relevant materials/examples as a course weakness.

It was a bit broad and it doesn’t directly apply to the Islamic culture, which is GOOD. We need to know how to communicate with different people that aren’t necessarily Muslims. (health-track student)

http://ijlter.org/index.php/ijlter

210

Not at all, but in a general way, yes, good communication skills to adapt with others. (business student) There were no explicit examples of our culture in the material. It certainly would’ve been a more engaging experience if there were such examples. (healthtrack student) 4.2.3. Relevance of oral activities/communication The large majority of participants (89%) mentioned oral activities and assignments as related to their improvement of communication skills with only 11% mentioning the writing activities. Of the oral activities, in-class discussions and group work/group activities, in-class role plays were mentioned 48% of the time by participants (including elements of non-verbal communication such as facial gestures and body language). These activities were mentioned more often than the formal assignments of role plays and presentations (41%). Students also mentioned oral activities as practicing their skills for the future and improving self-confidence and self-awareness in their explanations: Presentations and in-class discussions; it is a practical application for what we learned, and we will need these skills in [the] future; because these actions make me more confident. (several similar responses) n-class discussions because it is a practical training of the most common communication method which is face- to-face conversations. (healthtrack student) Further, to the question of course improvements, most responses were general; however, nearly one-third of students (29%) made recommendations related to both course effectiveness and course relevance scaled-survey items in their responses. Several students indicated that the course should be “non-graded, not obligatory,” “as a discussion course,” and use “more relevant and relatable examples,” “more related to our needs,” and “something we can use” with “more practical assignments.” Some students mentioned that the main purpose of the communication skills course should be clarified.

5. Discussion First-year university students in this study affirmed the positive role of communication skills in their current lives and future (M =2.91, as shown in Table 3) and, to a lesser degree, course relevance to them as individuals (M = 2.60, illustrated in Table 4). Participants responded that communication skills were important for their lives and careers, and diversity of viewpoints improves communication (all above 3.0/4.0). However, scaled and short-answer responses related to the communication skills course were not as decisive (below 3.0), which is explored in the following discussion.

In the short-answer responses, participants defined communication skills as predominantly an oral (89%), information transfer (69%) that builds relationships with others. While the introductory concept of communication was seemingly clear to students, a limited mention of written communication activities and assignments (11%) illustrates that the students perceived oral communication as the most relevant form of communication in the course. A plausible reason for the

http://ijlter.org/index.php/ijlter

211

focus on the oral mode is that students, as shown in their short-answer responses, perceived the spoken in-class activities including spontaneous role plays, discussions, and in-class presentations as more effective than the written assessments on improving their communication skills. As shown in Table 4, students disagreed (M = 2.45) that the overall assignments improved their communication skills. In other words, students, as suggested by regional studies (Al-Seghayer, 2011; Karolak & Guta, 2015; McLean, Shaban & Murdoch-Eaton, 2011) found practical, in-class communication activities more effective and relevant for improving their skills.

The most complex findings of this research are related to students’ perceptions about the course relevance. Students’ responses were divided regarding language of instruction and indecisive about the relevance of course material to Saudi/Arab culture. English as the language of instruction was supported by just over half of participants while 47% either suggested the course should be taught in Arabic or both English and Arabic. The primary reasons were related to the fact that English is the language of their future professional careers while for several participants, Arabic, as their ‘mother tongue,’ is preferred for increased understanding and communication. Several students also mentioned practicing English through the communication course while others felt it was hard to understand the communication concepts in English. This highlights the language proficiency concern reiterated in other studies (McLean, Shaban, & Murdoch-Eaton, 2011; Munz & Colvin, 2018). The implication here is that the language of instruction should be considered in relation to course purpose, benefit to academic programs and students, and students’ current language proficiency level.

Regarding course material relevance, scaled-item responses showed slight student agreement about overall course relevance (M = 2.60), but short-answer responses revealed considerable ambiguity about the cultural relevance of the course materials. In their explanations, students often began with words of uncertainty, ”I don’t know” or ”I think so”. Several mentioned, in concurrence with Nouraldeen and Elyas’ (2014) research, that the relevance of the communication skills course was directly related to their Islamic heritage and upbringing which values and encourages strong and polite communication skills.

However, a large number of students (45%) perceived the course materials as not related, which several felt offered opportunities to learn and adapt to communication with those from other cultures. Thus, these students did not view the connection between intercultural communication and their own cultural context—the fact that intercultural communication and competence builds from their own culture. This concurs with Karolak and Guta’s (2015) insight that some students may not be able to ”grasp the specific aspects of culture in a consistent manner” (p. 51) particularly if students have not had extended experience with other cultures (Havril, 2015). A few astute students commented that explicit examples from the Saudi culture would have made the course more engaging and relatable. Unlike the point about language and communication skills, the majority of participants seemingly did not recognize the relationship between culture and communication skills (Bonvillain, 2003; Nouraldeen & Elyas, 2014). This illustrates a missed pedagogical opportunity to connect communication skills and

http://ijlter.org/index.php/ijlter

212

culture through culturally-relevant teaching and application of communication skills for students’ future professional lives (Bassey, 2016). The indication then is that communication skills can explicitly support intercultural communication and still be clearly relevant and connected to the students’ native culture.

Also related to the issue of relevance and course effectiveness was course status.

Students’ responses were divided about communication skills as a required course. Half of the participants disagreed about the course status as required and, in the open short-answer response about course improvements, nearly one-third of students mentioned that the course should be not required or non-graded, a discussion type course, or more relatable to be relevant. A plausible reason for this finding is related to Arkoudis and Kelly’s (2016) critical discussion on the benefit of communication skills as part of an integrated disciplinary curriculum, not viewed as ’supplementary’ by students. To illustrate, Alotaibi and Alsaeedi’s (2016) study with Saudi medical students found that the highest-rated negative attitude scores toward learning communication skills were on the items “learning communication skills is too easy” and “communication skills teaching would have a better image if it sounded more like a science subject” (p. 794). Therefore, in agreement with the results of this study, students may perceive the communication skills subject as less important or relevant than other academic/scientific subjects, e.g., physics, biology, mathematics, or not directly related to their academic discipline. Although students agreed on the importance of communication skills (M= 3.29) and mentioned aspects of the course that improved their skills, the necessity of the course as a foundation-year requirement was not clear to half of the students.

6. Recommendations Based on the related literature and this study’s findings, there are several possible recommendations for teaching communication skills with relevance. Firstly, the purpose and focus of the communication skills course should be clear to both faculty and students. Explicit instruction on the focus and goals of the course (e.g., oral, written, intercultural, or a concentrated combination) and related practical assignments will facilitate students’ understanding of the course, its importance, and benefits. Focused instruction will further support the communication skills taught and, consequently, how these skills relate to and are used by students, i.e., how these skills are transferable to other academic and professional contexts.

As shown by this and previous studies—no matter the context, teaching interpersonal communication with relevance, considering culture, language, and related student factors is essential for communication. Communication, language, and culture are inextricably connected. Thus, communication skills instruction should reflect the nuances of the cultural context to be relevant. Examples, videos, and materials should reflect the student and his/her professional needs.

Communication skills curricula can have a global focus and a myriad of concepts but should be adapted to respond to the context of the culture, the Saudi context in this case. Activities and assignments that are practical and culturally relevant, such as case studies and role plays, can benefit the students by highlighting the complexities of communication. Similarly, as evident in short-answer responses,

http://ijlter.org/index.php/ijlter

213

the language of instruction matters to students. Thus, several factors should be considered regarding the language of instruction: purpose of the course/communication content, goals and language of the academic program, and students’ language proficiency considering their stage in the program (early or advanced), and their ability to understand and communicate in the course language.

Finally, communication is a discipline in and of itself. Effective communication skills instruction should reflect relevant elements of the communication discipline and students’ academic field. Thus, a co-constructed curriculum between discipline and communication experts/designers could ensure that disciplinespecific and general communication skills are negotiated, integrated, and assessed within the students’ curriculum. As Arkoudis and Kelly (2016) affirmed, “The literature is unequivocal that high impact student learning occurs when communication skills are integrated within disciplinary learning and assessment” (p. 4). Moreover, in agreement with Karolak and Guta (2015), general and disciplinary professional development sessions on teaching communication skills in the Saudi context would support instruction and facilitate enhanced communication between faculty members and students.

As with all research, there are limitations and areas for improvement to expand on in future research. Exploration of communication skills courses from other Saudi universities and regions of the country with a larger sample is encouraged for more conclusive findings. Further, due to the nature of self-reporting, perceptions may be influenced by a variety of factors including understanding concepts, language proficiency, and the current COVID-19 pandemic and its implications on university study and coursework. A longitudinal study with more in-depth qualitative data could provide further insight into students’ perceptions throughout their undergraduate study and more generalizable results related to the effectiveness of communication skills instruction over time.

7. Conclusion This study aimed to explore undergraduate Saudi students’ perceptions of a foundation-year communication skills course from the aspects of course effectiveness and relevance. The results indicated that the students in this study agreed that the course was effective in improving their communication skills and slightly agreed that the course was relevant to them as individuals, albeit with caveats. Students perceived the high importance of communication skills and viewed communication as primarily an oral, information transfer to build relationships with others. Participants also perceived in-class teaching and learning activities such as role plays, presentations, and discussions as effective for improving their communication skills. At the same time, a sizeable number of participants questioned the effectiveness and, primarily, the relevance of the course language of instruction, the relevance of course materials including examples and assignments, and communication skills as a requirement. In other words, the majority of participants perceived communication skills as highly important in their lives and future careers but were somewhat divided and

http://ijlter.org/index.php/ijlter

214

uncertain about the relevance of the course to them as individuals and could not consistently link the communication skills course to their culture.

These findings, though on only a sample of students enrolled in communication skills, reveal a fertile pathway for future research and teaching with relevance.

Further study could facilitate enhanced understanding of teaching communication skills in the Saudi context and lead to more relevant teaching and learning that would positively impact and benefit students in their careers and professional lives. As Saudi society becomes increasingly globalized, communication skills that are dynamic and adaptable are pertinent to advance knowledge and bring positive intercultural and intracultural exchange. Therefore, student-centered, relevant and explicit instruction is essential for deepened and evidenced impact on students’ communication skills.

Acknowledgement The author gratefully acknowledges the editors and reviewers for their valuable comments and insights, which significantly improved the manuscript.

8. References Al Haq, F. A, & Smadi, O. (1996). Spread of English and Westernization in Saudi Arabia.

World Englishes, 15(3), 307-317. doi:10.1111/j.1467-971X.1996.tb00117.x Alofisan, T., Al-Alaiyan, S., Al-Abdulsalam, M., Siddiqui, K. Hussain, I., & Al-Qahtani, M. H. (2016). Communication skills in pediatric training program: National-based survey of residents’ perspectives in Saudi Arabia. Journal of Family and Community Medicine, 23(1), 43–47. doi:10.4103/2230-8229.172233 Alotaibi, F. S., & Alsaeedi, A. (2016). Attitudes of medical students toward communication skills learning in Western Saudi Arabia. Saudi Medical Journal, 37(7), 791-795. doi:10.15537/smj.2016.7.14331 Alrashidi, O., & Phan, H. (2015). Education context and English teaching and learning in the Kingdom of Saudi Arabia: An overview. English Language Teaching, 8(5), 3344. doi:10.5539/elt.v8n5p33 Al-Seghayer, K. (2011). English teaching in Saudi Arabia: Status, issues, and challenges.

Riyadh, Saudi Arabia: Hala Printed Co. Al-Seghayer, K. (2015). Salient key features of actual English instructional practices in Saudi Arabia. English Language Teaching, 8(6), 89-99. doi:10.5539/elt.v8n6p89 Arkoudis, S. & Doughney, L. (2014). Good practice report: English language proficiency.

Sydney, Australia: Australian Government Office for Learning and Teaching. https://melbournecshe.unimelb.edu.au/__data/assets/pdf_file/0004/1489162/GPR_English_lang uage_2014.pdf Arkoudis, S., & Kelly, P. (2016). Shifting the narrative: International students and communication skills in higher education. International Education Research Digest 8, 2-12. https://www.ieaa.org.au/documents/item/664 Bassey, M. O. (2016). Culturally responsive teaching: Implications for educational justice.

Education Sciences, 6(35), 1-6. doi:10.3390/educsci6040035 Bonvillain, N. (2003). Language, culture, and communication: The meaning of messages (4th ed.). Upper Saddle River, NJ: Pearson Education, Inc. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.

http://ijlter.org/index.php/ijlter

215

Creswell, J. W. (2013). Qualitative inquiry & research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc. Crystal, D. (2003). English as a global language (2nd ed.). Cambridge, UK: Cambridge University Press.

Education and Training National Evaluation Commission. (2020). Alitar Alwatani lilmuahalaat [National Qualifications Framework]. Riyadh, Saudi Arabia: Education and Training National Evaluation Commission.

Elyas, T. (2008). The attitude and the impact of the American English as a global language within the Saudi education system. Novitas-ROYAL, 2(1), 28-48. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.557.9274 Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed). Thousand Oaks, CA: Sage Publications, Inc. Gay, G. (2010). Culturally responsive teaching: Theory, research, and practice (2nd ed.). New York, NY: Teachers College Press.

Ghasemi, A., & Zahediasl, S. (2012). Normality tests for statistical analysis: A guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2), 486489. doi:10.5812/ijem.3505.

Havril. A. K. (2015). Improving intercultural competence of female university students in EFL within Saudi Arabia. Procedia - Social and Behavioral Sciences, 192, 554–566. doi:10.1016/j.sbspro.2015.06.091 Karolak, M., & Guta, H. (2015). Intercultural communication in the context of Saudi Arab tertiary education. In Raddawi R. (eds.) Intercultural Communication with Arabs (4156). Springer, Singapore. https://doi.org/10.1007/978-981-287-254-8_4 Kramsch, C. J. (1998). Language and Culture. Oxford: Oxford University Press.

Ladson-Billings, G. (2013). The dreamkeepers: Successful teachers of African American children (2nd ed.). San Francisco, CA: Jossey Bass.

LaMorte, W. W. (2016). Central limit theorem. Boston University School of Public Health. https://sphweb.bumc.bu.edu/otlt/MPHModules/BS/BS704_Probability/BS704_Probability12.html McCroskey, J. C. (2001). An introduction to rhetorical communication. Boston, MA: Allyn & Bacon.

McLean, M., Shaban, S., & Murdoch-Eaten, D. (2011). Transferable skills of incoming medical students and their development over the first academic year: The United Arab Emirates experience.

Medical Teacher, 33, 297-305. doi:10.3109/0142159X.2011.565826 Munz, S. M., & Colvin, J. (2018). Communication apprehension: Understanding communication skills and cultural identity in the basic communication course.

Basic Communication Course Annul, 30(10), 172-199. https://ecommons.udayton.edu/bcca/vol30/iss1/10 Noblitt, L., Vance, D. E., & Depoy Smith, M. L. (2010). A comparison of case study and traditional teaching methods for improvement of oral communication and critical-thinking skills. Journal of College Science Teaching, 39(5), 26-32.

Nouraldeen, A. S., & Elyas, T. (2014). Learning English in Saudi Arabia: A socio-cultural perspective. International Journal of English Language and Linguistics Research, 2(3), 56-78. http://eajournals.org/wp-content/uploads/Learning-English-in-SaudiArabia-A-Socio-Cultural-Perspective1.pdf Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGrawHill.

Rüttermann, S., Sobotta, A., Hahn, P., Kiessling, C., & Härtl, A. (2017). Teaching and assessment of communication skills in undergraduate dental education – a survey

http://ijlter.org/index.php/ijlter

216

in German speaking countries. European Journal of Dental Education, 21, 151-158. https://doi.org/10.1111/eje.12194 Wyatt, M., & Nunn, R. (2019). Tracing the growth of a community of practice centered on holistic project-based learning in communication at an engineering university in the United Arab Emirates: Insights from a socially-situated teacher cognition perspective. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 20(2), Article 8. doi:10.17169/fqs-20.2.3170 Zughoul, M. R. (2003, May). Globalization and EFL/ESL pedagogy in the Arab world [Conference presentation]. 23rd Annual Convention of the Moroccan Association of Teachers of English (MATE), Fez, Morocco. https://files.eric.ed.gov/fulltext/ED479810.pdf

Appendix 1 Communication Skills Course Survey Scaled Items Course Effectiveness 1.

2.

3.

4.

5.

6.

7.

8.

Communication skills are important for my future career.

Communication is used in all major areas of life.

The skills learned in this course will help me in my other courses.

Communication skills should be a required course.

This course helped me understand myself more as a communicator.

I communicate better with my classmates/peers after this course.

If I apply good communication skills, I will be a better person for myself and for others.

Diversity (different viewpoints) improves communication skills.

Course Relevance Items

9.

The course book and materials deepened my understanding of communication.

10. The course material improved my learning experience.

11. The course material was related to me as a person; I could identify with it.

12. Overall, the assignments helped me become a better communicator. **Correlation is significant at the 0.01 level (2-tailed).

http://ijlter.org/index.php/ijlter

Correlation Coefficient 0.858**

0.000

0.887**

0.000

0.897**

0.000

0.628**

0.000

0.816**

0.000

0.827**

0.000

0.768**

0.000

0.706**

0.000

Correlation Coefficient 0.774**

Significance

0.929**

0.000

0.523**

0.000

0.790**

0.000

Significance

0.000

217

13.

14.

15.

16.

17.

18.

19.

Survey Short-Answer Items Should the course be taught in English or Arabic? Please explain your answer.

Were the course book, materials, and examples related to the Saudi/Arab or Islamic culture? Why or why not?

Which assignments or activities improved your communication skills the most?

How? Which assignments or activities improved your communication skills the least?

Why? After taking the course, define communication skills.

What communication skills were learned or improved in the course?

How would you improve the course (if applicable)?

http://ijlter.org/index.php/ijlter

218

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 218-234, February 2021 https://doi.org/10.26803/ijlter.20.2.12

Students’ Soft Skills, Hard Skills, and Competitiveness (SHC): A Suggested Model for Indonesian Higher Education Curriculum Hadiyanto, Noferdiman, Syamsurizal and Muhaimin Universitas Jambi, Kota Jambi, Indonesia https://orcid.org/0000-0003-4697-9544 https://orcid.org/0000-0002-9446-6359 https://orcid.org/0000-0002-5277-0205 https://orcid.org/0000-0002-6269-5931 Ina Krisantia Universitas Trisakti, Jakarta, Indonesia https://orcid.org/0000-0002-9982-686X

Abstract. This study develops a model for students and graduates of Soft Skills, Hard Skills, and Competitiveness (SHC) and seeks user validation after the development process. The development process uses the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model. A quantitative survey using questionnaires and descriptive quantitative analysis was applied in obtaining user validation as the study objective. The development results validated the students' SHC Development model, which has 3 main phases, including Input, Process and Output. The Input phase comprises course content, SHC component, and strategy of students’ center approach. The Process phase consists of blended course design, instruction and students' engagement, and evaluation. The Output phase comprises soft and hard skills achievement, as well as students’ competitiveness level. Users perceived the model’s phases and components as wholly acceptable, appropriate, and applicable for implementation through curriculum, teaching, and learning at the university. This study contributes to graduate preparation for employability skills in the challenging and complex working environment in revolution industry 4.0.

Keywords: 21st Century Skills; lifelong learning; employability; entrepreneurship; revolution industry 4.0

1. Introduction Higher Education in Indonesia is facing unprecedented challenges arising from the convergent impacts of the Asian Societies Market. Higher education is a growth engine for triggering a country to meet the current needs and prepare for ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

219

future challenges. The issue is related to HE curriculum changes and the fresh graduates’ ability to compete in Asian job market, meet employer’s expectations, adapt and update their competencies beyond today’s challenges. The fresh graduates’ competencies should be gained during their university education, embedded with the curriculum and its implementation.

Graduates’ competencies in the present study were developed based on current global skills understanding at the workplace, known as the 21st-century skills.

The terms used might differ across countries and disciplines, but the essential issues remain the same, such as generic, adaptive, key, soft, life, professional, and interpersonal skills. The researchers, employers, policymakers, and academicians agree that students should acquire these skills at the university and apply them at the workplace to meet the global enterprise needs (Rajadurai et al., 2018; Lippman et al., 2015; Bialik et al., 2015; ILO, 2014).

In line with the studies, the Indonesian Ministry of Science Technology and Higher Education via DIKTI (2016) launched the Guidance of Higher Education Curriculum that demanded universities to provide students with more soft skills than hard skills. Hard skills are acquired and developed through soft skill practices in the teaching and learning process. According to DIKTI (2020), the paradigm of Independent Learning and Independent Campus was launched by the Ministry of Education to provide students with soft and hard skills more than academic competence. Therefore, students are expected to possess individual competitiveness in the job market and their future career development. However, there is no clear guidance in developing graduate's Soft skills, Hard skills, and Competitiveness (SHC) in Universities unless incorporated into teaching and learning.

DIKTI (2020), Thomas (2016), Bialik et al. (2015), and Ahlstrom et al. (2014) suggested that the development of students’ hard and soft skills should be blended in the teaching and learning process. Learning strategies should provide the students with SHC, though this is not easy. The problem is how learning activities enable students to gain soft skills through the teaching and learning strategy. In solving this problem, a model was developed as a grand learning concept at an Indonesian University. The model guides the teaching of subjects that develop students’ soft and hard skills and competitiveness.

However, the defining and the concept were reviewed and found to follow the global trend of university curriculum and employability skills. This explains why the concept contributes to anticipating global issues, such as the lack of graduate employability skills and its combination in the teaching and learning process. Therefore, this study describes and discusses the process and outcomes of the model for developing students’ and graduate Soft skills, Hard skills, and Competitiveness (SHC) at a University in Indonesia.

1.1.

Concept Students SHC Development from Curriculum into Teaching and Learning Practice Indonesian Ministry of Education and Culture announced a new paradigm of Independent Learning and Independent Campus. The paradigm changes the learning delivery strategy to be more independent and involves some industries

http://ijlter.org/index.php/ijlter

220

as students’ learning field work. Furthermore, the paradigm regenerates the previous curriculum that emphasis on students acquiring soft and hard skills through their learning strategy. Indonesia’s higher education curriculum does not mention specific components of soft and hard skills, and competitiveness.

However, the learning objectives are mentioned as developing students’ hard skills, soft skills, and competitiveness. The mentioned soft skills include professionalism, communication, decision-making, problem-solving, critical thinking, and another similar attribute understood globally (DIKTI, 2020; DIKTI, 2016; DIKTI, 2016). Theoretical analysis was conducted in this study to draw the students’ SHC Development Model. Related studies and resources showed that students’ SHC development in universities is conceptualized into the input, process, and output phases discussed in the following section.

Input Phase Input phases are connoted with teachers’ preparation to define learning material, students’ soft and hard skills, and delivery method. Learning material and content of hard skills should be selected based on students' needs and 5 years latest resources. The delivery method should be selected to allow students to practice and develop their SHC (Adreeva et al., 2020). SHC is defined at the beginning based on the program curriculum that could be integrated with learning material and course design and practiced by students within and outside the classroom (Hadiyanto, 2020; Caggiano, Loredana & Jerónimo, 2020).

1.1.1.

Student-Centered Learning (SCL) is the most appropriate and popular approach to engaging students in obtaining qualified graduates. This regards the teaching and learning approach in developing students' soft and hard skills and competitiveness. SCL facilitates the students to acquire subject knowledge and hard and softs skills during the learning process (Hadiyanto et al., 2021; DIKTI, 2016). The SCL approach is dominated by constructivism theories in which students are allowed to practice individually and in groups. Students think and solve problems, communicate ideas, use IT, and work with other members of the class. However, giving students this opportunity does not mean teachers do not control the students' in-class activities. Instead, the teachers need to develop the main guidelines of learning activities to achieve the set goals (UPI, 2020; DIKTI, 2016; Bialik, 2015).

The teachers' SCL strategies include blended, inquiry, cooperative, collaborative, problem-based, and laboratory learning, as well as E-learning Usage. Suitable learning strategies are selected based on their appropriateness with course topics and students’ interests. The strategy should encourage students to practice their soft skills actively and explore and acquire hard skills (Khalil & Elkhider, 2016).

1.1.2. Process Phase The students’ SHC development should start from the syllabus and learning scenario by including soft and hard skills and competitiveness. The learning design, goals, and objectives for students' engagement and activities are process phase that should be considered for promoting soft and hard skills and competitiveness (Andreeva et al., 2020; Novikova, 2016; Bialik et al., 2015). A

http://ijlter.org/index.php/ijlter

221

good learning process involves students actively communicating, searching, using ICT, discussing, working together, self-reflecting, learning from others, and achieving goals. A course design is part of the learning process essential in learning and enabling students to practice soft and hard skills. Therefore, the course design should be flexible, revisable, and reflective during the learning process. Moreover, it should be exposed to the students at the first lecture meeting to assess its applicability and make necessary revisions to meet the students’ needs and avoid possible handicaps. The delivery method could be planned between the first and second week, and some improvement is made based on the previous reflection. This explains why the course design comes during the process and not at the beginning or during education input (Hadiyanto, 2020).

The SCL learning approach should be conducted through a social process, such as students' interactions, activities, and engagement. Teachers should encourage students to participate actively and contribute to their learning activities.

Therefore, the SCL learning methods, such as group projects, field visits, selflearning, exploring, and presentation, are applicable under some strategies mentioned in the input phase. For instance, allowing students to present their work is a popular method applied in teaching and learning. It allows students to share and communicate ideas, learn to attract attention, develop self-confidence before many people, and other related skills (The Ontario Public Services, 2016; Marando, 2012) Students’ soft and hard skills are developed during learning, meaning teachers should use the strategy to guide the students in the right way. This requires the teachers’ creativity and critical thinking to design, implement and facilitate the learning activities based on students’ competencies, knowledge and skill development. However, Hadiyanto et al. (2017) and Burce & O’Sullivan (2014) stated that teaching strategies must match the learning goal, and the expected hard and soft skills. DIKTI (2016), Hadiyanto et al. (2021), and Ahlstrom et al.

(2014) showed that the teachers’ principles that facilitate students to gain knowledge and skills include: a. Understanding how students learn b. Matching the learning and teaching principles c. Facilitating rather than control learning d. Motivating and encouraging students to interact in learning actively e. Assuring the practice of soft skills and acquisition of hard skills. f. Promoting and expecting student accountability for learning g. Providing timely, specific feedback on learner progress h. Individualizing learning experiences based on needs Students’ SHC is developed during the teaching and learning process. Therefore, SCL should apply in syllabus, lesson plan, and learning process to construct student’ SHC. Furthermore, teachers suggested applying teaching and learning strategies to optimize the gaining of soft and hard skills. Therefore, SHC students’ SHC self-evaluation and rubrics assessment could be developed based on theoretical analysis of SHC and academician judgment (Cimatti, 2016).

http://ijlter.org/index.php/ijlter

222

1.1.3. Output Phases Literature reviews show that Higher Education across countries agrees that university graduates should develop high soft and hard skills and competitiveness. The graduate should be ready to work on global and digital challenges, promote self-expertise and continue learning to improve their quality, impacting institutional and national competitiveness (Hadiyanto, 2020; DIKTI, 2020; UPI, 2020). Moreover, the graduate should be attributed soft and hard skills based on employee recruitment, employers’ interview, Indonesian curriculum and higher education paradigm. In this case, students’ competitiveness level is the outcome of soft and hard skills.

This study defines Soft Skills, Hard Skills, and Competitiveness (SHC) as a blended ability to achieve an objective (Hadiyanto et al., 2017; Andreeva et al., 2020; Novikova, 2016). Universities help students to develop soft and hard skills, which determine their competitiveness. Soft Skills are essential in the global work market and are practiced in learning activities. Also, they are practiced at workplaces through communication, IT, numeracy, learning, problem-solving skills, and working with others. Soft skills are developed during the learning process within or outside the campus and at the workplace. The practice of soft skills generates and develops students' hard skills in their disciplines (Hadiyanto, 2017, Khalil & Elkhider, 2016, UNJA, 2014).

Communication Skills involve expressing and exchanging ideas using feelings of thought and verbal and non-verbal media, including speech and written text (Khalil & Elkhider, 2016).

IT Skills are one’s ability to use digital technology of computers, integrated devices, and necessary applications for working. These applications include Microsoft Office, Internet Browser, E-mail, Messenger, Online Meeting, Digital File Manager, and Network (Hadiyanto et al., 2021; Burce & O’sullivan, 2014).

Numeracy Skills are the ability to use basic mathematic calculation, interpreting graphical information, timing, prioritizing tasks, and sequencing of job or activities (Balcar, Šimek & Filipová, 2018; Hadiyanto et al., 2017). Learning Skills are the ability to use strategies, evaluate the self-learning strategy, seek weaknesses, and develop better learning output. This includes gaining general and detailed information, knowledge, and skills to achieve the goal of learning (Hadiyanto et al., 2017; Alberta Education. 2011; Jones, 2009).

Problem-solving skills involve the ability to systematically tackle problems to develop an appropriate solution (Lippman et al., 2015; Ahlstrom et al., 2014).

Team-work skills are the capacity to interact effectively with other people individually and in groups. This includes understanding and responding to a client’s needs and working effectively as a team member to achieve a goal (Luara et al., 2016; Ahlstrom et al., 2014; ILO, 2014). Hard Skills are the ability to apply and generate specific knowledge and skills in real context and work.

Students acquire hard skills to practice their softs skills, such as presentation, discussion, and using applications for working out assignments and group projects (Ahlstrom et al., 2014, Marando, 2012 and Chan, Goh & Priest, 2015).

http://ijlter.org/index.php/ijlter

223

Competitiveness is the eagerness and the effort to consistently apply, maintain, improve and promote soft and hard skills in work, tasks and learning. It is categorized into lifelong learning, entrepreneurship, and employability.

Competitiveness was assumed to be affected by soft and hard skills (Balcar et al., 2018; Kumar, 2017; UNJA, 2020; Novikova, 2016). Entrepreneurship is the ability to apply soft and hard skills in creating, promoting, proposing, building, exploring, seeking, and offering business opportunities and services based on expertise (UNJA, 2020; DIKTI, 2020; Balcar et al., 2018; Liventsova, Rumyantsevaa & Syriamkina, 2016). Lifelong learning is the eagerness and willingness to learn beyond what someone knows currently. Furthermore, lifelong learning involves developing beyond current capacity to meet future challenges using multi-resource and comparing resources, friends, environment, and ICT (Hadiyanto et al., 2017; DIKTI, 2016; Bialik et al., 2015).

Employability is the ability to apply specific skills, subject knowledge, and standard quality of a graduate. It comprises certain competencies to adapt and meet the changing needs of employers and customers and the ability to be selfemployed (DIKTI, 2020: UPI 2020; Liventsova et al., 2016: Bialik et al. 2015).

2. Research Methodology The model of Students SHC was developed through research and development.

The development phases and steps were adapted from the ADDIE model by Branch and Robert Maribe (2009), Richey and Klein (2007), and Dick, Carey and Carey (2001). The main development phases applied included Analysis, Designing, Developing, Implementing, Evaluation, and Revision. Mixed-mode Quantitative and Qualitative methods were applied in some parts of the model development process. The instruments used were literature reviews, questionnaires, interviews, SHC assessment rubrics, FGD Protocol, and documentaries. The study used 3 groups of participants, including stakeholders, lecturers, and students.

2.1. The Phase of Model Development The first phase is the analysis of previous research and publication, HE curriculum, Vision and Mission, and strategic plan of the University of Jambi (Unja, 2020). Also, there was a survey on students’ needs, current literature review, stakeholders’ system recruitment, and interview. The second phase involved designing, which included categorizing and defining SHC components. Moreover, there were designing phases and features of students' SHC development in the learning process and strategy. The third phase involved developing a student's SHC Development model, which comprises Input, Process, and Output. Furthermore, the phase involved the expert, users, and try-out validations. The fourth phase involved the implementation in 4 faculties, while the last phase was evaluation and revision.

2.2. Participants of the Study There were 26 lecturers that participated in the study, representing Economics and Business, Teacher Training and Education, Animal Husbandry, Science Technology, and Humanities Faculty. A total of 773 students across the faculties

http://ijlter.org/index.php/ijlter

224

were involved, with 513 as questionnaire respondents, while 250 participated in experimental research. Furthermore, 5 people representing stakeholders participated in FGD. 2.3. Validator of the Development Process This study used 3 external and 4 internal expert validators to assess and contribute to the model’s improvement. In particular, 2 external validators were from Universitas Brawijaya, with expertise in curriculum, teaching, and learning Science at University. One external validator was from Universitas Negeri Yogyakarta, having expertise in the higher education curriculum. Moreover, the 4 internal validators were from various disciplines, one was a professor and expert in R & D. Of the other 3 validators, the first was an expert in chemistry education, the second in psychology education, while the third had expertise in agriculture learning at the university. Qualitative reviews were addressed by all validators related to contents, design of the model, components, and phases of students' SHC Development.

2.4. Sampling for User Validation User validation was conducted in the last phase of the study after revision.

Stratified random sampling was used to select the samples among the Universities’ teaching staff. It involved distributing the questionnaires to 5 faculties, including Economics and Business, Teachers Training and Education, Agriculture, Animal Husbandry, and Science Technology. Each faculty was provided with 50 questionnaires, resulting in a total of 250 questionnaires. Out of this total, 216 questionnaires were returned. The survey aimed at evaluating user acceptance, appropriateness and applicability of the model. This included phases and components of students’ soft and hard skills, as well as competitiveness development.

2.5. Method and Instrumentation A quantitative method was applied in investigating user validation. The instrument was developed and justified through a qualitative approach, literature review analysis, workshop among the research team, and piloted to 20 teachers. Reliability was measured by a consistency test. Pallant (2011) and Hair et al. (2009) suggested that Cronbach alpha coefficient (α) .60 for a construct comprises a maximum of 10 items, while coefficient .70 means the construct consists of more than 10 items. The consistency test resulted in α above .60 for components with a maximum of 10 items, and α above .70 for components with more than 10 items. Acceptance of Model Arrangement in General comprising 5 items yielded α .605, appropriateness of SHC component α .804, the applicability of SHC component α .743, appropriateness of phases SHC development α .819 and applicability of phases SHC development α .780.

2.6. Data Analysis of User Validation Descriptive analysis was applied in which the mean and standard deviation values were used to report the findings. The interpretation of the means score was as follow; 1.00 – 1.80 = Strongly Unacceptable, Inappropriate (SI), or Inapplicable (SIP)

http://ijlter.org/index.php/ijlter

225

1.81 – 2.60 = Unacceptable, Inappropriate, or Inapplicable 2.61 – 3.40 = Fair 3.41 – 4.20 = Acceptable, Appropriate, or Applicable 4.20 – 5.00 = Strong Acceptable, Appropriate (SA), or Applicable (SAP)

3. The Study Results 3.1. Result of Model of Students SHC Development The students' SHC Development model at a University should be started and integrated with the curriculum, course design, syllabus, lesson plan, as well as offline and online learning. Students SHC is developed in blended learning activities in every course of a program. This model was justified by global literature studies and is divided into Input, Process, and Output phases.

3.1.1. Input The Input phase involves planning and preparation before implementing students’ SHC Development Model into the teaching and learning process. In this phase, teachers should search hand-outs, materials, and resources for course content and consider components of soft and hard skills to be integrated into the syllabus and course design. Additionally, the delivery method and strategies should be defined to match the materials, hand-outs, and expected SHC outcomes.

3.1.2. Process The process phase is an implementation of students' SHC Development through the teaching and learning process. The syllabus and lessons plan unit was developed in this phase. The SHC components, delivery method and strategy, and teachers’ guidelines should be clearly stated in the syllabus and lesson plan.

Moreover, teachers need to act the lesson plan during class activities. Students are facilitated and guided to actively acquire soft and hard skills by interacting, discussing, exploring, and using multi-resources based on the lesson plan. Also, teachers must observe students' practice of soft and hard skills.

3.1.3. Output The output phase involves assessing the students’ achievement based on SHC standards. Students are expected to gain certain SHC levels through teaching and learning within a semester. Students’ self-evaluation questionnaire and SHC rubric assessment (Hadiyanto et al., 2017) are used to measure the level of SHC. Theoretically, high students’ soft and hard skills increase their competitiveness (Setiawan et al., 2020)

http://ijlter.org/index.php/ijlter

226

Figure 1: Model of Students’ SHC Development at A University in Indonesia (DIKTI, 2020; UNJA, 2020; UPI, 2020; Hadiyanto et al., 2017; Lippman et al., 2015; DIKTI, 2016; Bialik et al., 2015)

Result of A survey of User Validation of the Model The results show that the overall arrangement of SHC Model development is acceptable. Furthermore, teachers perceived that the model's arrangement into Input, Process, Output, and Layout indicated by arrows and lines is strongly acceptable. The other 3 elements were Arrangement of Part of Input, Process, and Output perceived as acceptable. Moreover, teachers, components under input, process and output, and Implementation flexibility were rated as acceptable. These findings implied that students SHC model development is acceptable by teachers, as shown in Table 2: 3.2.

Table 2: Overall Model Arrangement in General A. Acceptance of Model Arrangement in General 1. Arrangement of Model; Input, Process, and Output 2. Arrangement of Part of Input, Process, and Output 3. Layout, indicating arrow and lines 4. Arrangement of components under input, process and output.

5. Flexibility for Implementation Overall

Min.

Max.

Mean

S.td

Level

3,00

5,00

4,21

,695

SA

3,00

5,00

4,13

,542

Acceptable

3,00

5,00

4,23

,59

SA

3,00

5,00

4,06

,570

Acceptable

3,00 3,20

5,00 5,00

3,97 4,12

,483 ,353

Acceptable Acceptable

Respondents rated the overall component under SHC as appropriate. Also, soft skills and competitiveness of SHC components were rated as appropriate. Both mean scores were close to strongly appropriate, while the hard skills component was rated as strongly appropriate. Based on specific components of soft skills,

http://ijlter.org/index.php/ijlter

227

communication, IT, working with others was rated as strongly appropriate.

Numeracy, problem-solving, and working with others were rated as appropriate. Specific knowledge as one of the hard skills components was rated as strongly appropriate, while specific skills were rated as appropriate, with a mean score of 4.17, which was very close to strongly appropriate. The 2 specific competitiveness components, entrepreneurship, and employability were rated as appropriate, while lifelong learning was rated as strongly appropriate, as shown in Table 3: Table 3: Appropriateness of SHC Component B. Appropriateness of SHC Component a. Soft Skills 1. Communication 2.

3.

4.

5.

6.

Numeracy IT Problem Solving Learning How to learn Working with others Overall

b. Hard Skills 1. Specific knowledge 2. Specific skills Overall c. Competitiveness 1. Entrepreneurship 2. Employability 3. Lifelong Learning Overall SHC Component Appropriateness

Min.

Max.

Mean

S.td

Level

3,00

5,00

4,24

,537

SA

3,00 3,00 3,00 3,00 3,00 3,33

5,00 5,00 5,00 5,00 5,00 5,00

4,05 4,35 4,07 4,08 4,22 4,17

,524 ,547 ,440 ,428 ,458 ,339

Appropriate SA Appropriate Appropriate SA Appropriate

3,00 3,00 3,10

5,00 5,00 5,00

4,27 4,17 4,22

,520 ,480 ,439

SA Appropriate SA

3,00

5,00

3,77

,512

Appropriate

3,00 3,00 3,05 3,64

5,00 5,00 5,00 4,91

4,13 4,21 4,04 4,14

,454 ,490 ,363 ,283

Appropriate SA SA SA

3.3. Applicability of SHC Component Respondents rated the overall applicability of components under SHC as applicable. Overall Hard skills were rated as strongly applicable, while Soft skills and competitiveness were rated as applicable. All mean scores of the components were close to strongly applicable or at the top of the applicable mean. Based on specific components of soft skills, IT was rated as strongly applicable, while communication and working with others were rated as strongly appropriate. Numeracy, problem-solving, and working with others were rated as applicable. As one of the hard skills components, specific knowledge was rated as strongly applicable, while specific skills were rated as applicable. All specific components of competitiveness, entrepreneurship, employability, and lifelong learning were rated as applicable.

http://ijlter.org/index.php/ijlter

228

Table 4: Applicability of SHC Component C. a.

1.

2.

3.

Applicability of SHC Component Soft Skills Communication Numeracy IT

Min.

Max.

Mean

S.td

Level

3,00 3,00 3,00

5,00 5,00 5,00

4,13 3,94 4,26

,508 ,453 ,516

Applicable Applicable SAP

4. Problem Solving 5. Learning How to learn 6. Working with others Overall b. Hard Skills 1. Specific knowledge

3,00 3,00 3,00 3,33

5,00 5,00 5,00 5,00

3,97 4,00 4,17 4,08

,426 ,406 ,462 ,312

Applicable Applicable Applicable Applicable

3,00

5,00

4,25

,477

SAP

2. Specific skills Overall c. Competitiveness 1. Entrepreneurship 2. Employability 3. Lifelong Learning

3,00 3,00

5,00 5,00

4,17 4,21

,442 ,400

Applicable SAP

3,00 3,00 3,00

5,00 5,00 5,00

3,71 4,11 4,18

,507 ,452 ,468

Applicable Applicable Applicable

3,33 3,55

5,00 4,82

4,00 4,08

,330 ,247

Applicable Applicable

Overall SHC Component Applicability

Respondents rated the overall appropriateness of SHC development phases at university as appropriate. Also, the Input, Process, and Output phases were rated as appropriate. Respondents rated the course content and resources of Input phases as strongly appropriate, while the SHC and learning approach components were appropriate. Concerning the Process phase, the course design was claimed as strongly appropriate, while students’ engagement and evaluation were rated as appropriate. Respondents rated the elements of the Output phase, competence achievement, and graduate competitiveness as appropriate. The 2 specific components of competitiveness, entrepreneurship, and employability were rated as appropriate, while lifelong learning was strongly appropriate.

Table 5: Appropriateness of SHC Phases Development D. Appropriateness of SHC phases Development a. Input 1. Course Content Resources 2. Component of SHC 3. Learning Approach Overall b. Process 1. Course design 2. Students’ engagement

http://ijlter.org/index.php/ijlter

Min.

Max.

Mean

S.td

Level

4,00 3,00 3,00 3,33

5,00 5,00 5,00 5,00

4,27 4,14 4,13 4,18

,448 ,461 ,393 ,329

SA Appropriate Appropriate Appropriate

3,00 3,00

5,00 5,00

4,23 4,10

,444 ,464

SA Appropriate

229

3. Evaluation Overall c. Output 1. Competencies Achievement

3,00 3,00

5,00 5,00

3,94 4,09

,557 ,387

Appropriate Appropriate

3,00

5,00

3,97

,426

Appropriate

2. Graduate Competitiveness level Overall Overall Phases Appropriateness.

3,00 3,00 3,38

5,00 5,00 5,00

4,07 4,02 4,11

,376 ,328 ,298

Appropriate Appropriate Appropriate

Respondents rated the phases of students' SHC development as applicable. The Input, Process and Output phases were rated as applicable. Furthermore, respondent rated the course content resources, component of SHC and learning approach as applicable. Based on the Process phase, course design was strongly applicable, while students’ engagement and evaluation were rated as applicable.

The elements of Output phases, competencies achievement, and graduate competitiveness level were applicable.

Table 6: Applicability of SHC phases development E. Applicability of phases SHC development a. Input 1. Course Content Resources 2. Component of SHC 3. Learning Approach Overall

Min.

Max.

Mean

S.td

Level

3,00 3,00 3,00 3,00

5,00 5,00 5,00 5,00

4,19 4,14 4,09 4,14

,461 ,461 ,348 ,305

Applicable Applicable Applicable Applicable

Process Course design Students’ engagement Evaluation Overall c. Output

3,00 3,00 3,00 3,00

5,00 5,00 5,00 5,00

4,21 4,08 3,93 4,08

,433 ,448 ,548 ,369

SA Applicable Applicable Applicable Applicable

1. Competencies Achievement 2. Graduate Competitiveness level Overall Overall Phases Applicability

3,00 3,00 3,00 3,13

5,00 5,00 5,00 4,88

3,96 4,04 4,00 4,08

,436 ,333 ,305 ,275

Applicable Applicable Applicable Applicable

b.

1.

2.

3.

4. Discussion Teachers, stakeholders, students, policymakers were involved in developing the R and D model of Students SHC Development at the university. Cimatti (2016) stated that the university suggested seeking match definitions, model methods, tools, and the soft and hard skills applicable for learning practice. This way would enable the graduate to achieve high employability performance. The students’ SHC Development model is conducted in 3 phases, including Input, Process, and Output, and is validated as either internal or external. Internal validation confirms the components and phases of students SHC Development model, while external validation evaluates the impact of the model’s products

http://ijlter.org/index.php/ijlter

230

use. The users’ validation method was excellent in receiving feedback on how the model fits their course (Richey and Klein, 2007).

Teachers accepted the model of students' SHC development in the Input, Process, and Output phases and its layout. Also, they understood the position and the arrangement of the model and accepted the placement and arrow between elements of Input, Process, and Output. Md-Ali, Shaffie & Yusof (2016) showed that teachers understood and accepted the integration of soft and hard skills in teaching and learning to allow students to develop employability skills.

Teachers perceived that the inclusion of SHC components overall was appropriate. The soft skills and competitiveness of SHC components were appropriate. Both mean scores were close to strongly appropriate, while the hard skills component was strongly appropriate. The 2 specific components of competitiveness, entrepreneurship, and employability were rated as appropriate, while lifelong learning was strongly appropriate. These findings implied that components and sub-component of SHC fit their courses and meet today's challenge in higher education. According to Liventsova et al. (2016), universities worldwide should develop soft skills and competitiveness.

Therefore, teachers should be provided with a teaching and learning model that develops students’ soft and hard skills to nurture their competitiveness.

Respondents confirmed that the SHC model phases are applicable in teaching their courses. Overall hard skills were rated as strongly applicable, while soft skills and competitiveness were applicable. All mean scores of the sub-skills of SHC were close to strongly applicable or at the top of the applicable, mean score.

The phases, SHC, and their components are relevant for implementation in the teaching and learning of all courses at the university. These findings support Caggiano et al. (2020) that HE teachers perceive softs skills as relevant for integration with course skills in implementing curriculum design in Finland, Italy, and beyond. Furthermore, while Andreeva et al. (2019) showed that the students’ soft and hard skills application in learning develops their competitiveness.

The students' SHC development model fits with teachers’ competence in any field and subject and applicable for the classroom, outdoors, and laboratory learning. According to Patcsil et al. (2017), teaching and learning softs and hard skills should fit the current needs and teachers’ competence.

This study guides Higher Education teachers to develop their curriculum design, generate high-quality learning facilities, and assess the students' SHC development activities (Caggiano et al., 2020). Universities should enhance or include the SHC development components in curriculum planning and revision.

Curriculum and learning design should be enhanced to improve students’ soft and hard skills and increase their competitiveness. Furthermore, according to Novikova (2016), teachers should be facilitated to improve their professional skills. This is because they play a key role in taking creative solutions to enrich the learning process with expressive means and modern educational techniques.

http://ijlter.org/index.php/ijlter

231

The research contributes to the theoretical and practical side of the students and graduates with 21st-century skills. Practically, the research contributes to universities because this model was conducted through extensive library research related to 21st-century skills development in the University context.

The model focused on the vision, mission, and curriculum of the present university and faculty. This aimed to improve graduates' quality to provide them with the skills needed to compete globally and contribute to the nation (Rajadurai et al., 2018; ILO, 2014). Moreover, the results of R & D provide answers to academician questions on how to integrate 21st-century skills into the learning design and process. Therefore, universities interested in this model could adapt and integrated it into their situation. This is because the model was developed based on global trends and issues and could be used as guidance or resource.

The model’s theory and conceptual framework provide university teachers with choices on the appropriate content delivery method to enable students to achieve learning goals and acquire 21st-century skills (UPI; 2020; Md-Ali et al., 2016). Also, the results show educators that students and graduates with 21stcentury skills are developed through a course teaching and learning that impacts their specific knowledge.

This research adds more theoretical value to the available literature on the concept and model of 21st-century skills and other related definitions. It gives more information on the theory and innovative concept of 21st-century skills implementation in university curriculum and faculty. Furthermore, the research is relevant for revolution industry 4.0 and 5.0 (DIKTI, 2020; Md-Ali, 2016; Cimatti, 2016). Initial implementation should be conducted through some research by a programmer on improving the management and evaluation of the model implementation. Also, classroom action research should evaluate how teachers create, select and apply a learning technique to develop students' SHC. Experimental research should ascertain the model’s effectiveness and improve the students’ SHC based on this study’s definition.

These findings were forwarded to the policymakers of this university. Only a few universities worldwide provide extracurricular training to equip students with SHC. Implementing the university-level model to the teaching and learning process would reduce the number of unskilled graduates. Furthermore, it allows students to acquire SHC acquisition, and graduates contribute to faster development at their workplace (Setiwan et al., 2020; Patacsil & Tablatin, 2017).

Therefore, this research suggests to university policymakers to implement and improve students’ SHC development strategies.

5. Conclusion This study developed a model of students’ SHC development through teaching and learning process at a University. The results showed that the students’ SHC Development model in teaching and learning has 3 main phases, including Input, Process, and Output. The users’ validation results proved that the model

http://ijlter.org/index.php/ijlter

232

is acceptable, appropriate and applicable for integration and implementation in curriculum and students learning process. Moreover, the model is expected to meet global issues and challenge, and answer questions regarding the low awareness of policy maker at universities on the needs of graduates’ SHC capabilities.

Teachers need to reinterpret the SHC development model for possible implementation in their course. This is because the model could become very large when teachers implement it as a whole. For instance, teachers should select the best learning strategy from those stated in the model that fits their course and develops soft skills. Additionally, they need to select the soft skills components that could be implemented and assessed in their course. Other uses of the model could not be adopted in other universities due to certain challenges. Therefore, university curriculum, national orientation, and existing issues in their countries should be considered, and preliminary analysis is conducted. Furthermore, the model could be adjusted and applied in other universities by carefully analyzing the national and university curriculum and literature reviews of current issues and global challenges. A further qualitative and quantitative study should be conducted on the model’s innovation, as well as the strategies and effectiveness of its implementation.

Acknowledgement The study was funded by RISPRO LPDP PRJ 13/LPDP/2018, Ministry of Finance Indonesia.

6. References Ahlstrom, A. W., Yohalem, N., David, Ji, P., Hillaker, P., & David, P. (2014). From Soft Skills to Hard Data: Measuring Youth Program Outcomes. New York, USA: William T. Grant Foundation.

Andreeva, Е., Khrisanova, Е., Gunina, Е., & Vishnevskaya, М. (2020). Psychological and Pedagogical Support for the Formation of Students’ Competitiveness in the Educational and Professional Space of the University. International Scientific and Practical Conference on Education, Health and Human Wellbeing Advances in Social Science, Education and Humanities Research, 396, 50–54. https://doi.org/10.2991/iceder-19.2020.11 Balcar, J., Šimek, M., & Filipová, L., (2018). Soft Skills of Czech Graduates, Review of Economic Perspectives.

De Gruyter, Warsaw, 18(1), 45-60, http://dx.doi.org/10.2478/revecp-2018-0003 Bialik, M., Bogan, M., Fadel, C., & Horyathova, M. (2015). Character Education for the 21st Century: What Should Students Learn? Center for Curriculum Redesign. Boston, USA: Center for Curriculum Redesign.

Branch, R. M. (2009). Instructional Design: The ADDIE Approach. USA: Springer.

Burce, A., & O’sullivan, N. (2014, September 22-23). Teaching and Learning in Competency-Based Education [Conference paper]. The Fifth International Conference on e-Learning (eLearning-2014). Belgrade, Serbia: Innovative Together Ltd. Caggiano, K. S., Loredana P., & Jerónimo G., (2020). Towards Identifying the Soft Skills Needed in Curricula: Finnish and Italian Students' Self-Evaluations Indicate Differences between Groups Valeria. Sustainability, 12(4031), 1-10 http://doi.org/10.3390/su12104031.

http://ijlter.org/index.php/ijlter

233

Chan, J., Goh, J., & Priest, K. (2015). Soft skills, hard challenges: Understanding the nature of China's skills gap. Londong, UK: British Council.

Cimatti, B. (2016). Definition, Development, Assessment of Soft Skills and Their Role for The Quality of Organizations and Enterprises. International Journal for Quality Research, 10(1), 97–130. http://doi.org/10.18421/IJQR10.01-05 Dick, W., Carey, L., & Carey, J. O. (2001). The systematic design of instruction (5th ed.).

New York, USA: Addison-Wesley Educational Publishers.

DIKTI. (2016). Panduan Penyusuanan Kurrikulum Pendidikan. Jakarta, Indonesia: DIKTI. DIKTI. (2020). Guide Book of Independent learning – Independence of Campus. http://dikti.kemdikbud.go.id/wp-content/uploads/2020/04/Buku-PanduanMerdeka-Belajar-Kampus-Merdeka-2020.

Hadiyanto, Mukminin, A., Failasofah. Arif, N., Fajaryani, N., & Habibi, A. (2017). In Search of Quality Student Teachers in a Digital Era: Reframing the Practices of Soft Skills in Teacher Education. TOJET: The Turkish Online Journal of Educational Technology, 16(3), 71-78.

Hadiyanto. (2020, September 30 - October 01). Constructing A Prototype of Developing Students' 21st Century Skills; Soft Skills, Hard Skills, and Competitiveness at a University in Indonesia.

ICEMS 2019, Jakarta, Indonesia. http://doi.org/10.4108/eai.30-9-2019.2291171 Hadiyanto, Failasofah, Armiwati, Abrar, M., & Thabran, Y. (2021). Students’ Practices of 21st Century Skills between Conventional learning and Blended Learning.

Journal of University Teaching & Learning Practice, 18(3), 7-19.

Hadiyanto, Noferdiman, Moehamin, & Yuliusman. (2017). Assessing Students And Graduates Soft Skills, Hard Skills And Competitiveness. International Journal of Social Sciences, 3(2), 1885-1906.

ILO. (2014). Survey of ASEAN employers on skills and competitiveness. Emerging Markets Consulting. Bankok, Thailand: ILO. Khalil, M., & Elkhider, I. A. (2016). Applying learning theories and instructional design models for effective instruction. Advances in physiology education, 40(2), 147-56. http://doi.org/10.1152/advan.00138.2015 Lippman, L. H, Ryberg, R., Carney, R., Moore, K. A. (2015). Child Trend:. Workforce Connections: Key "Soft Skills" That Foster Youth Workforce Success: Toward A Consensus Across Fields. https://www.usaid.gov/sites/default/files/documents/1865/KeySoftSkills.pd f.

Liventsova, E., Rumyantsevaa, T., & Syriamkina, E. (2016). Development of Social and Entrepreneurial Skills of Students of Engineering and Technical Specialties In The Modern University. Proceeding MATEC Web of Conferences (Vol. 7, pp. 19). http://doi.org/10.1051/matecconf/201679010 Marando, A. (2012). Balancing Project Management Hard Skill and Soft Skills.

Massachusetts, USA: Braindeis University.

Md-Ali, R., Shaffie, F., & Yusof, M. F. (2016). Understandings and Conceptions of Soft Skills for Educators in Public Universities. Future Academy: The European Proceedings of Social & Behavioural Sciences, 694–701.

Novikova, N. (2016). Competitiveness of Students in the Culture Sphere in the Modern Educational Space.

SHS Web of Conferences, 9, 1–4. http://doi.org/10.1051/shsconf/20162902031 Patacsil, F. F., & Tablatin, C. L. S. (2017). Exploring the Importance of Soft and Hard Skills as Perceived by It Internship Students and Industry: A Gap Analysis.

Journal of Technology and Science Education, 7(3), 347-368. https://doi.org/10.3926/jotse.271

http://ijlter.org/index.php/ijlter

234

Rajadurai, J., Sapuan, N. M., Daud, S., Salina, D., & Abidin, N (2018). The Marketability of Technical Graduates from Higher Educational Institutions (HEIs) Offering Technical and Vocational Education and Training (TVET): A Case from Malaysia.

Asia-Pacific Edu Res. 27(137), 535–543. https://doi.org/10.1007/s40299-018-0372-7 Setiawan, T. W., Qonita, A. B., Asna, Arna, A., Khaidir, M., Wahab. A., Rifa, M. J., Rozikan, M., Mufid, A., Fahmi, K., Purwanto, A., & Muhaini. A. (2020). Effect of Hard Skills, Soft Skills, Organizational Learning and Innovation Capability on Islamic University Lecturers' Performance. Sys Rev Pharm, 11(7), 556-569.

The Ontario Public Services. (2016). Towards Defining 21st Century Competencies for Ontario. Ontario, Canada: Ontario Public Service.

Universitas Indonesia-UPI. (2020). Panduan Implementasi Merdeka Belajar - Kampus Merdeka Universitas Pendidikan Indonesia. Bandung, Indonesia: Universitas Pendidikan Indonesia. http://cs.upi.edu/v2/assets/docs/Panduan_MBKM_UPI.pdf.

Universitas Jambi-UNJA. (2020).

Strategic Plan of University of Jambi. https://lppm.unja.ac.id/wp-content/uploads/2021/01/RENSTRA-UNJASMART-2.pdf

http://ijlter.org/index.php/ijlter

235

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 235-250, February 2021 https://doi.org/10.26803/ijlter.20.2.13

Decision-making by Heads of Academic Department using Student Evaluation of Instruction (SEI) Mohammed Saleh Alkathiri Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia https://orcid.org/0000-0002-4079-7347

Abstract. This study investigates the ways in which heads of academic department use student evaluation of instruction (SEI) to make decisions about individual faculty members and/or whole academic departments.

The study utilized a convenience sample of 57 heads of department, who completed an online questionnaire with two main constructs, which were assessed at the interval level of measurement. The results of the study revealed significant differences between heads of department who tend to trust SEI results compared to those who tend not to trust SEI results.

The findings suggest there is a significant association between how heads of department perceive SEI and how they use it to make decisions about individual faculty members and their academic departments. In addition, analysis of the respondents as per two groups, according to their attitudes of trust or distrust toward SEI, showed that disparities within these groups were greater with respect to issues or decisions that affect individuals as opposed to whole departments. Therefore, the study concludes that decisions should not be made based solely on the results of SEI; rather, multiple sources of evaluation should be utilized to make proper decisions. The author strongly recommends that academic leaders should use SEI across multiple years or courses in order to obtain more reliable information. Future research may include qualitative studies on the topic and discipline-specific studies within certain academic departments or college clusters.

Keywords: faculty members; heads of academic department; higher education; student evaluation of instruction; teaching and learning

1. Introduction The use of student evaluation of instruction (SEI) is, and has long been, one of the most common assessment practices in higher education. In the United States, SEI is the predominant form of faculty evaluation, and approximately 88% of all liberal arts colleges use SEI for summative decisions (Seldin, 1999). In 1991, the U.S. Department of Education reported that 97% of 40,582 heads of department who participated in a survey used SEI to assess teaching (Cashin, 2003). Today, many universities still use SEI to determine whether to grant faculty tenure,

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

236

promote them, raise their pay based on merit, and offer them opportunities for professional development (Kelly, Ponton & Rovai, 2007). In fact, at institutions where the emphasis is on teaching, SEI is an influential measure used in promotion decisions (Emery, Kramer & Tian, 2003). Furthermore, universities use SEI for various other purposes, such as establishing the credibility of the education they offer, planning strategy, and improving curricula (Massy & French, 2001; Scott & Hawke, 2003). Nonetheless, the use of SEI in higher education has been controversial. Whilst supporters of SEI view it as a valid and reliable tool that can be used to facilitate decisions, those who oppose SEI claim that it is biased because of many factors that influence its results.

SEI has been researched more often than other topics in higher education for decades (Theall & Franklin, 2001). Whereas many studies have supported the use of SEI as a valid indicator of quality instruction, several studies have questioned its validity and reliability for faculty evaluation. This paper provides a literature review of support for and opposition to SEI. Moreover, it identifies a gap in the literature around the question of how heads of department use SEI results to make decisions about faculty members and academic departments in the context of Saudi higher education. It then presents original research devised to fill that gap.

The results inform the reader of the present practical reality of SEI and show how practitioners and policy makers use SEI results in their decisions. Finally, this paper provides conclusions and recommendations for academic leaders on the use of SEI in higher education. These will enable well-informed evaluations of individual faculty members and overall academic departments, and thus ultimately facilitate better decision-making in future.

2. Literature Review 2.1 History of student evaluation of instruction The primary purpose of using SEI is to improve the quality of teaching and learning (Delvaux et al., 2013). Since ancient times, students have had a voice on how they are taught. For example, at the time of Socrates and in the medieval period (Tucker, 2015), students expressed their opinions about their teachers.

However, the use of SEI, as it is currently known and applied, to evaluate faculty teaching started in the early 1900s (Algozzine et al., 2004). Wachtel (1998) provided a brief review of SEI, indicating that “the first teacher rating scale was published in 1915” (p. 191). In the 1920s, several universities in the United States started student evaluation procedures (Wachtel, 1998). Today, SEI is used in universities worldwide. Taking into consideration the fact that faculty members today perform multiple responsibilities (Alkathiri, 2018), the purpose of SEI has been expanded by universities, as well as by quality assurance bodies. It is now used, for example, to allocate performance funding, to gather evidence to decide which faculty members to promote, and to select the winners of teaching awards (Arthur, 2009; Hendry & Dean, 2002; Massy & French, 2001; Scott & Hawke, 2003; Shah & Nair, 2012; Tucker, 2014).

According to Centra (1993), research on SEI went through four distinct periods.

First, the period from 1927 to 1960 was known by the pioneer work of Remmers and his colleagues at Purdue University. Second, in the 1960s, the use of SEI was

http://ijlter.org/index.php/ijlter

237

voluntary in almost all universities. Third, the 1970s was the “golden age of research on student evaluations” (Centra, 1993, p. 49). During this period, new research on SEI evolved, including studies that showed evidence of the validity of SEI and advocated the use of SEI for formative and summative evaluations.

Lastly, the fourth period started in the early 1980s, when research on SEI continued to expand, providing further illustration of research findings, including studies of the meta-analysis type.

According to Theall and Franklin (2001), more studies have been conducted on SEI than any other topic in higher education. Many studies have been done in the United States, Australia, and Europe. In contrast, Saudi Arabia, and the Arab world more generally, has been relatively understudied. Research on SEI has covered various subtopics such as the validity, reliability, and usefulness of SEI; the dimensions of effective teaching to be evaluated; the bias in student and instructor responses; and the identification of teaching excellence (Tucker, 2014).

Furthermore, many reviews of the literature and meta-analysis studies on SEI have been conducted (Alderman, Towers & Bannah, 2012; Perry & Smart, 2007; Richardson, 2005). Nonetheless, research seldom includes discussions of the use of SEI by heads of department to make decisions that may affect individual faculty members or academic departments. The current paper is intended to contribute to the field of higher education by rectifying that omission, presenting and analyzing new evidence from Saudi Arabia where the study took place.

2.2 Support for student evaluation of instruction Many researchers have claimed that SEI is an important indicator of where quality is improving and where it needs to be improved in teaching and learning, and in student satisfaction (Alkathiri, 2020; Zineldin, Akdag & Vasicheva, 2011). In addition, for decades, scholars have suggested that SEI can be considered a valid indicator of effective instruction. For example, as indicated by Liu (2012), SEI can predict ratings gathered from other sources, such as former students and colleagues. In addition, McKeachie (1997) claimed that SEI is a source of evaluation of teaching effectiveness that is more valid than any other. According to Liu (2012), early studies on SEI acknowledged its importance in teaching and learning because of the way in which SEI can actually reflect the quality of teaching based on student perceptions. Furthermore, literature (see Liu, 2012) has urged the use of SEI since students are able to furnish information on (1) learning goals, (2) student-instructor rapport, (3) teaching methods, (4) student-instructor communication, and (5) consumer data.

Research comparing SEI in distance education and face-to-face courses has concluded that there is little difference between the two modes of teaching in terms of the ratings of whole courses, and the quality of their instruction (Kelly et al., 2007; McGhee & Lowell, 2003; Waschull, 2001). Supporters of the use of SEI claim that if students are trained in using SEI, “evaluative judgements [given] on a regular basis have strong positive impact on the improvement of [faculty’s] instructional skills” (Spooren, Mortelmans & Denekens, 2007, p. 667). With regard to factors that might be expected to affect SEI, Aleamoni (1999) indicated that there is no relationship between SEI and class size, gender of student, time of day when

http://ijlter.org/index.php/ijlter

238

a course is offered, level of course, or rank of instructor. In addition, many reviews of SEI have concluded that gender roles have no effect on it, or where such effects exist, they are not significant (Liu, 2012; Radmacher & Martin, 2001).

2.3 Concerns about student evaluation of instruction Although many studies support the use of SEI in higher education, others express opposition to it. For example, some studies have revealed a gender bias against female faculty members in SEI, with students evaluating male and female faculty members based on different dimensions (Basow, 1995; Chamberlin & Hickey, 2001; Liu, 2012). Basow (1995) analyzed 2,000 SEIs collected from undergraduate students over four years and found that male faculty members scored much better than female faculty members on most questions most of the time, aside from one year when the women scored better on two criteria (i.e., sensitivity and student comfort). Another issue of bias is that female students consistently score their female professors higher than their male peers do (Centra & Gaubatz, 2000).

Overall, it is evident that the gender of faculty member and students has an effect on SEI. Another concern about SEI is the impact of teaching mode, in that, sometimes, faculty members receive disproportionately lower ratings in face-to-face courses compared to faculty members who teach online classes (Carle, 2009). Students studying online show more diverse opinions than students in face-to-face courses when scoring the delivery mode for effectiveness (Kelly et al., 2007; Liu, 2012; McGhee & Lowell, 2003; Rovai et al., 2006). Furthermore, class size has been found to have an influence on SEI. For example, many studies examining the impact of class size on SEI have found that higher SEI scores correlate with smaller classes (Badri et al., 2006; Liaw & Goh, 2003; Liu, 2012).

Another problem with SEI is the impact of evaluation instruments on results.

According to Landrum and Braitman (2008), SEI scores decrease significantly when the number of points on an evaluation scale are changed from 10 to 5. On the 10-point scale, students would use a range of values that is larger than on the 5-point scale. Other studies have examined subtle factors that impact SEI. For example, elective courses score better than compulsory ones (Marsh & Roche, 1997); SEI at the end of a semester can be significantly predicted by students’ first impressions of the instructor (Buchert et al., 2008); undergraduate students give lower ratings than graduate students do (Marsh, 2007; Whitworth, Price & Randall, 2002); the faculty member’s rank and experience influence SEI (Rovai et al., 2006); and faculty-member characteristics such as enthusiasm and humor can positively impact SEI (Obenchain, Abernathy & Wiest, 2001).

According to Centra and Gaubatz (2000), SEI can be biased because characteristics of students and instructors that are irrelevant to teaching may potentially affect ratings. For example, a class right at the start of the day might receive a worse score than the same class at a later, less awkward time. Concerns around the reliability of SEI need to be taken seriously, considering the influences of extraneous factors on SEI. When making decisions regarding faculty members, academic leaders should take care when using SEI results from undergraduate

http://ijlter.org/index.php/ijlter

239

courses or small classes. SEI of a small sample (i.e., in small classes) might not be accurate nor reflect the actual quality of faculty teaching. In order to make accurate decisions, the types of courses and the reasons that students took the courses should be considered. Another suggestion for increasing the reliability of SEI and reducing distortions is to ask students to rate the extent to which they have attained their educational objectives (McKeachie, 1997; Zhao & Gallant, 2012).

3. Statement of Purpose and Research Question Conducting research using student-satisfaction data is a common practice in higher education (Alkathiri, 2020). Moreover, SEI is a crucial aid to decision-making there. SEI is used in colleges and universities for various purposes. These include providing formative feedback to faculty for instructional improvement; measuring teaching effectiveness in order to make administrative decisions on career advancement; helping students choose classes and instructors; and for research on teaching (Zhao & Gallant, 2012). Furthermore, according to Algozzine et al. (2004), a major reason for universities to use SEI is to make decisions on salary. That said, when making decisions based on SEI, institutional administrators need to be aware of the various findings of the ongoing research concerning its validity and reliability. The present study was devised to investigate the views of heads of department concerning the use of SEI to facilitate decisions. The author investigated the attitudes of heads of department and the ways in which these affect their use of SEI. The primary research question was: Does the difference in views of heads of department, at a public university in the Eastern Province of Saudi Arabia, have a significant effect on their use of SEI to make decisions about individual faculty members and about academic departments? The author investigated the use of SEI results by heads of department to make decisions about: (1) individual faculty members (including on promotion and awarding of tenure, effectiveness of teaching, professional development needs, and contribution to student learning experience); and (2) academic departments (including on effectiveness of teaching, professional development needs, and contribution to student cohort learning experience).

4. Methodology The present study focused on heads of department at a public university in the Eastern Province of Saudi Arabia. Its purpose was to determine how significantly the differences between the heads of department in their views on SEI affect their use of it to make decisions about individual faculty members and academic departments. This study’s main hypothesis was that the tendency of heads of department to trust or not to trust the results of SEI makes no difference on their decision-making about individual faculty members or the academic department.

The criterion variable was their overall tendency to trust the use of SEI to make decisions. The author utilized a quantitative research design and a convenience sample of heads of department, who were asked to take an online questionnaire to provide data. Convenience sampling involves the sample being drawn from the population that is available to the researcher (Taherdoost, 2016). Analyzing and evaluating the data on two constructs, the author developed an evaluation scale to assess the respondents’ responses regarding their experiences using SEI

http://ijlter.org/index.php/ijlter

240

to make decisions about individual faculty members (i.e., construct A) and about their academic department (i.e., construct B).

4.1 Respondents An online survey was distributed to 112 heads of department from 20 colleges in four clusters (i.e., health, engineering, sciences and management, and arts and education). Fifty-seven heads of department completed the survey, which equates to a 50.89% response rate. Just under six out of every ten respondents were women. Table 1 displays the counts and percentages of respondents by category.

Table 1: Respondent demographic data Categories of participating heads of department Male Sex Female Health Engineering College cluster Sciences and management Arts and education

Overall sample count (n = 57) 24 33 8 19 15 15

%M 42.1 57.9 14.0 33.3 26.3 26.3

4.2 Procedure The researcher sent out an email to 112 heads of department from all clusters, asking them to take an online questionnaire. Respondents completed the questionnaire voluntarily. No compensation was offered for completing the questionnaire. Respondents were shown a consent form prior to taking the questionnaire. Respondents’ completion and submission of the questionnaire were used to indicate consent.

4.3 Instrument The author developed an online questionnaire of eight question items (see Appendix 1). The questionnaire asked respondents to indicate their attitude toward each item using a 6-point Likert-type scale with three levels of agreement and three of disagreement. The first item, “Overall, I tend to trust the SEI results to make decisions in my job,” was intended to identify the level of trust in general terms. The seven other question items were subscales of two constructs, A and B (see Table 2). The two main constructs were assessed at the interval level of measurement.

http://ijlter.org/index.php/ijlter

241

Table 2: Percentages of some form of agreement, mean scores, and standard deviations of scores for questions and constructs (strongly disagree = 1, strongly agree = 6) Item no.

1

2 (A) 2.1 2.2 2.3 2.4 3 (B)

3.1 3.2 3.3

Question Overall, I tend to trust the SEI results to make decisions in my job as a head of department As a head of department, I would use the SEI results to make decisions about the individual faculty members Individual faculty members’ promotion/awarding of tenure Individual faculty members’ effectiveness of teaching Individual faculty members’ professional development needs Individual student learning experience As a head of department, I would use the SEI results to make decisions about the academic department Academic department’s effectiveness of teaching Academic department’s professional development needs Student cohort learning experience

Some form of agreement (%)

M

SD

40.3

3.2

1.0

38.2

3.2

0.72

3.5

1.9

0.69

24.6

2.9

1.2

64.9

3.8

0.93

59.7 95.9

4.0 5.0

0.93 0.66

87.7

4.6

0.96

100

5.2

0.57

100

5.2

0.63

Construct A, concerning the use of SEI results to make decisions about individual faculty members, is measured by questions 2.1, 2.2, 2.3, and 2.4. Construct B, concerning the use of SEI results to make decisions about an academic department, is measured by questions 3.1, 3.2, and 3.3. Using the online questionnaire, two kinds of scores were calculated for each construct: subscale scores and an overall score. The subscale score for each question item from 2.1 onwards is the mean value calculated from all of its responses. The higher the subscale score for an item, the more likely the participating heads of department would be to use SEI to make a decision about the matter in question. For instance, the high M value on question 3.2 implies that the participating academic leaders are more likely to use SEI to make decisions about an academic department’s professional development needs. Finally, an average of the subscale scores for each construct gives its overall tendency score.

4.4 Analysis The researcher used an independent t-test to assess if there was an association between overall trust in results of SEI by heads of department (i.e., “Overall, I tend to trust the SEI results to make decisions in my job as a head of department”) and their willingness to use SEI to make decisions about individual faculty members

http://ijlter.org/index.php/ijlter

242

and overall academic departments. As mentioned above, to obtain the overall tendency score for each construct (A or B), the relevant subscale scores were averaged. Table 2 displays percentages of some form of agreement, mean scores, and standard deviations of scores for each of the items and constructs. The Cronbach alpha coefficient of internal consistency was computed to report the reliability and correlations for each of the constructs, as presented in Table 3.

Table 3: Correlation of constructs and measures of internal consistency Construct A

B

Question numbers 2.1, 2.2, 2.3, 2.4

3.1, 3.2, 3.3

Subscale construct Decisions about individual faculty members Decisions about overall academic department

A

α .76

.78*

.87

* p < .01.

5. Results It is remarkable that all of the responses to question items 3.2 and 3.3 indicated some form of agreement. Both concern construct B: using SEI results to make decisions about the academic department. The least agreement was with question item 2.1: using SEI to make decisions about an individual faculty member’s promotion and awarding of tenure.

Overall, the percentage of some form of agreement could be considered low for the questions within construct A (concerning individual faculty members), in contrast to the percentage of some form of agreement for the questions within construct B (concerning the academic department). The individual items within the constructs were averaged. Table 3 shows that the results of the survey have high reliability, as well as significant correlations between the constructs.

Based on the respondents’ answers to the first question, two groups of heads of department were identified: one group with a tendency to trust the use of SEI to make decisions (answering 4, 5, or 6), and another group with a tendency not to (answering 1, 2, or 3). The t-test results showed statistically significant differences between construct means for the two groups. Therefore, the study’s main hypothesis is rejected because the tendency of the respondents to trust or not to trust the results of SEI makes a significant difference on their decision-making about the individual faculty members or the academic department.

For construct A, respondents in the “trusting” first group had a mean of 3.85, whereas respondents in the “distrusting” second group had a mean of 2.68. The difference was statistically significant (t(55) = 9.749, p > .01). For construct B, respondents in the first group had a mean score of 5.58, whereas the second group’s mean score was 4.59. The difference was statistically significant (t(55) = 8.315, p > .01). Furthermore, Cohen’s effect size was computed, and alpha level was reported in Table 3.

http://ijlter.org/index.php/ijlter

243

6. Discussion According to Muammar and Alkathiri (2021), “higher education institutions are faced with a constantly evolving set of aims: to meet the needs of students while responding to societal demands and stakeholders’ expectations in a context of continually changing expectations, roles, and outcomes” (p. 1). Despite the challenges that heads of department face in their positions, they have a key role in the success of academic departments as well as the achievement of major higher education objectives (Freeman Jr., Karkouti & Ward, 2020; Reznik & Sazykina, 2017). The present study should make educators in higher education aware of how much heads of department vary in terms of their tendency to trust the results of SEI for making informed decisions.

Although the position of head of department is fundamental in higher education institutions to implement forward changes, the position is unattractive to many faculty members (Freeman Jr. et al., 2020). The expectations of heads of department can be ambiguous, especially that they receive limited training prior to assuming the position (Freeman Jr. et al., 2020). According to Freeman Jr. et al.

(2020), “[c]hairs must balance the dual responsibilities of managing faculty and student affairs who they support and evaluate as they implement the mandates from higher administration. Similarly, they shuttle between their managerial roles and faculty roles while balancing work-life demands” (p. 895). Figure 1 shows the difference among respondents regarding using student evaluation of instruction to make decisions about individual faculty members and the academic department on various purposes.

As a head of department, I would use the SEI results to make decisions about ... (% some form of agreement) 100 80 60

40 20

The Individual Faculty Member’s …

Student Cohort Learning Experience

Professional Development Needs

Effectiveness of Teaching

Student Learning Experience

Professional Development Needs

Effectiveness of Teaching

Promotion/Tenure Awarding

0

The Academic Department’s …

Figure 1: Form of agreement (%) by respondents regarding using student evaluation of instruction for various purposes

As seen in Figure 1, respondents tended to use SEI results to make decisions about individual faculty members’ professional development needs and about their contribution to the individual student learning experience. However, respondents were less likely to use SEI results to make decisions regarding the effectiveness of individual faculty members’ teaching or about promoting them or awarding them

http://ijlter.org/index.php/ijlter

244

tenure. According to Smith (2005), “heads of department are overloaded with work, … large departments are difficult to manage and … collegiality is the ‘preferred’ model of decision-making” (p. 463). On the other hand, respondents were more likely to use SEI results to make decisions about their whole academic departments on matters such as effectiveness of teaching, professional development needs, and the student cohort learning experience. Learning about the significant differences amongst heads of department concerning the use of SEI to make decisions will help academic leaders to address the issue. It is hoped that this will result in specific measures that facilitate a better understanding of these different views on SEI, and promote well-informed decisions in higher education.

Many studies have reported that, in general, SEI is a valid indicator of the quality of instruction (Marsh & Roche, 2000; Theall & Franklin, 2001). Many faculty members in higher education have exhibited reasonably positive attitudes toward the validity of SEI and its usefulness for improving instruction (Nasser & Fresko, 2002). However, others have expressed concerns about SEI and its uses (Nasser & Fresko, 2002) because of various factors that may cause biases (Badri et al., 2006; Kelly et al., 2007).

This study set out to investigate the views of heads of academic department concerning SEI, and the effect of their views on the ways in which they tend to use SEI when making decisions about individual faculty members and their overall academic department. The sample of heads of department who tended to trust SEI results showed more agreement with the use of SEI to make decisions, in comparison to those who tended not to trust SEI. The comparisons were statistically significant with respect to two constructs: using SEI results to make decisions about (A) individual faculty members and (B) overall academic department. Therefore, it can be concluded that whether or not heads of department trust SEI results has an effect on their use of such information when making decisions about faculty members under their supervision as well as about the academic department that they chair.

Although there were significant differences between the group who trusted SEI and the one who did not, the mean scores for each group on the use of SEI results to make decisions were lower where those decisions concerned individual faculty members as opposed to whole academic departments. Furthermore, the disparity between the two groups was biggest when it came to decisions that affect individuals. Therefore, it is clear that decisions cannot be made based solely on the results of SEI; rather, multiple sources of evaluation should be utilized.

Moreover, when evaluating the effectiveness of a faculty member, the multi-dimensional nature of SEI should be considered. Further research may include qualitative studies on the topic to further our understanding of the use of SEI by heads of department. Additionally, discipline-specific studies within certain academic departments or college clusters are recommended.

7. Conclusion Fifty-seven heads of department completed an online questionnaire with two main constructs: using SEI results to make decisions about (A) individual faculty

http://ijlter.org/index.php/ijlter

245

members and (B) overall academic department. The study aimed at investigating the ways in which heads of academic department use SEI to make decisions about individual faculty members and their academic department. The results revealed statistically significant differences between those heads of department who tended to trust the results of SEI and those who tended not to trust the results of SEI. The study concludes that there is a significant association between how heads of department perceive SEI and how they use it to make decisions about their academic department and individual faculty members.

In addition, the disparities within the groups of respondents, according to their attitudes of trust or distrust toward SEI, were greater with respect to issues or decisions that affect individuals as opposed to overall department. Therefore, decisions should not be made based solely on the results of SEI; rather, multiple sources of evaluation should be utilized to make proper decisions. Based on the findings of the current study, the author strongly suggests that academic leaders should use SEI across multiple years or courses in order to obtain more reliable information.

8. References Alderman, L., Towers, S., & Bannah, S. (2012). Student feedback systems in higher education: A focused literature review and environmental scan. Quality in Higher Education, 18(3), 261-280. https://doi.org/10.1080/13538322.2012.730714 Aleamoni, L. M. (1999). Student rating myths versus research facts from 1924 to 1998.

Journal of Personnel Evaluation in Education, 13, 152-66. https://doi.org/10.1023/A:1008168421283 Algozzine, B., Gretes, J., Flowers, C., Howley, L., Beattie, J., Spooner, F., Mohanty, G., & Bray, M. (2004). Student evaluation of college teaching: A practice in search of principles.

College Teaching, 52(4), 134-41. https://doi.org/10.3200/CTCH.52.4.134-141 Alkathiri, M. S. (2018). Using art-based techniques in faculty training programmes. In T. Chemi & X. Du (Eds.), Palgrave studies in business, arts and humanities: Arts-based methods and organizational learning (pp. 265–290). Palgrave Macmillan. https://doi.org/10.1007/978-3-319-63808-9_12 Alkathiri, M. S. (2020). Teaching and learning experiences of college students in Saudi higher education in the 21st century. The International Journal of Learning in Higher Education, 27(1), 15-30. https://doi.org/10.18848/2327-7955/CGP/v27i01/15-30 Arthur, L. (2009). From performativity to professionalism: Lecturers’ responses to student feedback.

Teaching in Higher Education, 14(4), 441-454. https://doi.org/10.1080/13562510903050228 Badri, M., Abdulla, M., Kamali, M., & Dodeen, H. (2006). Identifying potential biasing variables in student evaluation of teaching in a newly accredited business program in the UAE. International Journal of Educational Management, 20, 43-59. https://doi.org/10.1108/09513540610639585 Basow, S. (1995). Student evaluations of college professors: When gender matters. Journal of Educational Psychology, 87, 656-665. https://doi.org/10.1037/0022-0663.87.4.656 Buchert, S., Laws, E. L., Apperson, J. M., & Bregman, N. J. (2008). First impressions and professor reputation: Influence on student evaluations of instruction. Social Psychology of Education, 11(4), 397-408. https://doi.org/10.1007/s11218-0089055-1

http://ijlter.org/index.php/ijlter

246

Carle, A. C. (2009). Evaluating college students’ evaluations of a professor’s teaching effectiveness across time and instruction mode (online vs. face-to-face) using a multilevel growth modeling approach. Computers & Education, 53(2), 429-435. https://doi.org/10.1016/j.compedu.2009.03.001 Cashin, W. E. (2003). Evaluating college and university teaching: Reflections of a practitioner. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp. 531–593). Kluwer Academic. https://doi.org/10.1007/978-94-010-0137-3_10 Centra, J. A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. Jossey-Bass. https://eric.ed.gov/?id=ED363233 Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching?

Journal of Higher Education, 70(1), 17-33. https://doi.org/10.1080/00221546.2000.11780814 Chamberlin, M. S., & Hickey, J. S. (2001). Student evaluations of faculty performance: The role of gender expectations in differential evaluations. Educational Research Quarterly, 25(2), 3-14. https://search.proquest.com/docview/216185804?pqorigsite=gscholar&fromopenview=true Delvaux, E., Vanhoof, J., Tuytens, M., Vekeman, E., Devos, G., & Van Petegem, P. (2013).

How may teacher evaluation have an impact on professional development? A multilevel analysis. Teaching and Teacher Education Journal, 36, 1-11. https://doi.org/10.1016/j.tate.2013.06.011 Emery, C. R., Kramer, T. R., & Tian, R. G. (2003). Return to academic standards: A critique of students’ evaluations of teaching effectiveness. Quality Assurance in Education: An International Perspective, 11(1), 37-47. https://doi.org/10.1108/09684880310462074 Freeman Jr., S., Karkouti, I. M., & Ward, K. (2020). Thriving in the midst of liminality: Perspectives from department chairs in the USA. Higher Education, 80, 895-911. https://doi.org/10.1007/s10734-020-00521-6 Hendry, G. D., & Dean, S. J. (2002). Accountability, evaluation of teaching and expertise in higher education. The International Journal for Academic Development, 26(4), 327-414. https://doi.org/10.1080/13601440210156493 Kelly, H. F., Ponton, M. K., & Rovai, A. P. (2007). A comparison of student evaluations of teaching between online and face-to-face courses. Internet and Higher Education, 10, 89-101. https://doi.org/10.1016/j.iheduc.2007.02.001 Landrum, R. E., & Braitman, K. A. (2008). The effect of decreasing response options on students’ evaluation of instruction. College Teaching, 56, 215-217. https://doi.org/10.3200/CTCH.56.4.215-218 Liaw, S., & Goh, K. (2003). Evidence and control of biases in student evaluations of teaching. The International Journal of Educational Management, 17(1), 37-43. https://doi.org/10.1108/09513540310456383 Liu, O. L. (2012). Student evaluation of instruction: In the new paradigm of distance education.

Research in Higher Education, 53, 471-486. https://doi.org/10.1007/s11162-011-9236-1 Marsh, H. W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education (pp. 319–383).

Springer. https://doi.org/10.1007/1-4020-5742-3_9 Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52(11), 1187-1197. https://doi.org/10.1037/0003-066X.52.11.1187 Marsh, H. W., & Roche, L. A. (2000). Effects of grading lenience and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent

http://ijlter.org/index.php/ijlter

247

bystanders?

Journal of Educational Psychology, 92(1), 202-228. https://doi.org/10.1037/0022-0663.92.1.202 Massy, W. F., & French, N. J. (2001). Teaching and learning quality process review: What the programme has achieved in Hong Kong. Quality in Higher Education, 1, 34-45. https://doi.org/10.1080/13538320120045067 McGhee, D. E., & Lowell, N. (2003). Psychometric properties of student ratings of instruction in online and on-campus courses. New Directions for Teaching & Learning, 96, 39-48. https://doi.org/10.1002/tl.121 McKeachie, W. J. (1997). Student ratings: The validity of use. American Psychologist, 52, 1218-1225. https://doi.org/10.1037/0003-066X.52.11.1218 Muammar, O. M., & Alkathiri, M. S. (2021). What really matters to faculty members attending professional development programs in higher education. International Journal for Academic Development. https://doi.org/10.1080/1360144X.2021.1897987 Nasser, F., & Fresko, B. (2002). Faculty views of student evaluation of college teaching.

Assessment & Evaluation in Higher Education, 27(2), 187-198. https://doi.org/10.1080/02602930220128751 Obenchain, K. M., Abernathy, T. V., & Wiest, L. R. (2001). The reliability of students’ ratings of faculty teaching effectiveness. College Teaching, 49(3), 100-104. https://doi.org/10.1080/87567550109595859 Perry, R. P., & Smart, J. C. (Eds.). (2007). The scholarship of teaching and learning in higher education: An evidence-based perspective. Springer. https://doi.org/10.1007/1-40205742-3 Radmacher, S. A., & Martin, D. J. (2001). Identifying significant predictors of student evaluations of faculty through hierarchical regression analysis. The Journal of Psychology, 135(3), 259-268. https://doi.org/10.1080/00223980109603696 Reznik, S. D., & Sazykina, O. A. (2017). Head of a university department: Competence and new activity priorities. European Journal of Contemporary Education, 6(1), 126-137. https://doi.org/10.13187/ejced.2017.1.126 Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment and Evaluation in Higher Education, 30(4), 387-415. https://doi.org/10.1080/02602930500099193 Rovai, A. P., Ponton, M. K., Derrick, M. G., & Davis, J. M. (2006). Student evaluation of teaching in the virtual and traditional classrooms: A comparative analysis. Internet and Higher Education, 9, 23-35. https://doi.org/10.1016/j.iheduc.2005.11.002 Scott, G., & Hawke, I. (2003). Using an external quality audit as a lever for institutional change. Assessment and Evaluation in Higher Education, 28(3), 323-332. https://doi.org/10.1080/0260293032000059667 Seldin, P. (1999). Current practices—good and bad—nationally. In P. Seldin & Associates (Eds.), Changing practices in evaluating teaching: A practical guide to improved faculty performance and promotion/tenure decisions (pp.

1–24).

Anker. https://www.wiley.com/ensa/Changing+Practices+in+Evaluating+Teaching:+A+Practical+Guide+to+Impr oved+Faculty+Performance+and+Promotion+Tenure+Decisions-p9781882982288 Shah, M., & Nair, C. S. (2012). The changing nature of teaching and unit evaluations in Australian universities. Quality Assurance in Education, 20(3), 274-288. https://doi.org/10.1108/09684881211240321 Smith, R. (2005). Departmental leadership and management in chartered and statutory universities. Educational Management Administration & Leadership, 33(4), 449-464. https://doi.org/10.1177/1741143205056305

http://ijlter.org/index.php/ijlter

248

Spooren, P., Mortelmans, D., & Denekens, J. (2007). Student evaluation of teaching quality in higher education: Development of an instrument based on 10 Likert-scales.

Assessment & Evaluation in Higher Education Journal, 32(6), 667-679. https://doi.org/10.1080/02602930601117191 Taherdoost, H. (2016). Sampling methods in research methodology: How to choose a sampling technique for research. International Journal of Advance Research in Management, 5(2), 18-27. https://doi.org/10.2139/ssrn.3205035 Theall, M., & Franklin, J. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction? In M. Theall, P. C. Abrami, & L. A. Mets (Eds.), The student ratings debate: Are they valid? How can we best use them?

[Special issue of New Directions for Institutional Research, 2001(109), 45-56. https://doi.org/10.1002/ir.3] Wiley.

Tucker, B. M. (2015). The student voice: Using student feedback to inform quality in higher education (Doctoral dissertation).

Curtin University, Perth. https://espace.curtin.edu.au/bitstream/handle/20.500.11937/2158/234289_Tuc ker%20B%202015.pdf Tucker, B. (2014). Student evaluation surveys: Anonymous comments that offend or are unprofessional. Higher Education, 68(3), 347-358. https://doi.org/10.1007/s10734014-9716-2 Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A brief review.

Assessment & Evaluation in Higher Education, 23(2), 191-212. https://doi.org/10.1080/0260293980230207 Waschull, S. B. (2001). The online delivery of psychology courses: Attrition, performance, and evaluation.

Computers in Teaching, 28, 143-147. https://doi.org/10.1207/S15328023TOP2802_15 Whitworth, J., Price, B., & Randall, C. (2002). Factors that affect college of business student opinion of teaching and learning. Journal of Education for Business, 77, 282-289. https://doi.org/10.1080/08832320209599677 Zhao, J., & Gallant, D. J. (2012). Student evaluation of instruction in higher education: Exploring issues of validity and reliability. Assessment and Evaluation in Higher Education, 37, 227-235. https://doi.org/10.1080/02602938.2010.523819 Zineldin, M., Akdag, H. C., & Vasicheva, V. (2011). Assessing quality in higher education: New criteria for evaluating students’ satisfaction. Quality in Higher Education Journal, 17(2), 231-243. https://doi.org/10.1080/13538322.2011.582796

http://ijlter.org/index.php/ijlter

249

Appendix 1 An online questionnaire for heads of academic department regarding whether they use student evaluation of instruction to make decisions about individual faculty members and their academic departments.

I am …

(1) Male (2) Female I am a head of department in the following college cluster …

(1) Health (2) Engineering (3) Sciences and Management (4) Arts and Education 1. Overall, I tend to trust student evaluation of instruction results to make decisions in my job …

(1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 2. As a head of department, I would use the student evaluation of instruction results to make decisions about the …

2.1 Individual faculty members’ promotion/awarding of tenure (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 2.2 Individual faculty members’ effectiveness of teaching (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 2.3 Individual faculty members’ professional development needs (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree

http://ijlter.org/index.php/ijlter

250

2.4 Individual student learning experience (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 3. As a Head of Department, I would use the student evaluation of instruction results to make decisions about the …

3.1 Academic department’s effectiveness of teaching (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 3.2 Academic department’s professional development needs (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree 3.3 Student cohort learning experience (1) Strongly disagree (2) Disagree (3) Slightly disagree (4) Slightly agree (5) Agree (6) Strongly agree Thank you for your input.

http://ijlter.org/index.php/ijlter

251

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 251-269, February 2021 https://doi.org/10.26803/ijlter.20.2.14

Fostering Media Literacy Skills in the EFL Virtual Classroom: A Case Study in the COVID-19 Lockdown Period Marina Bilotserkovets, Tatiana Fomenko, Oksana Gubina, Tetiana Klochkova and Oksana Lytvynko Sumy National Agrarian University, Sumy, Ukraine https://orcid.org/0000-0003-4692-3444 https://orcid.org/0000-0002-3048-7097 https://orcid.org/0000-0002-3575-5898 https://orcid.org/0000-0002-1173-6211 https://orcid.org/0000-0002-2241-3776 Maryna Boichenko Sumy State Pedagogical University, Sumy, Ukraine https://orcid.org/0000-0002-0543-8832 Olena Lazareva Kharkiv National Pedagogical University, Kharkiv, Ukraine https://orcid.org/0000-0003-4385-0139

Abstract. This investigation highlights the ways and means of students’ formation of media literacy skills under the conditions of total and emergent distance learning in the lockdown period of the COVID-19 pandemic. The case study involved 138 first-year students from Sumy National Agrarian University, Ukraine, who studied English as a foreign language (EFL). Analysis, synthesis, and generalization of scientific data were conducted to determine the requirements and materials for the survey. Media literacy of the participants in the experimental group was developed through performance of a series of social media projects, critical analysis of social media texts, and creation of social media content. Pedagogical observation and expert estimation were employed to obtain qualitative results of partiucipants’ progress during practical classes and extracurricular activities. Psychological techniques and mathematical methods were employed to measure and assess the quantitative data of the experiment. The outcomes of the study revealed the positive dynamics of the development of reflective-evaluative, collaborative, and searching-creative skills of participants in the experimental group as well as improvement in their English proficiency.

The result of this study is potentially appropriate for educators who are

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

252

interested in the application of media technologies in foreign-language teaching.

Keywords: distance learning; media literacy skills; pandemic period; social media; teaching EFL

1. Introduction An emergent shift of the majority of educational institutions worldwide to distance learning in the virtual academic environment occurred due to the spread of the COVID-19 pandemic. Most students had to study remotely from their homes through the internet (Assalahi, 2020; Goh & Sandars, 2020). In the period of total lockdown, students and academics had to use media, especially social media, for supporting the learning process, which was facilitated through various digital means (Zoom, Google Meet etc.) and educational platforms (Moodle, Google Classroom etc.). They applied and presented the results of their scientific works through social media groups (Facebook, Instagram, Twitter etc.); online libraries and torrent sites were often the only sources of knowledge and information for learners (Marinoni et al., 2020; Ogbonnaya et al., 2020).

In terms of learning English as a foreign language (EFL), media sources are of utter importance as they provide students with a variety of information about linguistic discourses, language structures, pronunciation, and grammatical patterns. In addition, they serve as a guidance in social interactions and cultural values in particular language communities. However, the spread of digital content following COVID-19 exposed students to a vast amount of information through media, often false or inaccurate (Marinoni et al., 2020). More than ever before, students are in crucial need of the media literacy skills that would allow them to identify the nature of information they are surrounded by; find out if it is useful and credible; and protect themselves from misinformation and take control over what they read, listen to, or watch in the media.

The aim of the paper was to explore how students’ media literacy skills were developed for efficient application of social media resources in the EFL virtual classroom in the lockdown period of the COVID-19 pandemic. The objecives of the paper included revising recent scientific research to prepare the materials and requirements for the case study; implementing social media activities in the process of EFL training of the experimental group of student participants; and determining whether social media technologies have the potential to improve the EFL proficiency of students.

2. Literature review Nowadays, media in education have a stable position. Ytreberg (2002) investigated the role of media technologies in constructing an educational environment involving interactive learning and student-created content. The author pointed out that media in education suggest both cognitive and affective practices. Media provoke discussion, self-esteem, and assessment of values because media content mostly has a strong emotional impact (Ytreberg, 2002).

Furthermore, studies have shown that people learn abstract ideas and new

http://ijlter.org/index.php/ijlter

253

concepts more easily when they are presented in both verbal and visual form (Schmidt, 2012). Another essential function of media, and in particular social media, is the significant transformation of teacher-student interactions and innovative support of cooperation between students and academia (Chen & Bryer, 2012).

When social media sources are employed in foreign-language learning, students become co-authors of learning information content and partners and co-producers of the educational process. They thereby contribute to the establishment of a new kind of student-centered approach, “an approach that puts the student’s ability to communicate and produce content in the focus of the educational activities” (Pfeffer, 2014, p. 93). Moreover, as a means of communication, social media include teaching technologies that can greatly contribute to the creation of an authentic linguistic environment for foreign-language learners (He, 2019). Forming communicative competence through social media, students are presented with a scenario where they can produce their information content from media models, applying and imitating the foreign-language patterns and intonations that tend to be most similar to real communicative practices and situations (Mc Dermott, 2013).

However, if educators and students want to wholly benefit from media implemented in their learning process, they need to embrace their analytical and critical abilities to become media literate. Modern scholars have defined media literacy as a combination of knowledge and skills necessary for people to orient themselves in an information-based environment, search and share information, interact with other people and computer software, create safe and reliable media content, filter the media content, and solve cognitive tasks (Andriushchenko et al., 2020; Hatlevik et al., 2015). Analysis of recent scientific sources has shown that media literacy deals with the leveling of artificially created information structures and understanding of the principles of their creation. In addition, it deals with the ability of people to interpret the meaning of media messages based on personal experience and such individual characteristics as personal requests and expectations, formed national and gender ideas, social and cultural backgrounds etc. Media literate citizens are aware of different opinions and form their own position on current issues (Dvorghets & Shaturnaya, 2015; Pfeffer, 2014).

Being cross-disciplinary by nature (Schmidt, 2012), media literacy has been investigated from different points of view, but pedagogic research exploring the integration of media literacy and learning foreign languages is scant. A critical media literacy approach included in the foreign-language curriculum has been vectored by academia worldwide to develop learners’ critical reading of media content, thereby enabling their perception of media messages as a distorted version of reality. Moreover, this approach has encouraged and motivated students’ progress in foreign-language learning by involving their culture and interests amidst the learning process. Instructors intend to assist learners in their discovery of ties between language and social, cultural, and political spheres of its application, as well as materials and topics that have analogies in their native-

http://ijlter.org/index.php/ijlter

254

language culture by exploring own identities and divarication in comparison to other peoples (Reagan & Osborn, 2002).

A critical intercultural approach to teaching EFL has been implemented to develop the intercultural communication skills of students, and their critical thinking, social, and reflexive skills (Gómez Jiménez & Gutiérrez, 2019). The National Council of Teachers of English (NCTE) formulated the core components for media literacy necessary in the process of teaching EFL. These are the abilities to navigate, process, and synthesize multiple sources of information; compile, analyze, and critically evaluate multimedia texts; create and spread messages for virtual communities worldwide; start fruitful and respectful cross-cultural relationships with others by adopting the ethical responsibilities crucial for complex environments; and improve proficiency and skillfulness with digital technologies (ACTFL, n.d.).

Exploring the scientific works on media literacy implementation, we intended to identify the skills necessary to compose students’ professionally oriented communicative competence in EFL. An overview of the latest studies (Baglari et al., 2020; Bal & Bicen, 2017) revealed the following three types of skills that are crucial for teaching EFL using a media literacy approach: • skills to search and choose important information out of multiple diversified internet sources and assess its status, reliability, and acceptance; • skills to construct and present professionally oriented content in various media formats and genres, taking into consideration the target audience and algorithms of how media influence the application of tools; and • skills to collaborate and benefit from the participation in different digitally mediated research organizations, social networks, and virtual project groups for gaining professional knowledge and intercultural experience.

3. Methodology 3.1 The goal of the investigation This case study was conducted to answer the following questions: 1) What are the most commonly used social media amongst students and what are they utilized for?

2) What do educators need to do to develop students’ critical media literacy skills through learning English?

3) In what way does the critical media literacy approach affect the improvement of students’ proficiency in EFL? 3.2 Participants A sample of 138 students was determined for the survey. The participants voluntarily participated in this study. The experimental group included 70 first-year students, and the control group consisted of 68 first-year students. The experimental and control groups were selected in such a way that the controlled parameters did not differ significantly. The data concerning the features of the participants of the study are presented in Table 1.

http://ijlter.org/index.php/ijlter

255

Table 1: Participant features N 1.

2.

Feature Age Gender

3.

English language knowledge level

4.

Number of English classes per week Future specialties

5.

Experimental group 17–19 years old 46% female 54% male B1 (according to the international certification levels) 2

Control group 17–19 years old 45% female 55% male B1 (according to the international certification levels) 2

Agronomy, Food Technologies, Livestock Technologies, Management, Geodesy

Agronomy, Food Technologies, Livestock Technologies, Management, Geodesy

3.3 Limitations The major methodological limitation of the research was that the outcome depended highly on the context and case, along with the application of mostly exploratory and descriptive methods. The study was carried out at Sumy National Agrarian University (SNAU), Ukraine. Its findings are thus limited to the sample of participants and the institution where the research took place.

The research was conducted from March to July 2020. The time interval for this survey was limited to the mentioned term because that period involved exclusive and emergent distance learning, where students and academics worked under the conditions of the total lockdown, caused by the COVID-19 pandemic. During this period, EFL classes were delivered utilizing digital and social media facilities such as Zoom, Moodle, Google Classroom, Facebook, and Instagram.

The obtained data were verified by cross-checking and member-checking. The reliability factor for the questionnaire and diagnostic techniques used in the survey produced values from 0.85 to 0.91 (at p < 0.01), indicating the high reliability of the questionnaire items. The sample, consisting of 138 participants, determined the margin of error (at the 90% confidence level) for this research at about 7%.

The authors did not attempt to quantitate data on the linguistic knowledge and skills improvement, regarding it as a prospect of future study.

3.4 Methods and pedagogical conditions A set of theoretical methods was used to prepare the requirements and materials for the survey. This included analysis, synthesis, explanation, and generalization of the scientific data. Pedagogical observation of the participants and the expert estimation method were employed to obtain qualitative results. The survey was administered by lecturers and tutors who were watching over the participants’ progress during practical classes and extracurricular activities. Psychological techniques and mathematical methods were applied to measure and assess the results of the study.

http://ijlter.org/index.php/ijlter

256

The research work on implementing media literacy tools into the course of learning EFL in SNAU was implemented according to the following pedagogical conditions: 1) creating a relaxed atmosphere, mutual trust, and psychological comfort in teacher-student and student-student interactions; 2) appropriately using the polysemous nature of information and the principle of improvization; 3) encouraging different interpretations of information, recognizing the equality of viewpoints of all participants concerning the information; and 4) focusing on close ties with the socio-cultural environment, interests, and life experience of the participants.

3.5. Organization of the experimental work To answer the research questions, experimental work was conducted. The participants involved in the study were divided into an experimental and a control group. During the experiment, participants of both groups conducted distance learning through online learning platforms. The control group used their usual textbooks, grammar books etc. instead of social media materials, thus using no media literacy tools during their EFL classes.

Teaching EFL to the experimental group was based on authentic social media materials. Participants were asked to identify its multifunctional nature and specifics of its presentation to different public groups; and to research the role of social media networks in the academic environment and professional socialization, and what its educational potential is.

At the beginning of each lesson, a specific survey was adminstered which aimed at identifying the range of student tastes, personalized features of the academic groups, and student expectations from the lesson. Several tests were also conducted to explore if the participants were ready to work with the media; how well they knew the modern world of media (social and mobile networks, mass media, popular sites, TV and radio programs, specialized publications etc.); if they possessed the skills to assess media events; whether they were able to defend own opinions; their propensity to fall under the influence of ‘authorities’ etc. (see Appendix 1). Media literacy tools were implemented with different creative tasks that contributed to the acquisition of knowledge about the methods of perception and analysis of social media texts, the application of this knowledge in various professionally oriented situations, and the development of experience in creating competent and correct social media texts.

In the course of the experimental work, social media posts, TV and radio programs, and materials of various internet sites, on the same topic but from different sources, were discussed and compared in attempts to detect their specific features. Such discussions were built on the principle of antithesis and paradox, organized as a dialogue, conversation, or dispute, which allowed the identification of different points of view that existed amongst the participants on the same issue. The participants’ critical analysis abilities were elicited by

http://ijlter.org/index.php/ijlter

257

employing the following key questions, formulated for the study on the basis of recommendations by Mendoza (2018) and other experts in digital and media literacy: 1) Who is the author of the article (post) under review? This point enabled participants’ realization that all media content is created by a diversity of authors, each of them having a unique discourse, agenda, vision, and background.

2) What technologies are applied to draw the attention of the audience?

This questions helped participants to systematize their knowledge on various tools and means used by digital media to keep the audience interested in videos, commercials, or apps.

3) What multiple interpretations of one particular message are possible?

Here, the participants could reflect on the point that different communities can perceive a specific message in different ways.

4) Whose discourses, traditions, lifestyles, positions, and values are expressed or absent? Sometimes, not all perspectives and voices are represented, especially from certain strata of the society. It was essential to find out what views were missing and why.

5) What reason is this message sent for? This point was brought up so participating students could try to understand the author motives in the creation and sharing of messages and how they benefited from them.

Social media facilities, such as online platforms, forums, and chats, were used as educational tools in various learning activities and projects of the participants.

Participants were required to set out the solution of the problem, express their own opinion, or argue their position on a specific aspect of the problem. The participants discussed multifaceted aspects of professional activities in the information society and developed the insights necessary to critically assess the messages embodied in the social media content. In addition, they practiced the application of strategies for analyzing, reflecting, critiquing, and interpreting social media content that consisted of both visual and textual elements (in particular professionally oriented social networks).

Some participants did not feel confident enough to report in English in front of others. Therefore, the participant groups were divided into smaller subgroups, with their chat rooms, where they discussed matters related to the project. The activities of the subgroups were organized into two parts. First, each subgroup worked on gathering information on a particular topic, whereafter the subgroup members reported their findings to the whole subgroup. Second, the collective preparation of a creative task as a result of mastering certain learning material took place. Consequently, all subgroups presented their projects to the whole group, and the participants from the other subgroups wrote comments and questions based on this. It thus turned into a written dialogue, where participants who were otherwise not so talkative became engaged and took part in the discussion. The participants created multimedia presentations to show them for disputation regarding the coursework and posted them on social networks. Projects on the following topics were presented: “Academic environment socializing: Online versus offline”, “Security and balance in social

http://ijlter.org/index.php/ijlter

258

network maintenance”, “Fake news: Who cares and who profits?”, “Gender stereotypes presented in media” etc.

During the period of the study, three online trainings were held on the following topics: “Professional communities in a social media environment”, “Successful fundraising and projects development by means of social media platforms”, and “Media literacy skills of a student as a condition of his/her successful personal and professional self-realization”.

The following were achieved during the trainings with the participants as an interactive form of education: 1) becoming aware of interaction through social media in the professional sphere; 2) mastering skills to critically analyze information and messages received from media; 3) developing self-awareness and self-determination of their social positions; and 4) forming skills to critically evaluate information reproduced by social media.

4. Results It was found that most participants had more than one social media account. The outcomes of this aspect are presented in Figure 1.

100% 80%

YouTube Instagram Telegram

60%

Facebook

40%

Blogs

20%

Twitter

0%

TikTok

Figure 1. The most commonly used social media networks amongst participants

The most commonly used social media networks amongst the participants were YouTube (81%), Instagram (73%), Telegram (38%), and Facebook (27%). Fewer participants reported that they filmed video blogs or read or wrote blogs (25%), or used Twitter (18%) or TikTok (11%). These social media networks were used by the participants for educational purposes (to share organizational information, learning materials, and assignments or to discuss university-related issues in chats or comments); as well as for socialization, small talk, and entertainment, such as posting humorous photographs or video clips.

Since media literacy is perceived as a dynamic phenomenon that reflects the ability of individuals to effectively use the potential of the media environments for personal and professional purposes, consciously perceive and critically

http://ijlter.org/index.php/ijlter

259

evaluate information from media sources, as well as effectively apply media technology, three components of student media literacy were identified. These are reflective-evaluative, collaborative, and searching-creative media literacies.

The assessment of indicators was recorded at three basic levels – high, medium, and initial, which was fully consistent with the tradition of the pedagogical research. The corresponding results are given in Table 2.

Table 2: The dynamics of participants’ critical media literacy formation Level

High

Critical media literacy

Group

Medium

Initial

EG

CG

EG

CG

EG

CG

Reflectiveevaluative

Before study

18.1

17.3

30.2

34.0

51.6

48.6

After study

30.7

19.3

48.4

43.1

20.8

37.5

Collaborative

Before study

22.6

18.7

39.1

37.9

38.2

43.3

After study

37.2

18.4

56.1

38.9

6.6

42.6

Before study

18.7

11.4

42.8

42.1

38.4

46.4

After study

36.3

12.3

53.1

43.0

10.5

44.6

Searchingcreative

Percentage of students

The dynamics of the formation of participants’ reflective-evaluative media literacy component was measured by means of an instrument by Karpov and Ponomareva (2000), adapted for this study. It expressed the degree of formation of emotional states and attitudes. The instrument assigned points as follows: a) proficiently formed reflective-evaluative skills – 3 points; b) satisfactorily formed reflective-evaluative skills – 2 points; c) superficially formed reflectiveevaluative skills – 1 point; and d) a lack of reflective-evaluative skills – 0 points.

The changes in the levels of reflective-evaluative skills are presented in Figures 2a and 2b.

60 50 40 30 20 10 0

before after High

Medium

Initial

Levels of reflective-evaluative skills of students

a) Experimental group

http://ijlter.org/index.php/ijlter

Percentage of students

260

60 50 40 30 20 10 0

before after

High Medium Initial Levels of reflective-evaluative skills of students b) Control group Figure 2. Changes in the levels of reflective-evaluative skills of participants for the a) experimental group and b) control group

Percentage of students

The collaborative component, which measured the participants’ skills to collaborate and benefit from the participation in different digitally mediated entities, was determined using the authors’ instrument for the self-assessment of collaborative skills (see Appendix 2). The instrument assigned points as follows: a) proficiently formed collaborative skills – 3 points; b) satisfactorily formed collaborative skills – 2 points; c) superficially formed collaborative skills – 1 point; and d) a lack of collaborative skills – 0 points. The changes in the levels of collaborative skills are presented in Figures 3a and 3b.

60 50 40 30 20 10 0

before after

High

Medium

Initial

Levels of collaborative skills of students

a) Experimental group

Percentage of students

50 40 30

before after

20 10 0

High Medium Initial Levels of collaborative skills of students b) Control group Figure 3. Changes in the levels of collaborative skills of participants for the a) experimental group and b) control group

http://ijlter.org/index.php/ijlter

261

Percentage of students

The searching-creative component, which expresses a set of explorative searching-creative skills, was determined through a modified instrument of ‘Media research in professional activity’ (Kuzmina, 2011). The instrument assigned points as follows: a) proficiently formed searching-creative skills – 3 points; b) satisfactorily formed searching-creative skills – 2 points; c) superficially formed searching-creative skills – 1 point; and d) a lack of searching-creative skills – 0 points. The changes in the levels of searchingcreative skills are presented in Figures 4a and 4b.

60 50 40 30 20 10 0

before

after

High Medium Initial Levels of searching-creative skills of students

a) Experimental group Percentage of students

50 40 30 before

20

after

10 0

High Medium Initial Levels of searching-creative skills of students

b) Control group Figure 4. Changes in the levels of searching-creative skills of participants for the a) experimental group and b) control group According to Table 2 and Figures 2a, 2b, 3a, 3b, 4a, and 4b, we could trace the positive dynamics of media literacy formation amidst the participants from the experimental group, who were taught EFL based on social media materials through the implementation of social media tools. As for the participants of the control group, these indicators remained almost unchanged. Thus, the generalization of the research outcomes proved the effectiveness of the implementation of social media tools and means into the process of learning EFL by the participants.

The next stage of the study was carried out to investigate in what way mastering media literacy tools and means affected participants’ proficiency in EFL. The participants were administered a questionnaire to determine the prior factors that had contributed to the improvement in their EFL proficiency in the process

http://ijlter.org/index.php/ijlter

262

of practicing social media activities (see Appendix 3). The data from the processed questionnaires are presented in Table 3.

Table 3: The priority of factors that influenced the participants’ improvement in English proficiency by means of social media implementation N 1.

2.

3.

4.

5.

Factors that influenced participants’ improvement in English proficiency Increasing motivation Communication skills practice Authentic learning materials from social media ICT application in the process of learning Interactive teaching techniques

Priority for the participants (%) 92% 82% 76% 73% 58%

Most participants (92%) identified increased motivation for studying EFL as major factor influencing their improvement in English proficiency. They highly appreciated social media projects and searching activities, singling out the abundance of information on the English language sites necessary to prepare them for fully fledged professional activities. They also highlighted the importance of being able to critically analyze the available media resources and being capable of joining and collaborating with international professional virtual communities through social media. Communication skills practice was also identified as a prime concern by many participants (82%), who were positively disposed toward participation in Moodle and other social media forums. By communicating through social media, they could learn from each other by posting comments on other students’ forums and presenting blogs to their groupmates. Seventy-six percent (76%) of participants admitted that social media offered them the opportunity to learn different speech patterns and discourse information directly from native speakers. Almost the same number of participants (73%) stressed the possibility to learn EFL with the help of ICT, various gadgets, and software. Teaching by means of social media is interactive by nature, with a considerable number of participants (58%) regarding this factor as crucial for their EFL-knowledge and -skills improvement.

In conclusion, the improvement of participants’ EFL proficiency was greatly affected by the development of their reflective-evaluative, collaborative, and searching-creative skills. These skills served to motivate and enable them to utilize social media potential for achieving professional and personal progress.

5. Discussion The global academic community faced many challenge brought on by the COVID-19 pandemic. On the one hand, it was a test of students’ preparedness to learn independently through the internet, to cope with learning tasks and problems within virtual communities. On the other hand, it tested teachers and educational institutions’ awareness of and readiness for exclusive and emergent distance education and their ability to foster student skills essential for training and functioning in the virtual media environment.

http://ijlter.org/index.php/ijlter

263

Being representatives of Generation Z or centennials (Bubich et al., 2016), modern students are bound to information accessibility, capable of quickly processing considerable amounts of information and multi-tasking. They spend ample time on media and social networks for pleasure and learning (Brocca, 2020). The latest research investigating the academic process in the period of the COVID-19 pandemic has shown that students can often act as passive receivers of knowledge, preferring to accept and internalize any presented information (Assalahi, 2020). In one study, students were interviewed during the course of the study, claiming that they underwent learning without the permanent guidance of lecturers, thus searching for information and doing assignments on their own. Moreover, they regarded their studies ineffective without discussing learning material with a teacher and groupmates (Rahiem, 2020).

To deal with the pandemic problems, educators encouraged learners to take advantage of a wide range of virtual learning facilities. They utilized various social media sites, amongst them Facebook, Instagram, Telegram, WhatsApp, and WeChat, to apply synchronous or asynchronous classroom discussion, facilitate peer explanations using either voice notes or texts, and deliver assignments and quizzes. Lecturers were obliged to provide students with access to online materials, resources, links to digital libraries, repositories such as Open Educational Resources, internet streaming, torrents, or broadcasts.

Academia contacted students and delivered lessons through educational platforms such as Zoom, Google Hangouts, learning management systems (LMSs) etc. (Lapada et al., 2020; Rahiem, 2020). The circumstances caused by the pandemic have caused m-learning to become widespread, involving the utilization of mobile and portable IT devices. Significant features hereof include a focus on learning mobility and emphasis on the progressive adjustment of the mobile population by social institutions. Furthermore, the smartphone has been identified as the most popular gadget amongst young people (Lapada et al., 2020).

Such vast expansion of media, and social media in particular, into the domain of education stimulated “a social process in all educational contexts’’ and empowered ‘’building effective learning communities, where collective knowledge is created and advanced while supporting the growth of individual knowledge” (Lahiri & Moseley, 2015, p. 73). Therefore, it is essential for students to develop core media literacy skills. Amongst these skills should be the ability to pool knowledge amongst the abundance of information in the virtual environment and the ability to assess the trustworthiness and authenticity of information sources. In addition, students should possess the ability to analyze and synthesize found information and to participate in various online communities regarding and respecting diverse points of view, grasping and following alternative norms (Lantz-Andersson, 2016).

Forming the media literacy skills of EFL students by implementing social media resources into EFL courses is a stimulating and demanding task for the language teacher who seeks to master interdisciplinary knowledge aquistion whilst inspiring students to master it alongside them (Westman, 2019). Media content is

http://ijlter.org/index.php/ijlter

264

used in the EFL classroom to connect theoretical knowledge with real-life events and policies. However, to be aware of the true sense of these phenomena, students need to be media literate. The following are the purposes of using media in an EFL setting: 1) To develop different linguistic competences. Recent surveys have proven that students’ receptiveness toward the use of multiple social media tools for foreign-language learning enhances their experience of its application (Bilotserkovets & Gubina, 2019; Mc Dermott, 2013). In addition, it provides students with the training necessary to take computer-based tests of English, such as PTE Academic, TOEFL etc., to participate in telecollaboration in the field of language learning and online academic exchange programs around the world (Kobzhev et al., 2020). Lastly, it enables successful cross-cultural communication between students (Dvorghets & Shaturnaya, 2015).

2) To focus on the values and tendencies of the foreign-language society.

Gómez Jiménez and Gutiérrez (2019) noted that the implementation of social media tools within an educational context contributes to the creation of an advantageous academic environment, turning EFL classrooms into peculiar spaces where “different linguistic and cultural worlds come into contact” (p. 94). Through this, students’ linguistic and intercultural competences together with their media literacy skills are promoted.

3) To provoke the discussion of ideas reflected in the media content. In the process of incorporating media literacy tools into the EFL training process, students develop their critical thinking skills. They do this by studying the influence of subjective factors on the objectivity of media content and forming an interest in studying different sources of media information and comparing diverse opinions. It is important for every ordinary citizen, especially a young one, be able to critically comprehend media information and identify opportunities for the positive or negative impact of information disseminated by different types of media to prevent their possible manipulative impact (Van Den Beemt et al., 2020).

The outcomes of our study revealed that educators implementing media literacy tools should provoke students to participate in dialogue, teaching them not to be afraid to express their opinion, as well as to argue it, to form their critical thinking skills. This is essential because the media environment is both a field for producing meanings for the mass consciousness and a means of manipulating it (Ytreberg, 2002). Media literate students are supposed to be able to critically and consciously evaluate media messages. They need to be taught to ‘read between the lines’, identify the language of manipulation and persuasion, differentiating between the text and the subtext (Krakid et al., 2014). Besides, “ignoring what students do with a foreign language outside the classroom or refusing to engage with students in social media, teachers will never truly understand their needs and never fully realize the potential of social media as a language learning tool” (Tantarangsee et al., 2017, p. 476).

http://ijlter.org/index.php/ijlter

265

6. Conclusion The findings of the study revealed improvement in the experimental group’s media literacy skills and English-language proficiency compared to that of the control group. Being restricted to using mostly digital and virtual facilities because of the lockdown due to the COVID-19 pandemic, students and academics were able to express themselves and communicate through social media, search for information, and consciously perceive and critically interpret information obtained from different media. In addition, they were able to separate reality from its virtual imitation (i.e. understand the reality constructed by media sources), comprehend the manipulating tools they create, successfully use media technologies to solve professional problems etc. Concurrently, the positive dynamics of EFL-proficiency improvement was observed due to the growing motivation of applying English in real communication practice, authentic English learning materials, and ICT and interactive teaching technologies constructed into the academic process with the help of social media. However, beyond this study, additional experimentation is needed to quantitate and evaluate the progress of students in written and spoken English involved in social media and internet and web content evaluation activities via the use of critical questioning on the part of the students.

7. References ACTFL.

(n.d.). Literacy in language learning. https://www.actfl.org/guidingprinciples/literacy-language-learning Andriushchenko, K., Rozhko, O., & Tepliuk, M. (2020). Digital literacy development trends in the professional environment. International Journal of Learning, Teaching and Educational Research, 19(7), 55-79. https://doi.org/10.26803/ijlter.19.7.4 Assalahi, H. (2020). Learning EFL online during a pandemic: Insights into the quality of emergency online education. International Journal of Learning, Teaching and Educational Research, 19(11), 203-222. https://doi.org/10.26803/ijlter.19.11.12 Baglari, H., Sharma, M. K., Marimuthu, P., & Suma, N. (2020). Pattern of social media use among youth: Implication for social media literacy. Mental Health Addiction Research, 5, 1-5. https://doi.org/10.15761/MHAR.1000189 Bal, E., & Bicen, H. (2017). The purpose of students’ social media use and determining their perspectives on education. Procedia Computer Science, 120, 177-181. (9th International Conference on Theory and Application of Soft Computing, Computing with Words and Perception, ICSCCW 2017, 22-23 August 2017, Budapest, Hungary.) https://doi.org/10.1016/j.procs.2017.11.226 Bilotserkovets, M., & Gubina, O. (2019). Target language teaching by means of e-learning: A case study. Revista Romaneasca pentru Educatie Multidimensionala, 11(4), 17-29. https://doi.org/10.18662/rrem/154 Brocca, N. (2020). Sozial medien in bildung und fremdsprachdidaktik: Einleitung [Social media in education and foreign language teaching: An introduction]. heiEDUCATION Journal, 5, 9-23. https://doi.org/10.17885/heiup.heied.2020.5.24155 Bubich, O. A., Gilevich, E. G., Lushchinskoi, O. V., &. Savich, E. V. (2016). Clip thinking and organization of the pedagogical process at the university. Modeling of effective speech communication in the context of academic and professionally oriented interaction. BSU, 65-71. https://elib.bsu.by/bitstream/123456789/159350/1/6571.pdf

http://ijlter.org/index.php/ijlter

266

Chen, B., & Bryer, T. (2012). Investigating instructional strategies for using social media in formal and informal learning. The International Review of Research in Open and Distributed Learning, 13(1), 87-104. https://doi.org/10.19173/irrodl.v13i1.1027 Dvorghets, O. S., & Shaturnaya, Y. A. (2015). Developing students’ media literacy in the English language teaching context. Procedia – Social and Behavioral Sciences, 200, 192-198.

Goh, P. S., & Sandars, J. (2020). A vision of the use of technology in medical education after the COVID-19 pandemic.

MedEdPublish, 9(1). https://doi.org/10.15694/mep.2020.000049.1 Gómez Jiménez, M. C., & Gutiérrez, C. P. (2019). Engaging English as a foreign language students in critical literacy practices: The case of a teacher at a private university.

Profile: Issues in Teachers’ Professional Development, 21(1), 91-105. https://doi.org/10.15446/profile.v21n1.71378 Hatlevik, O. E., Ottestad, G., & Throndsen, I. (2015). Predictors of digital competence in 7th grade: A multilevel analysis. Journal of Computer Assisted Learning, 31(3), 220-231. https://doi.org/10.1111/jcal.12065 He, H. (2019). Media literacy education and second language acquisition. The International Encyclopedia of Media Literacy, 1-7. https://doi.org/10.1002/9781118978238.ieml0125 Karpov, A. V., & Ponomareva, V. V. (2000). Psychology of reflexive control mechanisms (p. 283). IPRAN. Kobzhev, A., Bilotserkovets, M., Fomenko, T., Gubina, O., Berestok, O., & Shcherbyna, Y. (2020). Measurement and assessment of virtual internationalization outcomes in higher Agrarian education. Postmodern Openings, 11(1Supl1), 78-92. https://doi.org/10.18662/po/11.1sup1/124 Krakid, A.-M., Skledar Matijevid, A., & Babovid, N. J. (2014, September 11-12). Seeing is (not) believing: Teaching media literacy through ELT. [Conference session]. In The International Language Conference on The Importance of Learning Professional Foreign Languages for Communication between Cultures, Maribor, Slovenia. https://bib.irb.hr/datoteka/803370.Kraki_Skledar_Matijevi_Jurina_Babovi.pdf Kuzmina, M. V. (2011). Diagnosis of forming of media culture of students in process of creation of educational video materials. Theory and Practice of Social Development, 8, 162-164.

Lahiri, M., & Moseley, J. L. (2015). Learning, unlearning and relearning with cutting edge technologies. International Journal of Learning, Teaching and Educational Research, 13(3), 62-78.

Lantz-Andersson, A. (2016). Embracing social media for educational linguistic activities.

Nordic Journal of Digital Literacy, 11(1), 50–77. https://doi.org/10.18261/issn.1891-943x-2016-01-03 Lapada, A. A., Miguel, F. F., Robledo, D. A. R., & Alam, Z. F. (2020). Teachers’ Covid-19 awareness, distance learning education experiences and perceptions towards institutional readiness and challenges. International Journal of Learning, Teaching and Educational Research, 19(6), 127-144. https://doi.org/10.26803/ijlter.19.6.8 Mc Dermott, G. (2013). The role of social media in foreign language teaching: A case study for French. Recherche et Pratiques Pédagogiques en Langues de Spécialité, 32(2), 141-157. https://doi.org/10.4000/apliut.4234 Marinoni G., Van Land, H., & Jensen, T. (2020). The impact of Covid-19 on higher education around the world.

IAU Global Survey Report.

http://ijlter.org/index.php/ijlter

267

https://www.unibasq.eus/wp-content/uploads/2020/06/iau_covid19_and_he _survey_report_final_may_2020.pdf Mendoza, K. (2018, November 19). 5 Questions students should ask about media.

Common sense education. https://www.commonsense.org/education/blog/5questions-students-should-ask-about-media Ogbonnaya, U. I., Awoniyi, F. C., & Matabane M. E. (2020). Move to online learning during COVID-19 lockdown: Pre-service teachers’ experiences in Ghana.

International Journal of Learning, Teaching and Educational Research, 19(10), 286-303. https://doi.org/10.26803/ijlter.19.10.16 Pfeffer, T. (2014). Academic media literacy and the role of universities. Perspectives of Innovations, Economics and Business, 14(2), 83-93. https://doi.org/10.15208/pieb.2014.10 Rahiem, M. D. H. (2020). The emergency remote learning experience of university students in Indonesia amidst the COVID-19 crisis. International Journal of Learning, Teaching and Educational Research, 19(6), 1-26. https://doi.org/10.26803/ijlter.19.6.1 Reagan, T., & Osborn, T. (2002). Critical curriculum development in the foreign language classroom. In T. G. Reagan & T. A. Osborn (Eds.), The foreign language educator in society: Toward a critical pedagogy (pp. 70-81). Coursework.

Schmidt, H. (2012). Media literacy education at the university level. The Journal of Effective Teaching, 12(1), 64-77. https://files.eric.ed.gov/fulltext/EJ1092140.pdf Tantarangsee, C., Kosarassawadee, N., & Sukweses, A. (2017). The use of social media in teaching and learning: A case of SSRU’s faculty members. International Journal of Innovation, Management and Technology, 8(6), 471-476. https://doi.org/10.18178/ijimt.2017.8.6.773 Van Den Beemt, A., Thurlings, M. & Willems, M. (2020). Towards an understanding of social media use in the classroom: A review. Technology, Pedagogy and Education, 29(1), 35-55. https://doi.org/10.1080/1475939X.2019.1695657 Westman, P. (2019). Using critical media literacy to support English language teaching and practice.

Revista Lusófona De Estudos Culturais, 1(1). https://doi.org/10.21814/h2d.242 Ytreberg, E. (2002). Erving Goffman as a theorist of the mass media. Critical Studies in Media Communication, 19(4), 481-497. https://doi.org/10.1080/07393180216570

http://ijlter.org/index.php/ijlter

268

Appendix 1 Questionnaire “Social media in students’ life” Dear students, please answer the questions. The questionnaire will be anonymous. The value of the research depends on how openly and in detail you answer the questions.

Thank you in advance for your participation!

Section 1: Kinds of social media 1. What kind of social media do you usually use?

2. What purposes do you mostly use social media for?

Choose the activities mostly you do on social media.

Activities

The activities you do most often (√)

communication with classmates, friends, relatives making new acquaintances searching for interesting information posting some ads reading blogs reading the news searching interest groups using multimedia (viewing photos, movies, listening to music) uploading photos learning languages communication with foreigners

Section 2: Social media use for educational purposes 1. What kinds of social media do you use for educational purposes?

2. How frequently do you use social media for study?

3. What kinds of social media do you use in learning English?

4. How often do you use social media in learning English?

Appendix 2 Questionnaire “Self-assessment of media competence skills” Dear students, please evaluate your ability and readiness (on a five-point scale, where: 1 – no, 2 – rather “no” than “yes”, 3 – rather “yes” than “no”, 4 – yes, 5 rather obvious) to work with social media: Skills to benefit from the virtual participation in a team work to interact with other team members in the virtual social environment to explore and acquire new roles, such as mentors, mediators and group leaders in a virtual group to exchange practical and academic information, experiences, social support with other participants

http://ijlter.org/index.php/ijlter

1

2

3

4

5

269

to develop learning abilities through virtual peer collaboration to achieve mutual understanding among the members of the group to develop more comprehensive understandings of learning topics through discussion group chats to plan the schedule and the algorithms of the team work in a virtual group to validate a virtual group activities to assess the outcomes of the participation in a virtual group

Appendix 3 Questionnaire “Social media activities in learning English” Section 1: What social media activities do you usually use for learning English?

Dear students! Please, point out the priority of social media activities that had contributed to your English language improvement.

Social media activities for learning English

The prior social media activities (√)

searching for information use of authentic learning materials reading news communication with foreigners social media projects participation in the Moodle and other social media forums

Section 2: How did the implementation of social media help you in learning English?

Dear students! Read the suggested statements carefully. Assess their significance to you.

The factors that influence the students’ English proficiency improvement Increasing motivation Communication skills practice Authentic learning materials from social media ICT application in the process of learning Interactive teaching techniques

http://ijlter.org/index.php/ijlter

(√)

270

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 270-289, February 2021 https://doi.org/10.26803/ijlter.20.2.15

Analysis of Future Teachers’ Perceptions on the Evaluation of Learning: A Case Study of Chile José Hernández Sepúlveda Universidad Católica del Maule, Talca, Chile https://orcid.org/0000-0002-6168-4368 Rodrigo Panes Chavarría Universidad del Bio Bio, Chillan, Chile https://orcid.org/0000-0002-2400-6701 Karla Rosalía Morales Mendoza Universidad Católica del Maule, Talca, Chile https://orcid.org/0000-0001-5807-3876 Abstract. Practising teachers and future Chilean teachers show weaknesses in developing learning evaluation processes efficiently. The polysemic concept of evaluation in Initial Teacher Training (FID, Spanish acronym for Initial Teacher Training) is one of the main difficulties for the students of pedagogy as well as their limited possibilities to implement practical evaluation processes. In this context, the present work analyses the perceptions of 189 students from different pedagogy careers (Chile) on the evaluation of learning. The current study is descriptive research with a quantitative approach in which a questionnaire was designed and used as research instrument to address the conceptual, functional and experiential dimensions of the evaluation. The results indicate that the perceptions of the evaluation of the students are up to date and according to specialised literature.

Likewise, the results show that the training experiences are based on traditional perceptions, such as the control and value given to the evaluation instrument. However, there were statistically significant differences between the groups which were measured by the KruskalWallis H test. The projection of the study leads us to suggest that for the training of future teachers, instances of reflective action in the evaluation should be considered, based on professional practices. The recognition of educational communities and their environments is necessary to give rise to an evaluative praxis that relates the perceptions of students to the practical implementation of the evaluation processes to have a favourable impact on the training of future teachers.

Keywords: evaluation; initial teacher training; perception; teachers

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

271

1. Introduction The evaluation of learning over time has increased in complexity owing to the great variety of conceptions, functions, and purposes that have been incorporated into this process. Currently, it is not easy to conceptualise the term because the idea of evaluation causes some confusion in educational institutions. The term might also mean to measure a student's performance, to grade performance, to select the best, to improve teaching, and to build learning, among others (Santos Guerra, 2003; Stobar, 2010). Therefore, it can be said that the concept of learning evaluation is a polysemic term (Castillo & Cabrerizo, 2010) since in the educational field, although different notions can be valued, in all of them there are common and valid elements to construct a definition (Santos Guerra, 2003; Escudero, 2003). For the present study, it is important to contextualise and illustrate the evolution of learning evaluation.

For this purpose, a theoretical analysis of the moments and movements contributing to the development of the evaluation over time is carried out. This describes the results of teachers and future teachers quantitatively corresponding to the Teaching Portfolio and National Diagnostic Assessment Initiates Test as applied by the Centre for Improvement, Experimentation and Pedagogical Research (henceforth, CPEIP).

2. State of the Art: Evolution of the Evaluation At the end of the 19th century and the beginning of the 20th century, the evaluation of learning focused fundamentally on standardised tests to measure all kinds of school skills. This was done to determine the position that future teachers obtained within the group norm, generalising the results with a common pattern, identifying the differences and/or inequalities between groups, which in practice only focused on measurement (Fernández, 1981; Escudero 2003).

In the middle of the 20th century, Tyler (1950) made a significant advance in the development of evaluation. From the behavioural approach, he overcame mere psychological evaluation by relying on a more methodical and organised perspective of the evaluation process. His approach was focused on the use of clearly defined objectives through the construction and use of appropriate instruments to obtain information. This was to determine to what extent learning was achieved in the teaching programmes and curricula, generating an important change in the conception of evaluation that existed at that time (Escudero, 2003; Castillo Cabrerizo, 2010).

During the 1960s, the evaluation of learning evolved again since it not only had to focus on the learning objectives that students achieved, but it was also necessary to consider collecting information on a comprehensive educational programme (Cronbach, 1963). This perspective of evaluation is consistent with the incorporation of the interpretive paradigm in which evaluative practice becomes a process that enables establishing the merit or value of what is evaluated. Moreover, at this time the concepts of summative and formative assessment were incorporated (Scriven, 1967). Summative assessment is understood as a reductionism of the educational process, given that the

http://ijlter.org/index.php/ijlter

272

emphasis is on measuring or grading learning, while formative assessment seeks the student's self-regulation as it favours the learner's processes of reflection on his or her own training process.

Starting in the 1970s, there was a proliferation of evaluation models characterised by conceptual, methodological and paradigmatic plurality.

Authors such as Guba and Lincoln (1982) indicated that there were more than 40 evaluation models which pointed to two areas. The first considered the students and the methods, while the second focused on the change that occurred with the students based on the educational action according to the formulation of previously designed learning objectives. At the same time, well-known taxonomies, developed mainly by authors such as Bloom (1969), Gagné (1971) and Mager (1973), were incorporated into the evaluation.

During the 1970s there were two periods with clear conceptual and methodological differences. One period followed the line of Tyler, which was based on the achievement of predefined objectives and was supported by the positivist paradigm, while the other alternative models were based on the constructivist interpretive paradigm. Among the best known are the responsible evaluation of Stake in 1975, the democratic evaluation of Macdonald in 1976, the illuminative evaluation of Parlett and Hamilton in 1977 and the evaluation as an art critic of Eisner in 1985, among others (Alcaraz, 2015; Carbajosa, 2011; Escudero, 2003).

More recently, at the end of the 20th century and the beginning of the 21st century, the evaluation of learning again faces a process of evolution, since after the Bologna treaty (1999), the curriculum by objectives used for years in professional training was replaced by a curriculum based on competency. This new approach is not exempt from criticism because for some authors, the idea of a curriculum by competencies corresponds to normative instruments with which the convergence of school systems is sought (Sacristán et al., 2008). Researchers who oppose the new training model consider that it takes competencies as a reference for the structure of a globalised curriculum in which these will be used to choose procedures and proposals for evaluation.

The change in the training approach presented a great challenge in higher education, both in the design and in the development and evaluation of the professional training processes (Tejada & Ruiz, 2016). In the field of evaluation, authors such as Dochy, Segers and Dierick (2002) argued that the greatest difficulty for the universities is to move from a culture of examination to a culture of evaluation, where evaluative practices are developed with a formative idea. The second one aimed to improve the learning of students and involve them in the evaluation processes (Cano, 2008; Ruay, 2019; Stobart, 2010; Tejada & Ruiz, 2016). Bonsón and Benito (2005) considered that it is necessary to advance from the evaluation of learning to the evaluation for learning, or as the authors Carless, Joughin and Mok (2006) and Bound and Falchikov (2007) put it, to a learning-oriented assessment.

http://ijlter.org/index.php/ijlter

273

In this new evaluative framework, the competencies that the students of pedagogy careers managed to acquire in the dimension of educational activities were considered. This is because the evaluation of learning is one of the fundamental activities that are developed in educational organisations; therefore, it is considered a basic teaching competence (Perrenoud, 2004; Zabalza, 2003). Additionally, the specialised literature shows that evaluative practices are indisputably instances of improvement of learning as well as of teaching quality (Black & William, 1998; Ruay, 2019; Tejada & Ruiz, 2016).

3. Competency Assessment as an Improvement Alternative The incorporation of the competency model in teacher training involves a great change in teaching since there is a transition from teaching to learning, where the evaluation processes, the design, and development of the professional teaching construction are taken into consideration. This is because it requires an adequate alignment between the curricular pedagogical designs, the methodology, and the evaluation of teacher training without losing sight of the graduation profiles and the standards associated with the individualities of each training process (Biggs, 2010; Moreno Oliver, 2014; Yáñiz & Villardón, 2006, as cited in Tejada & Ruiz, 2016).

Consequently, evaluation from the competency approach acquires meaning, significance and relevance according to the conditions in which it is evidenced (Tejada & Ruiz, 2016). That is why the actions that are implemented must approach real problems, where contextual situations are a source of specification of competencies with the student as the protagonist (Salazar-Gómez & Tobón, 2018).

In Initial Teacher Training (hereafter, FID) it is not always possible to maintain the connection with the professional scenario; therefore future professionals must make/adopt? approaches based on the simulation of problems or situated reality (Littlewood, 2011; McGaghie et al., 2010; Ros & Conesa, 2013). This is to be done so that the actions are carried out from an authentic evaluation in which feedback to the student prevails, as well as an assessment of the performance achieved (Tejada & Ruiz, 2016). The training process is complex and multidimensional; therefore it needs to be evaluated in different ways (Fernández, 2010), putting aside the old culture of evaluation that focused on written and oral tests based on the management of rote knowledge and the generalisation of the results.

This new perspective of evaluation involves various evaluative strategies and instruments so that the necessary background information can be collected about the acquisition of generic and specific competencies to assess the results that students in training are expected to achieve in concordance with the planning and programming of learning for their professional teaching development. In other words, evaluation by competencies is a complex process that must be oriented to the student's action, based on real or simulated work situations that favour tasks of authentic evaluation (Del Pozo, 2013; AshfordRowe, Herrington & Brown, 2014). The evaluation by competencies also focuses

http://ijlter.org/index.php/ijlter

274

on the fact that the learners manage to integrate the repertoire of knowledge, abilities, and aptitudes to solve the problems that demand acting professionally, favouring the learning process and not only measuring the meaningless results.

On the other hand, assessment by competencies has different perspectives, including assessment of learning, for learning, as learning and from learning.

Some authors such as Brown and Pickford (2013), Ion, Silva and Cano (2008), Nicol, Thomson and Breslin (2014) and Villardón (2006), cited in Tejada and Ruiz (2016) consider that the meaning is given to the classical formative and summative evaluation from a complementary approach in the way that the first (formative) corresponds to the development of competencies. Therefore, it emphasises learning activities and their permanent improvement, continuous feedback, reflection on the process, and self-evaluation. However, in the second (summative) evaluation, competency is evaluated by connecting it with performance, levels of achievement, and evidence (Tejada & Ruiz, 2016).

From the perspective of the development of evaluation by competencies, the educational system in general and the FID in particular are obliged to innovate and reformulate their pedagogical and evaluative practices. Since the competencies arise as a response to the need to articulate the knowledge that future teachers must possess to face the workplace, this undoubtedly leaves evaluation at a didactic crossroads. This is because the implementation is not immune to the tensions and challenges that new teaching practices seek to address (Cano, 2008).

For authors such as Scriven (2007), Hall and Burke (2008) and Kaftan, Buck and Haack (2006), evaluation has its essence in the formative as an integral learning process as long as it establishes a formative design that enhances the educational process and the construction of new knowledge. Therefore, the evaluation training that students receive in their initial training must also contribute to the development of self-learning and self-regulation so that students are aware of their practices and learning (Ríos & Herrera, 2017).

The evaluation by competencies should be oriented to the development of learning for life, relating knowledge, knowing how to do and knowing how to be, since at this point the different learning experiences intersect from a transversal perspective, highlighting the educational daily life as the main setting (Jaimes & Callejas, 2009). Likewise, Tobón (2004) stated, “the evaluation of competencies must integrate the qualitative with the quantitative, because with words it cannot be measured, and with numbers, it cannot be understood or explained”, which forces us to think that both contribute to the improvement of teaching and learning processes. Bolívar (2008) cited in Cano (2008): “The best way to evaluate competencies is to put the subject before a complex task, to see how he/she manages to understand and solve it by mobilizing knowledge. The evaluation instruments used cannot be limited to tests to see the degree of mastery of content or objectives, but rather propose complex situations, belonging to the family of situations defined by the competencies, which will also require a complex

http://ijlter.org/index.php/ijlter

275

production by the student to solve the situation, since he/she needs knowledge, attitudes, metacognitive thinking.” (p. 184) In this way, it can be clarified that the evaluation by competencies is a global construct that must be designed and structured to enhance the educational process, establishing criteria that allow appreciating and demonstrating the learning and performance obtained by students in training. It corresponds to concern about the effectiveness of the activities designed in the study programmes to ensure the fulfilment and achievement of the students' learning regarding each graduation profile (Cano, 2008).

Finally, the evaluation by competencies must be understood from the training point of view to favour the permanent improvement of the teaching and learning processes. Likewise, this should promote self-regulation and selflearning of students through responsibility, criticism and self-criticism of their own educational process (Stobart, 2010; Tejada & Ruíz, 2016). Equally, it is necessary to consider that the constant improvement of learning is the ultimate goal. Therefore, students are the main actors in evaluative practices and actions that when analysed with teachers in dialogic spaces, allow students to become aware of the level reached.

4. Teacher Evaluation System and Future Teachers in the Chilean National Context The Ministry of Education, through the CPEIP, developed teacher evaluation with the objective of strengthening the teaching profession and contributing to improving the quality of education in Chile. The results published in 2018 with the item titled “Evaluation and Correction Guideline Used” indicate that 34% of all the teachers evaluated at the national level were in the category of ‘Competent and Outstanding’. The remaining 64% were in the category of ‘Unsatisfactory and Basic’. Later in the item, ‘Relationship between evaluation and objectives’ indicates that 44% of the teachers were competent and outstanding, and 56% were unsatisfactory and basic. Finally, in the item ‘Analysis and use of evaluation results’ only 17% of teachers were competent and outstanding, and 83% were unsatisfactory and basic (Ministerio de Educación, 2019a).

On the other hand, the results also highlighted that the best-evaluated teachers, that is, those who achieve competent and outstanding performances, show low results in the tasks that correspond to the evaluation of learning and reflection from the results (Table 1).

http://ijlter.org/index.php/ijlter

276

Table 1: Portfolio performance in competent and outstanding teachers Evaluated aspect

Total % proficient outstanding teachers

Description of the competition

Proposes instructions, questions or tasks in the evaluation that are clear, and in its correction guideline, it correctly identifies the expected performance Performs assessment activities that address Relationship between evaluation and are consistent with all of the learning and objectives objectives you set out to measure Manages to recognise the influence of the Accountability for pedagogical decisions, both in the learning results achieved and not achieved Demonstrates knowing the characteristics of Analysis based on the students and incorporates them when the characteristics planning or conducting his/her classes, of your students seeking to promote learning Identifies the difficulties that their students Formative use of present during the learning process. Reflects learning on them and the way to approach them difficulties formatively Provides feedback to the students, allowing them to learn from their own performance Feedback to your since it encourages them to complement students their answers, analyse the steps they followed to arrive at a result or identify the reason for their successes or errors Source: Mineduc (Ministry of Education, Chile, 2019a) Evaluation correction guideline

of &

and

41.8%

51.1%

45.0%

47.2%

60.9%

26.7%

On the other hand, in the case of future teachers, the results of the National Diagnostic Evaluation of Initial Teacher Training (END - FID) of 2018 (CPEIP, 2018) showed that in the standard called ‘Knowing how to apply evaluation methods to observe the progress of students and knowing how to use the results to provide feedback on learning and pedagogical practice’, the future teachers of pedagogy careers who participated in the study did not have the necessary knowledge to achieve optimal performance according to the applied instrument (Ministerio de Educación, 2019b). (Table 2).

Table 2: National diagnostic evaluation in initial teacher training Pedagogy career

Pedagogy in English

Pedagogy in English

Performance standard 3.0 Knows the curriculum of Secondary Education and uses its various curricular instruments to analyze and formulate pedagogical and evaluative proposals 6.0 Knows how to apply evaluation methods to observe the progress of students and knows how to use the

http://ijlter.org/index.php/ijlter

% correct

% Min

% Max

50.4%

0.0%

80.0%

58.9%

33.3%

69.1%

277

results to provide feedback on learning and pedagogical practice 3.0 Knows the curriculum of Pedagogy in Secondary Education and uses its Spanish various curricular instruments to 58.7% 35% 69.1% Language and analyse and formulate pedagogical Communication and evaluative proposals 6.0 Knows how to apply evaluation Pedagogy in methods to observe the progress of Spanish students and knows how to use the 61.7% 40.7% 73.1% Language and results to provide feedback on Communication learning and pedagogical practice 3.0 Knows the curriculum of Secondary Education and uses its Pedagogy in Physical various curricular instruments to 42.3% 30.0% 53.9% Education analyse and formulate pedagogical and evaluative proposals 6.0 Knows how to apply evaluation methods to observe the progress of Pedagogy in Physical students and knows how to use the 50.1% 28.6% 62.7% Education results to provide feedback on learning and pedagogical practice 3.0 Knows the curriculum of Secondary Education and uses its Pedagogy in Religion and various curricular instruments to 58.7% 30.0% 76.0% Philosophy analyse and formulate pedagogical and evaluative proposals 6.0 Knows how to apply evaluation methods to observe the progress of Pedagogy in Religion and students and knows how to use the 55.6% 44.9% 66.7% Philosophy results to provide feedback on learning and pedagogical practice 3.0 Knows the curriculum of Secondary Education and uses its Pedagogy in Differential various curricular instruments to 46.6% 33.3% 61.5% Education analyse and formulate pedagogical and evaluative proposals 6.0 Knows how to apply evaluation methods to observe the progress of Pedagogy in Differential students and knows how to use the 46.0% 33.3% 56.9% Education results to provide feedback on learning and pedagogical practice Source: National Diagnostic Evaluation in Initial Teacher Training (Ministerio de Educación, 2019b).

The results of the national diagnostic evaluation in the FID showed that the students of pedagogy careers obtain low performances in the standards that are associated with the evaluation of learning, a situation that is repeated in the professional field. The teachers of the Chilean educational system also obtained low results in the evaluation task framed in the portfolio of the Teacher Evaluation (CPEIP, 2018; Teacher Performance Evaluation System, 2019). There is a need to analyse the perceptions of final-year students of the pedagogy

http://ijlter.org/index.php/ijlter

278

careers of the Catholic University of Maule about the evaluation of learning and how their evaluative experiences have been owing to many reasons. Some of these are the following: • The profound transformations that the concept of evaluation and its functions have undergone throughout history; • The low results obtained by pedagogy students and teachers of the educational system in the field of learning evaluation; • The incorporation of a new training model for the evaluation programmes of university pedagogy careers; and • The scarce presence of scientific studies on the ideas that future teachers have regarding the evaluation of learning in Chile.

5. Materials and Methods According to the objectives set, the approach that fits the study is quantitative (Hernández, Fernández & Baptista, 2010). The chosen design corresponded to the non-experimental, descriptive exploratory survey type (Tejedor, 2000; Colás, 1994). Such decisions make it possible to inquire about a little-studied topic and provide an important source of information to learn about perceptions of and possible differences between groups of students.

5.1 Research Instruments To assess the perceptions of the students, a Likert-type questionnaire with two differentiated parts was designed. In the first one, information was requested on demographic variables (gender, age, degree, grade). In the second part, and to collect information related to the objectives of this study, 31 items were presented, distributed in three dimensions (Table 3), where the respondents marked their degree of agreement from 1 (totally disagree) to 5 (totally agree).

Table 3: Psychometric characteristics of the instrument No. of dimensions I dimension

Name of the dimension

Perceptions about learning assessment Perception of the role of evaluation in II dimension teaching and learning processes Perception of evaluation during the III dimension training process and professional practices Source: Own elaboration

No. of descriptors/ dimension 10 (ten) 14 (fourteen) 7 (seven)

For the elaboration of the instrument, a review of specialised literature was carried out, giving significance to the instruments’ items. Subsequently, it was evaluated by five experts from the Universidad Católica del Maule based on the sufficiency, clarity, and relevance of each of the items and the contributions of which indicate a C.V.I. (Content validity index) corresponding to 0.9. After evaluation by experts, the survey was subjected to a first analysis to estimate its reliability. It yielded a Cronbach's alpha coefficient of .760, indicating good internal consistency.

http://ijlter.org/index.php/ijlter

279

5.2 Participants For the present study, using a non-probability sampling, 189 fourth-year students of the Pedagogy career courses from the Catholic University of Maule were chosen as the participants of the study (Cardona, 2002). These are distributed according to the following five career courses: Pedagogy in English (24), Pedagogy in Spanish Language and Communication (18), Pedagogy in Physical Education (65), Pedagogy in Religion and Philosophy (46), and Pedagogy in Differential Education (36).

The ages of the participants vary between 22 and 24 years. With regard to gender, women (n = 123; 65.1%) predominate over men (n = 66; 34.9%). All of the participants have taken the learning assessment course; moreover, each one of them has been assessed throughout their training process from the first levels of teaching. It should be noted that all future teachers would continue to be evaluated once they pass their teaching careers by the teacher professional development system governed by Law No. 20.903 of the Ministry of Education of Chile (Teaching Career Act, Ministry of Education, 2016).

5.3. Procedures and Data Analysis A pollster applied the questionnaire during the regular sessions of the courses.

On each occasion, the participants were informed about the objectives of the study and were asked to sign an informed consent to safeguard the ethical principles of the research.

The data analysis considered descriptive and inferential techniques. Using SPSS software, measures of central tendency (mean) and dispersion measures (standard deviation) were calculated, and percentages were obtained for each variable of the study. Moreover, the Kruskal-Wallis H test was used to search for statistically significant differences between the groups.

6. Results Following the order of the objectives, the results that account for the perceptions of the students for each dimension of the study are presented in Table 4.

Subsequently, the results obtained by comparing the perceptions raised in the instrument are shown according to the career courses taken by the students.

Table 4: Perceptions about learning evaluation Descriptors

M

SD

Evaluating is the same as grading.

When one talks about evaluating, the focus is on the evaluation instrument.

The most effective evaluation corresponds to the written evaluation since it better predicts a student's learning achievement.

The evaluation of learning is considered a communication instrument that facilitates the construction of knowledge within the classroom.

1.48 3.26

http://ijlter.org/index.php/ijlter

.93 1.3

TD % 73 13.2

PD % 14.3 17.5

NA/ND % 6.9 17.5

PA % 3.9 33.9

TA % 2.1 18

2.13

1.1

36

31.7

19

10.1

3.2

3.51

1.1

3.7

15.3

23.8

40.2

16.9

280

The psychometric evaluation has the 2.47 1.0 24.9 15.3 49.7 8.5 1.6 same function as the formative pedagogical evaluation.

Evaluation in education is a linear 3.10 1.3 16.9 11.1 29.6 29.6 12.7 process.

The evaluation of learning must be 4.26 .97 1.6 6.9 6.9 32.8 51.9 fully integrated into the teachinglearning process that is being carried out.

The competency-based assessment 2.62 1.0 19 21.7 38.1 20.1 1.1 has the same emphasis as pedagogical assessment.

Evaluating is the same as measuring.

2.60 1.3 26.5 22.8 24.3 16.9 9.5 Assessing is focusing on student 2.28 1.3 36 25.9 19.6 11.1 7.4 grades.

M = Mean, SD = Standard Deviation, TD = Totally Disagree, PD = Partially Disagree, NA / ND = Neither Agree / Neither Disagree, PA = Partially Agree, TA = Totally Agree

In general terms, the items of the dimension perceptions of the students about the evaluation of learning show us that they are capable of responding to questions related to the evaluation. Only 25.3% on average of the respondents are not capable of the items consulted.

At a particular level, the values show us that 87.3% of the sample totally disagreed or partially disagreed before the consultation whether evaluating is the same as grading (M = 1.48, SD = 0.93). In the same way, a rating below the mean was noted when they are asked whether evaluating is the same as measuring (M = 2.6, SD = 1.3), or whether assessing is focusing on students' grades (M = 2.28, SD = 1.3).

These responses correspond to the items’ moderately low ratings, the psychometric evaluation has the same function as the formative pedagogy evaluation (M = 2.47, SD = 1.0) and competency-based assessment has the same emphasis as pedagogical assessment (M = 2.62, SD = 1.0). A total of 67.7% of the respondents were declared to be totally or partially in disagreement while referring to the effectiveness of the written evaluation (M = 2.13) to predict student achievement.

On the other hand, over 50% of the participants (PA = 33.9%, TA = 18%) positively valued the evaluation focused on the instrument (M = 3.26, SD = 1.3).

Correspondingly, the group of pedagogy students recognised the evaluation as a process integrated into teaching and learning (M = 4.26; PA = 32.8%, TA = 51.9%) and as a constitutive part of the communication instruments that facilitate the construction of knowledge within the classroom (M = 3.51, SD = 1.1). The above results are shown in Table 5.

http://ijlter.org/index.php/ijlter

281

Table 5: Perception of the role of evaluation in teaching and learning processes Descriptors

M

SD

TD % 3.2

PD % 9.0

NA/ND % 13.2

PA % 37.6

TA % 37.0

Evaluation in education is an 3.96 1.7 important tool to measure the achievement of student learning.

Learning assessment is essential to 3.46 1.1 6.3 13.2 24.3 40.2 15.9 grade students.

Learning assessment allows the 3.59 1.1 6.3 9.5 21.7 43.9 18.5 teacher to compare their students based on the results they obtain.

It is important to grade students to 3.12 1.2 10.6 22.8 20.1 37.6 9.0 see whether they should be promoted or failed.

The evaluation is an instrument that 3.48 1.4 12.7 13.8 14.8 30.7 28.0 allows to know the students.

Evaluation is a mechanism that 3.78 1.0 2.6 7.9 21.2 45.5 22.8 regulates the teaching-learning process.

Student assessment is a useful 3.96 0.9 2.1 5.3 15.9 47.6 29.1 practice for the teacher.

Once an assessment instrument has 4.54 0.8 2.1 1.6 3.7 25.4 67.2 been applied, it is necessary to analyse the results with the students.

Learning assessment allows students 3.76 0.9 0.0 11.1 23.8 42.9 22.2 to self-regulate their educational processes.

The evaluation procedures of the 4.59 0.8 .5 3.2 7.4 14.8 74.1 teaching-learning process must be known to the students.

The evaluation in some cases is used 4.23 0.9 1.6 2.1 14.8 34.9 46.6 as a control tool.

In the teaching-learning processes, 2.02 1.2 47.1 22.2 17.5 8.5 4.8 the most important thing is to pass.

In education, standardised 2.16 1.2 41.3 21.2 21.7 12.2 3.7 assessments account for the real learning that students have.

Assessment of learning can improve 4.1 1.0 2.6 3.7 16.9 34.9 41.8 pedagogical practices.

M = Mean, SD = Standard Deviation, TD = Totally Disagree, PD = Partially Disagree, NA / ND = Neither Agree / Neither Disagree, PA = Partially Agree, TA = Totally Agree

The results show that, in the second dimension, the participants can position themselves (NA / ND = 17.6%, versus 82.4%) against the questions consulted. It is possible to assess low scores and negative results when they are consulted about the function of the standardised evaluation to account for the real learning of students (M = 2.16, SD = 1.2) and in the item that relates the evaluation to pass during the teaching and learning process (M = 2.02, SD = 1.2).

Positive and moderate ratings are found when the participants were consulted about the importance of grading in the teaching and learning process. This is the case as in the following items: it is important to grade students to see whether they should be promoted or failed (M = 3.12, SD = 1.2) and learning assessment is essential

http://ijlter.org/index.php/ijlter

282

to grade students (M = 3.46, SD = 1.1), the responses are predominantly partially in agreement in these two cases (37.6% and 40.2%, respectively).

On the evaluation as a tool for the teacher, it is possible to appreciate values above the mean in items such as learning assessment allows the teacher to compare their students based on the results they obtain (M = 3.59, SD = 1.1) and the evaluation is an instrument that allows knowing the students (M = 3.48, SD = 1.4). Similarly, a high degree of agreement is obtained (PA = 42.9% and 45.5% and TA = 22.8% and 22.2%) respectively when referring to whether the learning assessment allows students to self-regulate their educational processes (M = 3.76, SD = 0.9) and whether the evaluation is a mechanism that regulates teaching-learning process (M = 3.78, SD = 1.0).

Positive and strong ratings were found in six questions consulted. Three of them are the student assessment is a useful practice for the teacher (M = 3.96, SD = 0.9), the evaluation in education is an important tool to measure the achievement of student learning (M = 3.96, SD = 1.7) and learning assessment can improve pedagogical practices (M = 4.1, SD = 1.0). The remaining three are those with the highest degree of agreement and they are: • the evaluation in some cases is used as a control tool (PA = 34.9%, TA = 46.6%), • once an assessment instrument has been applied, it is necessary to analyze the results with the students (PA = 25.4%, TA = 67.2%), and • the evaluation procedures that are carried out in a teaching-learning process must be known by the students (PA = 14.8%, TA = 74.1%).

Table 6 shows the mean values and the standard deviation of the dimension perceptions about the evaluation during the training processes for the total sample. Moreover, the percentages of responses that support the subsequent descriptive analysis are indicated.

Table 6: Perception of evaluation during the training process and professional practices Descriptors

M

SD

Teachers working in schools are well trained to develop learning assessment.

Initial teacher training provides a solid basis for evaluating student learning.

University teachers have developed an evaluation that allows them to improve the learning processes.

Your university professors have made evaluation an instance of learning.

The evaluations that have been applied in your pedagogy agreed with the objectives of the subject or branch.

Your evaluation processes at the university were always transparent.

2.82

http://ijlter.org/index.php/ijlter

1.1

TD % 11.1

PD % 30.2

NA/ND % 30.2

PA % 22.8

TA % 5.8

3.20

1.2

10.6

18.5

23.8

34.4

12.7

3.03

1.0

8.5

21.2

36.0

28.0

6.3

3.21

1.1

10.1

16.4

25.9

38.1

9.5

3.45

1.1

6.3

12.2

26.5

40.2

14.8

2.98

1.2

12.2

25.9

23.8

27.5

10.6

283

In your university study process, after 3.03 1.1 11.1 22.8 24.3 36.0 5.8 being evaluated, you were given some kind of feedback.

M = Mean, SD = Standard Deviation, TD = Totally Disagree, PD = Partially Disagree, NA / ND = Neither Agree / Neither Disagree, PA = Partially Agree, TA = Totally Agree

The results obtained in this dimension which relates to knowing/determining the perceptions of how the experience has been in their professional practices and how they have been evaluated in their initial teacher-training programme show a mean of 3.1 with a standard deviation of 1.1. Under such an average mean, it is possible to understand the responses such as the teachers who work in schools are well trained to develop the learning assessment (M = 2.82, SD = 1.1) and the evaluation processes to which they have been subjected at the university were always transparent (M = 2.98, SD = 1.2).

Around the sample mean, the items university teachers have developed an evaluation that allows them to improve their learning processes (M = 3.03, SD = 1.0), and in their university study process, after being evaluated, they were given some type of feedback (M = 3.03, SD = 1.1) show a tendency to be evaluated slightly positively, TD and PD = 29.7% versus 34.3% for PA and TA in the case of the first referred item, and TD and PD = 33.9% versus 41.8% in the case of the second item referring to the subsequent feedback received after the evaluation.

On the mean of the scale, a level of responses not greater than 50% of degrees of the agreement for the following items was obtained with a predominance of responses that tend to be partially in agreement: • initial teacher training provides a solid basis for evaluating student learning (M = 3.2, SD = 1.2), PA= 34.4%, • their university professors have made the evaluation an instance for learning (M = 3.21, SD = 1.1) PA= 38.1%, and • the evaluations that have been applied during the training stage were in agreement with the objectives of the subjects or branches (M = 3.45, SD = 1.1) PA= 40.25%.

To analyse possible statistically significant differences in the students' perceptions according to the different training courses (pedagogy career), the Kruskal-Wallis H test was carried out. The results are shown in Table 7.

Table 7: Kruskal-Wallis H test for group differences Descriptors Learning evaluation is essential to grade students.

It is important to grade students to see whether they should be promoted or failed.

Evaluating is the same as grading.

Teachers working in schools are well trained to develop an assessment of learning.

Learning evaluation allows students to selfregulate their educational processes.

The psychometric evaluation has the same

http://ijlter.org/index.php/ijlter

Chisquared 12.416

Gl

p

4

0.015

Pedagogy careers P.I. < O.P.

20.182

4

0.00

P.I. < O.P

9.524 19.559

4 4

0.049 0.001

P.I. < O.P P.I. < O.P

12.490

4

0.014

P.I. < O.P.

10.290

4

0.036

P.E D. <

284

function as the formative pedagogical O.P. evaluation.

The evaluation procedures that are carried out 22.208 4 0.000 P.R. < O.P. in a teaching-learning process must be known to the students.

The evaluation in some cases is used as a 9.980 4 0.041 P.R. < O.P. control tool.

In education, standardised evaluations 34.100 4 0.000 P.E D. < account for the real learning that students O.P. have.

The evaluations applied to you in your initial 26.619 4 0.000 P.E D. < teacher training always agreed with the O.P. objectives of the subject or branch.

The evaluations applied to you in your initial 15.349 4 0.004 P.E D. < teacher training always agreed with the O.P. objectives of the subject or branch.

In your university study process, after being 22.432 4 0.000 P.E D. < evaluated, you were given some kind of O.P. feedback.

Evaluation is focusing on student grades.

15.619 4 0.004 P.I. < O.P. P.I. = Pedagogy in English, P.R. = Pedagogy in Religion and Philosophy, P.E.D = Pedagogy in Differential Education, O.P. = Other pedagogies

The results of the Kruskal-Wallis H test show the presence of 13 items with a statistically significant difference between the groups. According to the dimension of the instrument, three of them are presented in the first dimension about the perception of what is evaluation, and six in the second dimension about the function and purposes of evaluation. Finally, four items in the third dimension of evaluation during the training process and professional practices show statistically significant differences.

Concerning the perceptions of the students who show statistically significant differences are the students of Pedagogy in English who report six differentiating items with their peers. These correspond to the following items: • learning evaluation is essential to grade students, • it is important to grade to promote or fail, • evaluating is the same as grading, • teachers in schools are trained to evaluate, • evaluation for self-regulation, and • evaluation is focusing on student grades.

The students of Pedagogy in Religion and Philosophy show differences with the other groups in two questions related to the transparency of the evaluation process and post-evaluation feedback. The students of Pedagogy in Differential Education present significant differences in five items of the instrument, as compared to the other pedagogies. These items are the following: • the psychometric evaluation and the formative evaluation fulfill the same function, • the standardized evaluation accounts for real learning, • the evaluations applied in its training were consistent with the objectives, • the transparency in the evaluation processes to which it was subjected, and • whether he was given feedback after being evaluated.

http://ijlter.org/index.php/ijlter

285

7. Discussions and Conclusion The evolution that the concept of evaluation has undergone throughout history cannot be developed outside of an epistemological interpretation. Alcaraz (2015) and Carbajosa (2011) indicate that its development is based on the researchers’ different paradigmatic perspectives. The ways of conceptualising the evaluation of learning are culturally anchored and follow a paradigmatic look to conceive reality in certain moments and contexts. Therefore, we can identify two viewpoints, one from a positivist and the other from an interpretive perspective.

In the first category, Tyler (1967), Thorndike and Hagen (1970), Mager (1973) and Chadwick (1976) developed an evaluation oriented to measurement, statistics and systemic study. However, in the second one, researchers such as Scriven (1981), Stake (1999), House (1994), Eisner (1985), Stenhouse (1984), MacDonald (1974) and Elliott (1993) conceived evaluation as an understanding exercise that involves the entire teaching-learning process (Carbajosa, 2011).

Therefore, it is relevant to consider that the evaluative practices implemented during the initial teacher training process will positively or negatively regulate the professional performance of the future teachers because the transposition of the evaluation is related to the evaluative competencies acquired during their training process. Evaluations determine and condition the contents, strategies, learning and efforts in educational processes directly with learners (Barbera, 2003; Biggs, 2005; Bonson & Benito, 2005; Cano, 2008; Dochy, Segers & Dierick, 2002; Gibbs, 2003; Santos Guerra 1998, 2003; Tejada & Ruiz, 2016; Zabalza, 2003).

Pedagogy students perceive the evaluation of learning as a curricular device that enables improving the teaching and learning processes in students. This is because the participants of this study had an up-to-date perception of the evaluation of learning, its concept and function in the educational processes.

This updated perspective considers the evaluation of learning as a means that facilitates the construction of knowledge within the classroom which must be fully integrated into the training processes to achieve a good regulation of teaching and learning. Some researchers state that evaluation with proper implementation enables improving the teaching practices of future teachers (Cabra-Torres, 2011; Gil–Flores, 2012; Santos Guerra, 2003; Sánchez González, 2010; Stobar, 2010).

The results also indicate that the majority of students have a positive assessment of the evaluation experiences developed in their training process. It is also possible to understand that students have incorporated theoretical knowledge related to what is learning assessment. It is necessary that the knowledge acquired by the respondents be reflected in the training activities that they receive from their teachers so that there is a decidedly favourable attitude to the development of a more conscious, reflective, and formative assessment of learning. There is a perception among the learners that the focus of the assessment is mainly on the instrument rather than on the relevance of the whole evaluative process.

The students also perceive that in some cases, evaluation is used as a control tool by their teachers. It is necessary to analyse the perception of students in this area

http://ijlter.org/index.php/ijlter

286

since they are contrary to what the specialised literature raises. The students must necessarily experience formative instances in their learning process and the way in which they are evaluated is precisely one of them (Sanmartí, 2007).

Results show that in the initial teacher training processes, authentic evaluation activities must be implemented and adjusted to educational purposes so that the future teachers are involved in the evaluation process and participate in real improvement activities. Statistically significant differences were found in the perceptions of the respondents of the different career courses about the function of the evaluation, specifically, those that refer to the grading, standardisation, control, coherence, transparency, and self-regulation, with transparency being the item that generates differences in all groups. This would indicate that the evaluation to which the students have been subjected lacks elements that benefit this area since the characteristics present and collected through the instrument require its deepening in subsequent studies since the pedagogical device tends to be homogeneous in a heterogeneous educational context.

The results obtained allow us to analyse how pedagogy students perceive and develop evaluation in their formative work and how teacher trainers in the evaluation area could reflect on students' perceptions of professional competencies in the field of evaluation to advance and improve the teaching practices of future teachers. It is suggested future teachers be trained through reflective instances and grounded practices of evaluation, oriented towards the particular characteristics of each pedagogical discipline, the particular environment of the school, and the curricular demands. Through these changes, progress can be made to improve the learning processes that students experience daily in their educational communities.

Acknowledgements This work was carried out under the Doctoral Programme in Education in Consortium at the Universidad Católica del Maule, Chile, and the DIUBB Project 195623 4/IenDU-FID of the Universidad del Bio-Bio, Chile.

8. References Alcaraz, N. (2015). Aproximación histórica a la evaluación educativa: De la generación de la medición a la generación ecléctica [Historical approach to educational evaluation: From the measurement generation to the eclectic generation]. Revista Iberoamericana de Evaluación Educativa, 8(1), 11-25.

Ashford-Rowe, K., Herrington, J., & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205-222. https://doi.org/10.1080/02602938.2013.819566 Barbera, E. (2003). Estado y tendencias de la evaluación en Educación Superior. [Status and trends of evaluation in Higher Education]. Revista de la Red Estatal de Docencia Universitaria, 3(2), 47- 60.

Biggs, J. (2005). Calidad del aprendizaje universitario. [Quality of university learning].

Madrid: Narcea.

Black, P., & Williams, D. (1998). Assessment and classroom learning. Assessment in Education, 4(1), 7-71.

Bloom, J. (1969). Taxonomía de los objetivos de la educación. [Taxonomy of educational objectives]. Buenos Aires: Talleres Gráficos Litodar.

http://ijlter.org/index.php/ijlter

287

Bonson, M., & Benito, A. (2005). Evaluación y aprendizaje. In A. Benito y A. Cruz (Eds.), Nuevas claves para la docencia universitaria en el Espacio Europeo de Educación Superior [New keys to university teaching in the European Higher Education Area] (pp. 87-100). Madrid: Narcea Bound, D., & Falchikov, N. (2007). Rethinking assessment in higher education learning for the long term. London: Routledge.

Cabra-Torres, F. (2011). Reseña del libro Tiempos de pruebas: Los usos y abusos de la evaluación de Gordon Stobart. [Review of the book Testing Times: The Uses and Abuses of Evaluation by Gordon Stobart] Magis Revista Internacional de Investigación en Educación, 4(7), 211-213.

Cano, M. (2008). La evaluación por competencias en la educación superior [Competencybased assessment in higher education]. Profesorado. Revista de Currículum y Formación de Profesorado, 12(3),1-16.

Carbajosa, D. (2011). Debate desde paradigmas en la evaluación educativa [Debate from paradigms in educational evaluation]. Perfiles educativos, 33(132), 183-192.

Cardona, M. C. (2002). Introducción a los métodos de investigación en educación [Introduction to research methods in education]. Madrid: EOS. Carless, D., Joughin, G., & Mok, M. (2006). Learning-oriented assessment: Principles and practice. Assessment & Evaluation in Higher Education, 31(4), 395-398.

Castillo, S., & Cabrerizo, J. (2010). Evaluación educativa de aprendizajes y competencias [Educational assessment of learning and competencies]. Madrid: Pearson Educación, S.A. Centro de Perfeccionamiento, Experimentación e Investigaciones Pedagógicas - CPEIP (2018). Resultados nacionales. Evaluación nacional diagnóstica de la formación inicial docente [National Results. National diagnostic test of initial teacher training].

Chile: Ministerio de Educación.

Chadwick, C. (1976). Tecnología educacional para el docente [Educational technology for teachers]. Buenos Aires: Paidós.

Colás, B. (1994). Investigación educativa [Educational research]. Sevilla: Alfar.

Cronbach, L. (1963). Course improvement through evaluation. New York: Teachers College Record.

Del Pozo, J. (2013). Competencias profesionales. Herramientas de evaluación: El portafolios, la rúbrica y las pruebas situacionales [Professional competencies. Assessment tools: The portfolio, rubric and situational tests]. Madrid: Narcea.

Dochy, F., Segers, M., & Dierick, S. (2002). Nuevas vías de aprendizaje y enseñanza y sus consecuencias: Una era de evaluación [New avenues of learning and teaching and their consequences: An era of assessment]. Red-U, 2(2), 13- 30.

Escudero, T. (2003). Desde los test hasta la investigación evaluativa actual. Un siglo, el XX, de intenso desarrollo de la evaluación en educación [From tests to current evaluative research. A century, the 20th, of intense development of evaluation in education.]. Relieve, 9(1), 11- 43.

Fernández, A. (2010). La evaluación orientada al aprendizaje en un modelo de formación por competencias en la educación universitaria [Learning-oriented assessment in a competency-based training model in university education]. Revista de Docencia Universitaria, 8(1), 11-34.

Fernández, B. R. (1981). Perspectivas históricas de la evaluación conductual [Historical perspectives on behavioral assessment]. Madrid: Ediciones Pirámide.

Gagné, R. (1971). Las condiciones del aprendizaje [Learning conditions]. Madrid: Aguilar.

Gibbs, G. (2003). Uso estratégico de la evaluación en el aprendizaje [Strategic use of evaluation in learning]. In A. Brown y A. Glasner (Eds.), Evaluar en la universidad.

Problemas y nuevos enfoques (pp. 61-75). Madrid: Narcea

http://ijlter.org/index.php/ijlter

288

Gil-Flores, J. (2012). La evaluación del aprendizaje en la universidad según la experiencia de los estudiantes [The assessment of learning in the university as experienced by students]. ESE. Estudios Sobre Educación, 22, 133-153.

Guba, G., & Lincoln, Y. (1982). Effective evaluation. USA: Jossey-Bass.

Hall, K., & Burke, W. (2003). Making formative assessment work: Effective practice in the primary classroom. United Kingdom: Open University Press.

Hernández, R., Fernández, C., & Baptista, M. (2010). Metodología de la investigación (5ª edición) [ Research Methodology, 5th edition].

México: McGraw-Hill Interamericana.

Jaimes, G., & Callejas, M. (2009). La autonomía, los procesos de pensamiento y las TIC: competencias del siglo XXI. Temática contemporánea de aplicación en escuelas, colegios y universidades [Autonomy, thought processes and ICT: 21st century competencies. Contemporary topics of application in schools, colleges and universities]. Bogotá: Limusa Noriega.

Kaftan, J., Buck, G., & Haack, A. (2006). Using formative assessments to individualize instruction and promote learning. Middle School Journal, Westerville, 37(4), 44-49.

Littlewood, K. (1973). High fidelity simulation as a research tool. Best Practice & Research Clinical Anesthesiology, 25, 473-487. https://doi.org/10.1016/j.bpa.2011.08.001.

Mager, R. (1973). Análisis de metas [Goal analysis]. México: Trillas.

McGaghie, W. C, Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review of simulation-based medical education research: 2003-2009. Medical Education, 44, 50-63. https://doi.org/10.1111/j.1365-2923.2009.03547.x Ministerio de Educación. (2019a). Centro de Perfeccionamiento, Experimentación e Investigaciones Pedagógicas. Resultados Nacionales. Resultados Nacionales Evaluación Docente 2018 [National Results of Teacher Evaluation 2018.

Ministerio de Educación. (2019b). Centro de Perfeccionamiento, Experimentación e Investigaciones Pedagógicas. Resultados Nacionales. Evaluación Nacional Diagnostica de la Formación Inicial Docente 2018 [National Diagnostic Test of Initial Teacher Training 2018.].

Perrenoud, P. (2004). Diez nuevas competencias para enseñar [Ten new competencies for teaching]. Barcelona: Graó.

Ros, M., & Conesa, C. (2013). Adquisición de competencias a través de la simulación y juego de rol en el área contable [Acquisition of competencies through simulation and role-playing in the accounting área]. Estudios sobre el Mensaje Periodístico, 19, special number, 419-428. https://doi.org/10.5209/rev_ESMP.2013.v19.42049.

Ríos, D., & Herrera, D. (2017). Los desafíos de la evaluación por competencias en el ámbito educativo [The challenges of competency-based assessment in education]. Educação e Pesquisa, 43(4), 1073-1086.

Ruay Garces, R. (2019). Nuevos dispositivos de evaluación de aprendizajes en un enfoque curricular por competencias [New learning assessment devices in a competency-based curricular approach]. In Consejo de Decanos de Facultades de Educación del Consejo de Rectores de las Universidades Chilenas, Educación, escuela y profesorado: Aportes desde el Consejo de Decanos de Facultades de Educación del Consejo de Rectores de las Universidades Chilenas (pp. 317 – 336). Concepción: Editorial Universidad de Concepción.

Sacristán, J., Pérez, A., Martínez, J., Torres, J., Angulo, F., & Álvarez, J. (2008). Educar por competencias, ¿qué hay de nuevo? [Educating by competencies, what's new?].

España: Ediciones Morata.

Salazar-Gómez, E., & Tobón, S. (2018). Análisis documental del proceso de formación docente con la sociedad del conocimiento [Documentary analysis of the teacher training process with the knowledge society]. Espacios, 39(53), 17-29.

http://ijlter.org/index.php/ijlter

289

Sánchez González, M. (2010). Técnicas docentes y sistemas de Evaluación en Educación Superior [Teaching Techniques and Evaluation Systems in Higher Education].

España: Narcea.

Sanmartí, N. (2007). 10 ideas clave: Evaluar para aprender [10 key ideas: Evaluate to learn].

Madrid: Graó.

Santos Guerra, M. (1998). Evaluar es comprender [To evaluate is to understand]. Buenos Aires: Editorial Magisterio del Río de la Plata.

Santos Guerra, M. (2003). Una flecha en la diana. Evaluación como aprendizaje [An arrow in the bull's eye. Evaluation as learning]. Madrid: Ediciones Narcea, S. A. Scriven, M. (1967). The methodology of evaluation. In R. W. Tyler, R. M. Gagné, & M. Scriven (Eds.), Perspectives of curriculum evaluation. Chicago, IL: Rand McNally.

Scriven, M. (2007). The logic of evaluation. In: H. V. Hansen et al. Dissensus and the search for common ground (pp. 1-16), Windsor, ON: OSSA. Sistema de Evaluación del Desempeño Profesional Docente. (2019). Informe de resultados Evaluación Docente 2019 [2019 Teacher Evaluation Results Report]. Chile: Ministerio de Educación.

Stobart, G. (2010). Tiempos de pruebas: Los usos y abusos de la evaluación [Testing times: The uses and abuses of evaluation]. España: Ediciones Morata.

Tejada, F., & Ruiz, C. (2016). Evaluación de competencias profesionales en Educación Superior: Retos e implicaciones [Assessment of professional competencies in Higher Education: Challenges and implications]. Educación XX1, 19(1), 17-38. https://doi.org/10.5944/educXX1.12175.

Tejedor, F. (2000). El diseño y los diseños en la evaluación de programas [Design and designs in evaluative programs]. Revista de Investigación Educativa, 18(2), 319–339.

Thorndike, R. & Hagen, E. (1970). Test y técnicas de medición en psicología y educación.

México, Trillas.

Tobón, S. (2004). Formación basada en competencias: Pensamiento complejo, diseño curricular y didáctica [Competency-based education: Complex thinking, curriculum design and didactics]. Bogotá: ECOE. Tyler, R. W. (1950). Basic principles of curriculum and instruction. Chicago: University of Chicago Press.

Tyler, R. W. (1967). Changing concepts of educational evaluation. In R. E. Stack (Comp.), Perspectives of curriculum evaluation. AERA Monograph Series Curriculum Evaluation, 1. Chicago, IL: Rand McNally.

Zabalza, M. (2003). Diseño curricular en la universidad. Competencias del docente universitario [Curriculum design at the university. Competencies of university teachers]. Madrid: Narcea.

http://ijlter.org/index.php/ijlter

290

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 290-303, February 2021 https://doi.org/10.26803/ijlter.20.2.16

Career Readiness of Graduating Office Technology and Management Students in Polytechnics in Southeast Nigeria Lynda Chineze Ezechukwu, Oliver Okechukwu Okanazu, Muhammadu Madu-Saba Babalulu, Ayoola Abimbola Arowolo and Chijioke Jonathan Olelewe University of Nigeria, Nsukka, Nigeria https://orcid.org/0000-0001-5847-8347 https://orcid.org/0000-0003-2942-4723 https://orcid.org/0000-0003-0832-2365 https://orcid.org/0000-0002-8482-2241 https://orcid.org/0000-0002-1104-053X

Abstract. Career readiness of graduating Office Technology and Management (OTM) students of polytechnics in three states of South-East Nigeria was assessed. Career readiness was determined based on the extent to which the students possessed the attitudes and skills required for advancing on the OTM career path. The skills were aggregated into six clusters: office, managerial, entrepreneurial, ICT (information and communication technology), communication, and socio-psychological skills.

Two research questions guided the study, namely (i) the extent to which the graduating OTM students possessed the requisite skills, and (ii) the career readiness index of the students. A total of 179 HND (Higher National Diploma) II students of OTM at the time of the study constituted the study population. A questionnaire, structured on a five-point scale in line with the indicators for career readiness was used to collect data from the participants.

The instrument was face validated and trial tested prior to the study, yielding a 95-reliability coefficient. Mean and standard deviations were used to answer research question one, while research question two was answered using the career readiness index formula. Findings of the study showed an average career readiness index of 3.06 for the graduating OTM students in the polytechnics studied. The indicators showed that the students were lacking in office, managerial, entrepreneurial and communication skills, but rated very high in ICT and socio-psychological skills. The study concludes that there is a need for improvement and recommends, among others, that the institutions should strengthen their partnership with industry for workbased training of students to support improved effectiveness of the OTM programme.

Keywords: career readiness; entrepreneurial; ICT; communication and socio-psychological skills; office technology and management

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

291

1. Introduction The role of institutions of higher learning in the economic development of a nation in terms of equipping graduates with core skills that enhance their employability and career prospects cannot be overemphasized. Although career choice and employment prospects are major considerations for students opting for higher education (Jackson & Wilton, 2017), their preparedness to take advantage of the openings in the chosen career path is more important than the mere choice of a career. Traditionally, the term ‘career’ was conceived as a series of work experiences associated with a single paid employment within a fixed organizational setting. However, new ways of conceptualizing careers have emerged, driven by changes in organizational hierarchies, employment relationships and workforce demographics, which have resulted in greater opportunities for the mobility of employees, thereby making employment relationships less permanent (Gyansah & Guantai, 2018; Tomlinson, Baird, Berg & Cooper, 2018). Furthermore, the advent of information and communication technology (ICT) has brought about tremendous changes in the outlook of the present-day office environment (Efunboade, Adegoke, Ogunmesa & Daramola, 2020). Office Technology and Management (OTM), formerly referred to as Secretarial Studies in Nigeria, evolved out of a need to meet the technological and managerial demands of today’s workplace. The new name OTM was approved by the supervising body for Nigerian polytechnics, the National Board for Technical Education (NBTE), in 2004. In the curriculum document for OTM, the NBTE stated the core skills which the OTM programme seeks to convey to students to prepare them for a career in OTM (NBTE, 2004). These skills include office, managerial, entrepreneurial, ICT, communication, and socio-psychological skills (Esene, 2017). This study was designed in line with these six core skills as shown in Figure 1, to assess the career readiness of graduating OTM students of polytechnics in south-east Nigeria based on the extent to which they possessed these skills.

2. Literature Review Abston and Soter (2020) defined career as a sequence of attainment and demonstration of requisite competencies that broadly prepare students for a successful transition into the world of work. In other words, career relates to helping individuals make the most of work opportunities and includes the impacts on and relationships between work roles and wider life roles (Career Development Institute, 2017). Career development is not just a decision that takes place at the entry point to a job; it is a lifetime process, which entails the different roles and responsibilities in which one engages throughout one’s working adulthood into retirement, including one’s education, family, leisure activities, paid or volunteer work, and so on (Career Development Institute, 2017). It involves the totality of an individual’s psychological, sociological, and educational events which eventually result into occupational choice and other life roles. In other words, it describes one’s growth and progress in one’s chosen career path (Gyansah & Guantai, 2018). This growth in career improves the quality of individuals’ lives in terms of who they become, their sense of purpose, socioeconomic status and positive impact on the communities they belong to. Career

http://ijlter.org/index.php/ijlter

292

development requires specialized training in a particular direction or path, and this makes the role of education prominent in preparing for a career.

Career-readiness is defined in terms of having the requisite cognitive, academic, work and social skills, knowledge or experiences, which enable the individual’s transition from school into the workplace and make viable a career path in the present-day context (Villares & Brigman, 2018). The Association for Career and Technical Education (2011) described career readiness as involving a blend of academic, technical, and employability skills. There is a great concern about the gap that exists between the school and the workplace. Graduates have been seen, by a group of researchers (Azih & Ejeka, 2015), to experience much difficulty making the transition from academic life to work, even though the school is supposed to provide the experiences and training to help individuals make intelligent occupational choices to advance their career paths. The authors further noted that the change from secretarial studies to OTM occurred mainly because graduates of secretarial studies were not able to build a career in their chosen vocation, due to a mismatch between the self-identity of the students upon graduation and that which is required for secretarial studies professionals in today’s office. Thus, secretarial studies changed to OTM in order to bridge that gap by providing functional training that would help the graduates of OTM make a career in office management.

According to Esene (2017), OTM is an educational programme designed to develop individuals’ abilities, skills, work ethics, and attitudes, as well as to impart the necessary knowledge required for effective entry and steady progress in gainful employment. The two main reasons for designing the Office Technology and Management programme were, first, to address the needs of the conventional office environment, which has long been the desire of the Secretarial Studies students and practitioners, and, second, to enhance students’ knowledge by equipping them with the requisite skills for today’s modern office/work environment. These changes led to a change in curriculum with more emphasis on information and communication technology (ICT), and managerial and entrepreneurial competencies (Azih & Ejeka, 2015). OTM, thus, is an activitybased educational programme aimed at equipping students with office, technological and managerial skills, attitudes, competencies, and work habits that are required for success in occupations that have to do with secretarial and office management (Omoniyi & Elemure, 2014). The office skills include typewriting, shorthand, bookkeeping and more (Adam, 2015). Managerial and entrepreneurial skills comprise planning, organizing, directing, decision making, problem solving, personal drive, creativity, self-reliance, and more (Suarta, Suwintana, Sudhana & Hariyanti, 2017). ICT skills include web browsing, e-mail, database and spreadsheet management, and other computer and technology skills (Chepkonga, 2015), while communication skills include reading, writing, speaking, and listening, which are involved in the process of receiving, generating, or transmitting messages (Suarta et al., 2017). Socio-psychological skills comprise inter-personal and intra-personal skills (Low, Botes, Rue & Allen, 2016). Possession of these skills indicates graduating OTM students’ preparedness for the labour market, and, by implication, their employability. The quality of

http://ijlter.org/index.php/ijlter

293

higher education programmes in terms of graduate career readiness and employability has been the focus of much research in recent times; and in many cases, the lack of employability skills in higher education graduates was reported (Abayadeera & Watty, 2016; Low et al., 2016). Suarta et al. (2017) underscored the need for institutions of higher learning to design and implement educational programmes to produce graduates who not only possess the technical skills associated with their discipline, but also interpersonal skills which meet the needs and expectations of employers. Relatedly, Villares and Brigman (2018) opined that preparing students to participate and compete favourably in today’s 21st century workplace will require the implementation of definite career readiness interventions.

Figure 1:

Conceptual framework for the assessment of career readiness of graduating OTM students

Research Questions Two research questions guided the study: Q1: To what extent do the graduating OTM students possess office, managerial, entrepreneurial, ICT, communication and socio-psychological skills?

Q2: What is the career readiness index value of the graduating OTM students?

3. Methodology Study Design and Population This study used a survey research design and involved public polytechnics that offer OTM programmes in Enugu, Ebonyi and Anambra States, South-east Nigeria. The population was a total of 179 HND II (final year) OTM students in

http://ijlter.org/index.php/ijlter

294

the three states, comprising 49 students in Enugu State, 45 in Ebonyi State and 85 in Anambra State. Due to the relatively small size of the population, total population sampling was used in the study.

The Instrument for Data Collection A questionnaire containing 79 items was structured to collect information on the career readiness of the OTM students. The questionnaire consisted of seven sections (A – G). Section A collected demographic details of respondents, while sections B - G collected data regarding the office, managerial, entrepreneurial, ICT, communication and socio-psychological skills respectively. A five-point Likert-type rating scale of very highly possessed (VHP), highly possessed (HP), possessed (P), slightly possessed (SP) and not possessed (NP) was used. Before the study, validation of the questionnaire was obtained independently from three experts in the Department of Business Education, University of Nigeria, Nsukka, and the questionnaire was trial-tested on 35 OTM students of another polytechnic outside the area of the study. The result of the testing gave a reliability coefficient of 0.95 using Cronbach's alpha.

Data Collection Procedure Verbal consent was obtained from the heads of department in the various institutions and the students/respondents themselves. With the assistance of the students’ class representatives, hard copies of the questionnaire were administered in person to the respondents in their classrooms after a briefing. All the 179 copies of the questionnaire administered were retrieved and used for data analysis.

Analysis of Data Data collected were analysed using mean (𝑥̅ ) and standard deviation. In answering the research questions, the extent to which students possessed the skills under each career readiness indicator/cluster, was rated from the means using real limits as follows: 4.50 - 5.00 = very highly possessed (VHP), 3.50 - 4.49 = highly possessed (HP), 2.50 - 3.49 = possessed (P), 1.50 - 2.49 = slightly possessed (SP) and 1.00 - 1.49 = not possessed (NP). The career readiness index (research question two) of the graduating OTM students/ respondents was computed using a formula adapted from Arowolo and Ede (2012), namely 𝐶𝑅𝑖 =

∑∞ 179(Os, Ms, Is, Es, Cs, Ss) 𝑁

where: CRi = Career-readiness index; OS = mean score in office skills; MS = mean score in managerial skills; ES = mean score in entrepreneurial skills; IS = mean score in ICT skills; CS = mean score in communication skills; SS = mean score in sociopsychological skills; N = total number of measures.

The career readiness index was interpreted based on the real limits: very high career readiness index = 4.50 - 5.00; high career readiness index = 3.50 - 4.49; average career readiness index = 2.50 - 3.49; low career readiness index = 1.50 2.49; and very low career readiness index = 1.00 - 1.49.

http://ijlter.org/index.php/ijlter

295

4. Results

Office skills rating of graduating OTM students for career readiness The overall mean score rating for office skills was 2.50 as shown in Table 1 and Figure 2, indicating that the OTM students possessed office skills. The results further revealed that the students highly possessed skills in writing shorthand consonants, and creating word documents, with mean scores of 3.83 and 3.66 respectively, while they slightly possessed skills in usage of stylus keyboard for shorthand writing, applying the principles of vocalization position in shorthand writing and also typing with eyes off the keyboard, with mean scores of 1.53, 1.84 and 1.82 respectively. Standard deviation values for the items ranged between 0.64 - 1.64, implying that the mean ratings did not vary widely among the respondents.

Table 1: Mean ratings of office skills possessed by graduating OTM students S/No 1 2 3 4 5 6 7 8 9 10 11 12

Item Ability to write shorthand consonants correctly Ability to use stylus keyboard to do shorthand writing Ability to write shorthand at a minimum speed of 80 wpm Ability to apply the principles of vocalization position in shorthand writing Ability to read back fluently dictated passages at the rate of 100 wpm to 120 wpm Ability to use the keyboard without fixing eyes on the keys Ability to create word documents on the computer Ability to open, save and close word documents in Microsoft Office Ability to use the cut, copy and paste functions when working in Microsoft Word Ability to set paragraphs and align texts in Microsoft Office Word Book-keeping ability Ability to keep detailed records of accounts Overall mean

𝑥̅ 3.83 1.53 2.55 1.84

SD 1.11 0.71 0.85 0.87

2.60

1.16

1.82 3.66 2.37

0.84 1.64 0.64

2.03

0.80

2.54

0.72

2.36 2.84 2.50

0.80 1.22 0.70

Key: 𝑥̅ = mean; SD = standard deviation

Managerial skills rating of graduating OTM students for career readiness As presented in Table 2 and Figure 2, the results on OTM students’ managerial skills revealed that the OTM students slightly possessed managerial skills with an overall mean of 2.31. The students slightly possessed skills in team leading, encouraging others to express themselves freely, inspiring others, respecting the opinions of others, understanding how and when to delegate a function, handling stress, setting long-term goals, accepting criticism, managing human and material resources, and working with people of different ethnic groups, tribes or religions.

On the other hand, the result (Table 2) showed that the OTM students highly possessed skills in running an effective and productive meeting. The standard deviations of the items were between 0.59 - 1.01 showing that the mean ratings by the respondents were not diverging that much.

http://ijlter.org/index.php/ijlter

296

Table 2: Mean ratings of managerial skills possessed by graduating OTM students S/no 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Item Ability to lead a team Ability to encourage others to freely express themselves Ability to be confined in by people easily Ability to inspire others to achieve their goals Ability to listen to and respect the opinion of others when deliberating in a team Ability to run an effective and productive meeting Ability to understand how and when to effectively delegate a function Ability to confidently engage people or team members in a difficult conversation Ability to inform people about their duties and responsibilities, and to clarify rules and policies Ability to set specific and challenging but realistic performance goals Ability to handle stress without transferring own mood to others Ability to set goals and establish a long-term vision Ability to accept criticism Ability to manage human and material resources Ability to work with people of different ethnic groups, tribes, or religions Overall Mean

𝑥̅ 2.15 2.19 2.03 2.26 1.81

SD 0.66 0.74 0.64 0.59 0.69

4.13 1.65

0.92 0.62

2.05

0.66

2.27

0.68

2.63

0.85

2.30

0.78

2.32 2.28 2.45 2.07

1.01 0.77 0.97 0.85

2.31

0.60

Key: 𝑥̅ = mean; SD = standard deviation

Entrepreneurial skills rating of graduating OTM students for career readiness From the results of entrepreneurial skills possessed by the OTM students as shown in Table 3, the OTM students had an overall mean of 2.89, which is rated as possessed (P). The students slightly possessed skills in seeing business opportunities, as well as in regularly coming up with creative ideas. The standard deviation values were between 0.64 - 1.16, showing that there was not much variation in the respondents’ mean ratings on entrepreneurial skills.

Table 3: Mean ratings of entrepreneurial skills possessed by graduating OTM students S/No 1 2 3 4 5 6 7 8 9 10

Item Ability to constantly see business opportunities or ideas that have commercial values Ability to grow or build businesses Ability to be creative and regularly come up with new ideas Ability to proffer solutions to business problems Ability to convert lectures to marketable ideas Ability to stand one’s ground even in difficult decisions Ability to come up with more than one way to solve a problem Ability to be decisive when making important decisions in business Ability to look at business problems from all angles to find the best solution Ability to be active, have functional skills and ideas to create jobs

http://ijlter.org/index.php/ijlter

𝑥̅ 1.72

SD 0.70

2.48 3.13 3.45 3.40 3.04 3.13

0.66 0.64 1.11 1.11 0.82 1.11

2.99

1.16

2.77

0.95

2.96

0.84

297

11

Ability to explore government programmes for entrepreneurship Overall mean

2.70

1.01

2.89

0.48

Key: 𝑥̅ = mean; SD = standard deviation

ICT skills rating of graduating OTM students for career readiness The OTM students had an overall mean score of 3.97 in ICT skills as shown in Table 4. This implies that the participants highly possessed skills in ICT. The table also shows that the students were very proficient in the usage of Microsoft Word with a mean score of 4.70, which implies that the skill was very highly possessed.

The standard deviation values were between 0.65-1.49, indicating that the mean ratings of the respondents were very close to one another.

Communication skills rating of graduating OTM students for career readiness The mean rating of the responses on communication skills possessed by the OTM students as shown in Table 5 shows that the students had an overall mean score of 2.87. Although the results show that the OTM students highly possessed skills in listening beyond subject content, reflecting on speakers’ body language, reading books other than school books, understanding what they read, and using leisure time for reading, it was also revealed that the students slightly possessed skills in taking notes during lectures, in using the dictionary, logically organizing ideas when writing, and in accurately paraphrasing information they had read.

The standard deviation values of the items ranged between 0.66 - 1.34, thus indicating that there was not much variation in the mean ratings of the respondents on communication skills.

Table 4: Mean rating of ICT skills possessed by graduating OTM students S/No 1 2 3 4 5 6 7 8 9

10 11 12

Item Ability to access and navigate the computer independently Ability to use word processing software, e.g., Microsoft Word Ability to prepare a slideshow using Microsoft Power Point Ability to use Microsoft Excel to analyse information, do basic calculations, and view data in the spread sheet Ability to use Microsoft Groove and One Note to share notes for team projects Ability to design simple flyers, memos, and more on Microsoft Publisher Ability to organize own e-mails, contacts, and schedules using Microsoft Outlook Ability to use search engines to search for information Ability to download material and save files and other information effectively on a computer or external storage device Ability to upload notes, research work, discoveries, etc. on the cyberspace Ability to do online video conferencing using Skype, Google Hangout, Yahoo Messenger, etc.

Ability to chat with family or friends, using WhatsApp, Facebook, etc.

Overall mean

Key: 𝑥̅ = mean; SD = standard deviation

http://ijlter.org/index.php/ijlter

𝑥̅ 4.49 4.70 4.17 4.34

SD 0.85 0.65 1.06 0.93

3.20

1.34

4.00

1.09

3.52

1.36

3.84 4.31

1.33 1.00

3.55

1.28

3.17

1.49

4.26

1.12

3.97

0.51

298

Table 5: Mean Rating of Communication Skills Possessed by Graduating OTM Students S/No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Items Ability to take notes during lectures Ability to use the dictionary Ability to logically organize ideas when writing Ability to write an accurate paraphrase of information that has been read Ability to speak clearly without mumbling Ability to use metaphors appropriately Ability to sustain a discussion for a long time Ability to tailor a conversation to suit various audiences Ability to introduce self, confidently and appropriately, to others Ability to listen beyond subject content for the speakers’ emotional meaning Ability to listen to people patiently and without interruption Ability to reflect on the speaker’s body language Ability to read books other than school books Ability to understand words when reading Ability to use leisure time for reading Overall mean

𝑥̅ 2.44 2.46 2.40 1.68

SD 0.75 0.73 0.66 0.75

2.79 2.79 2.75 2.89 2.79

0.99 0.90 0.99 0.97 0.91

3.72

1.20

3.51 3.58 3.54 3.96 3.74 2.87

1.34 1.25 1.27 1.04 1.17 0.64

Key: 𝑥̅ = mean; SD = standard deviation

Socio-psychological skills rating of graduating OTM students for career readiness The mean rating of responses on socio-psychological skills possessed, as shown in Table 6, reveals that the students had an overall mean score of 3.79. This implies that socio-psychological skills were highly possessed. All the items related to socio-psychological skills were ranked as highly possessed. The standard deviation values of the items ranged between 0.80 - 2.33, showing that the mean ratings of the respondents did not vary distinctively.

Table 6: Mean Rating of Socio-Psychological Skills Possessed by Graduating OTM Students S/No 1 2 3 4 5 6 7 8 9 10 11 12

Items Ability to relate freely to people from a variety of backgrounds Ability to negotiate on own terms with others Ability to contribute own best to the success of group work, projects, assignments, etc.

Ability to take a leadership role in the class, church, community meetings, etc.

Ability to take different kinds of food (especially those from other cultures) Ability to make important decisions on own without consulting people Ability to be dedicated to the highest quality of work in assignments, projects, tests, exams, etc.

Ability of always being conscious of time Ability to have a written out long-term plan for life Ability to have a written down “to-do” list for each day Ability to ensure that daily “to-do” list is fulfilled every day Ability to give undivided attention to one task at a time

http://ijlter.org/index.php/ijlter

𝑥̅ 3.96 4.14 4.37

SD 1.01 0.85 0.80

3.89

1.21

3.50

1.28

4.32

0.85

4.15

1.01

3.77 4.03 3.30 3.46 3.20

1.26 2.33 1.18 1.26 1.31

299

13 14

Ability to plan one’s day, and to set aside time to rest every day Ability to know what one wants Overall mean

3.49

1.39

3.49 3.79

1.39 0.39

Key: 𝑥̅ = mean; SD = standard deviation

Mean Rating

5

3.97

4 3

2.5

2.31

2.89

3.79 2.87

2 1 0

Figure 2: Mean ratings of graduating OTM students on their possession of the core skills for career readiness

Career-Readiness Index of Graduating OTM Students The result of the career readiness index presented in Table 7 shows that the career readiness index of the graduating OTM students was 3.06. Thus, based on the real limits of the career readiness index, the OTM students were found to be average on the career readiness index.

Table 7: Career-Readiness Index Value of the Graduating OTM Students Career-readiness Indicators

Mean ratings

Career-Readiness Index C𝑹𝒊 =

Office skills Managerial skills Entrepreneurial skills ICT Skills Communication skills

2.50 2.31 2.89 3.97 2.87

Socio-Psychological skills

3.79

http://ijlter.org/index.php/ijlter

∑∞ 𝟏𝟕𝟗(𝐎𝐬,𝐌𝐬,𝐈𝐬,𝐄𝐬,𝐂𝐬,𝐒𝐬) 𝑵

3.06

300

5. Discussion The possession of office skills among the graduating OTM students as observed in this study can be attributed partly to the extensive emphasis placed on shorthand writing in the polytechnics. Agbongiasede (2014) noted that shorthand writing is a key course or subject in the study of OTM, and priority is given to teaching and learning shorthand in the polytechnics. Although the students were found to be proficient in taking shorthand dictations on their notepads, the findings also revealed that the students lacked skills in using modern technologies like the stylus pad in taking shorthand dictations. The lack of skills in using stylus pads for taking shorthand dictations could be attributed to inadequate or absence of modern teaching facilities for teaching and learning of shorthand in the institutions. Ubulom, Enyekit and Egwe (2010) observed that the teaching of skill subjects such as typewriting and shorthand requires some stimulus to be effective and this can be achieved by using instructional materials. Therefore, the observed lack of skills among the students in using modern office technologies such as stylus pad for taking shorthand dictations might be attributed to the required instructional materials either not being available or not being utilized. The findings also showed that OTM students were not able to type without fixing their eyes on the keyboard, even though the participants tended to exhibit high proficiency skills in the use of computers in creating Word documents, as well as in performing other actions such as saving, copying, and pasting in Microsoft Word. This finding is in agreement with Udo’s (2014) finding, that a large percentage of students can only type with computer or typewriter keyboards using two or three fingers instead of the required standard of nine fingers, and can hardly take their eyes off the keyboard when performing typing tasks.

The findings on managerial skills possessed by the OTM graduating students showed that the OTM students lacked skills in team leading, and also had a lack of confidence to engage with people in general or members of their team in difficult conversations. A professional secretary should be proficient as a manager in terms of delegation of duties, listening to and respecting others’ opinions in a team, being goal-oriented, able to work with people from varied backgrounds, and able to handle stress. With the growing demand for organizations to compete favourably in a global economy, the need for graduates who have high managerial competencies becomes inevitable. Lack of such skills results in graduates being considered not suitable in today’s workplace (Udo, 2014). Regarding entrepreneurial skills possessed by the graduating OTM students, the study showed that the students were skills deficient in creating and seeing business opportunities and lacked skills in growing or building a business. Although entrepreneurship studies have been included in the curriculum for OTM in Nigeria, this study found that the students were still lacking skills needed for creating businesses. OTM programmes should prepare and equip learners with the skills that make them not merely capable but also willing to undertake and run a small business of any kind with the sole aim of making profit. This they can achieve if they have the requisite skills for that task. This implies that the aim of entrepreneurship education as stated by the NBTE (2004), namely to equip OTM students with enterprising skills, is not yet being achieved.

http://ijlter.org/index.php/ijlter

301

The finding that the graduating OTM students highly possessed ICT skills agrees with recent reports on the positive impacts of ICT (Ghavifekr & Rosdy, 2015; Wakil, Muhamad, Sardar & Jalal, 2017). Today’s students have a natural inclination to get acquainted with ICT skills, owing to the preponderance of electronic gadgets such as computers, cell phones, digital music players and cameras, along with software for games, chats, electronic mail and more, which have become an integral part of everyday life. The finding has positive implications for career readiness of the OTM students in the sense that ICT has revolutionized the landscape of the current world of work, as skills in ICT have become prerequisite for effectiveness in any workplace. In this regard, the high possession of ICT skills is to the advantage of the graduating OTM students.

Assessing the extent to which ICT resources were utilized in teaching business education students in south Nigeria, Okolocha and Nwadiani (2015) reported that ICT resources were poorly utilized. Thus, it is possible that the high level of ICT skills among OTM students reported in this study ensued from the students’ exposure to the generally digitalized environment rather than from ICT resources being used to teach them in school.

Furthermore, communication is central to keeping people together and is a core skill needed to make a career in OTM. Communication has intellectual, sociocultural, psychological, technological and linguistic dimensions. Suarta et al.

(2017) posited that communication skills are critical for the career success of a secretary as the person who represents the organization. A professional secretary must be able to logically organize ideas in writing and communicate with clients and colleagues effectively. The finding that the graduating OTM students highly possess socio-psychological skills justifies the essence for the inclusion of sociopsychology in the OTM programme. The students of OTM are expected to possess inter-personal and intra-personal competencies that would enhance their effectiveness in the workplace. While inter-personal skills are everyday life skills involved in relating with others, whether individually or in groups, intra-personal skills (within-the-self skills/internal abilities and behaviours) are the capacity of individuals to guide themselves in any way possible toward achieving important goals (Widjaja & Saragih, 2018). OTM students are expected to master these skills because it is impossible to sustain momentum toward accomplishing set goals without such skills. Changes in the nature of work and workplace have shown that what counts as a ‘skill’ also has evolved. This means that formal knowledge may be only a small portion of what enables graduating students in OTM to successfully confront the ambiguities of today’s workplace practice (Otamiri, 2014). The high rating of ICT skills also might have influenced students’ high rating of their socio-psychological skills. Today’s students function in a networked digital society with a continual chain of interaction via social media among people from different spheres and religious backgrounds. This interaction, according to Widjaja and Saragih (2018), impacts greatly on the socio-psychology of individuals.

The relatively low mean ratings in office skills, managerial skills, entrepreneurial skills and communication skills (Figure 2) contributed majorly to the ensuing average career readiness index identified. No indispensable secretary can survive

http://ijlter.org/index.php/ijlter

302

without office skills such as working on a keyboard, book keeping, taking minutes of meetings, and other skills mentioned (Adam, 2015). Furthermore, communication skills are imperative for the effectiveness of an indispensable secretary. A professional secretary must be proficient as a manager, communicate effectively, possess core technical skills and must be versed in the use of current technologies used in today’s offices (Aliata & Hawa, 2014). The economy of any nation depends, to a significant extent, on the trained work force supplied from the tertiary institutions; career readiness of the graduating students, therefore, is invaluable. The observed average career readiness index of graduating OTM students implies that the students merely possess the skills that they need to succeed in the workplace. In the highly competitive workplace of the 21st century, one needs more than average career readiness in order to succeed. Since career readiness is closely related to employability (Abayadeera & Watty, 2016; Low et al., 2016; Suarta et al., 2017), the observed average career readiness of the graduating OTM students has the potential to impede their career or employment prospects upon graduation. By extension, this will create a deficit in manpower supply for jobs that require OTM graduates.

6. Conclusion and Recommendations The study assessed career readiness of graduating OTM students in three polytechnics in south-east Nigeria, based on the extent to which the students possessed the attitudes and skills required for advancing on the OTM career path.

The graduating OTM students rated high in ICT and socio-psychological skills, but rated low in managerial skills. Overall, the findings indicated an average career readiness index for the graduating OTM students. Given the observed low level of managerial and entrepreneurial competencies, the study concludes that the OTM programme of Nigerian polytechnics is not yet fully achieving its stated goals in adequately preparing OTM students to fit into their career path in the current world of work. Hence, a need for improvement exists regarding the implementation of the OTM programme in the polytechnics.

Based on the study findings, it is recommended that the OTM curriculum should be re-evaluated to identify possible causes of skills deficiencies, especially those with a bearing on the managerial proficiency of the OTM students. Modern technologies and tools, such as the stylus pad for taking shorthand dictations, should be provided and adequately incorporated into the training of the OTM students. More emphasis should be placed by the institutions and instructors on the practical aspect of the training and not mere theory. Modern facilities, such as simulators, should be provided for improved effectiveness of teaching and learning of OTM courses in the institutions. Furthermore, monitoring and proper accreditation of the institutions offering OTM programmes, based on the availability of these modern facilities and qualified staff, will enhance the effectiveness of the OTM programme and help teaching and training staff to achieve its objectives. The educational institutions should strengthen their partnership with industries for work-based training of their students. Provision of mentoring programmes in entrepreneurship for the OTM students will facilitate the students’ entrepreneurial skills development and consequently their career readiness. Finally, periodic workshops, awareness campaigns, and regular

http://ijlter.org/index.php/ijlter

303

retraining should be organized for OTM lecturers on how best to impart the 21stcentury skills required of their graduates for effective work participation in the knowledge-driven economy.

7. References Abayadeera, N., & Watty, K. (2016). Generic skills in accounting education in a developing country: Exploratory evidence from Sri Lanka. Asian Review of Accounting, 24(2), 149-170. https://doi.org/10.1108/ARA-03-2014-0039 Abston, K. A., & Soter, H. A. (2020). A professionalism conundrum: Development of business students’ career readiness. Development in Business Simulation Experiential Learning, 47, 266-271.

Adam, A. K. (2015). The efficacies of secretarial profession by Ghana education service and higher education institutions. Journal of Education and Practice, 6(18), 81-113. https://files.eric.ed.gov/fulltext/EJ1079710.pdf Agbongiasede, E.nE. (2014). Office technology and management curriculum in polytechnics: Appraisal. Nigeria Journal of Business Education, 1(3), 30-36.

Aliata, M. I., & Hawa, S. A. (2014). Modern office technology and the performance of the professional secretary in contemporary organisation in Ghana. Journal of Information and Knowledge Management, 3(4), 52-57.

Arowolo, A. A., & Ede, E. O. (2012). Assessment of the e-readiness of technical colleges in teaching and learning of mechanical trades in Lagos State and Abuja (FCT). Moldova: LAP (Lambert Academic Publishing).

Association for Career and Technical Education. (2011). Expanding career readiness through career and technical student organizations. Career readiness series. https://www.hsredesign.org/wpcontent/uploads/2018/07/CTSO_Career_Readiness.pdf Azih, N., & Ejeka, C. A. (2015). A critical analysis of National Board for Technical Education old and new curriculum in office technology and management for sustainable development. British Journal of Education, 3(9), 41-50.

Career Development Institute. (2017). Definitions: Career development and related roles. https://www.thecdi.net/write/CDI_Definitions_FINAL.pdf Chepkonga, S. (2015). An investigation of the relationship of ICT training of principals in ICT integration in management public secondary schools: A case of Nairobi County, Kenya. Journal of Education and Practice, 6(18), 194-200. https://files.eric.ed.gov/fulltext/EJ1079787.pdf Efunboade, A. O., Adegoke, O. F., Ogunmesa, O., & Daramola, E. O. (2020). Eliminating challenges to office technology and management profession in Nigeria: ICT approach. International Journal of Innovative Research and Advanced Studies, 7(10), 32-34.

Esene, R.A. (2017). Challenge facing office educators in the implementation of office technology and management curriculum and new technologies in polytechnics in South-south geo-political zone of Nigeria. World Educator Forum, 1-20.

Ghavifekr, S., & Rosdy, W.A.W. (2015). Teaching and learning with technology: Effectiveness of ICT integration in schools. International Journal of Research in Education and Science, 1(2), 175-191. https://files.eric.ed.gov/fulltext/EJ1105224.pdf Gyansah, S. T., & Guantai, H. K. (2018). Career development in organizations: Placing the organization and employee on the same pedestal to enhance maximum productivity. European Journal of Business and Management, 14(10), 40-45.

http://ijlter.org/index.php/ijlter

304

Jackson, D., & Wilton, N. (2017). Career choice status among undergraduates and the influence of career management competencies and perceived employability.

Journal of Education and Work, 30(5), 552-569.

Low, M., Botes, V., Rue, D. D., & Allen, J. (2016). Accounting employers’ expectations: The ideal accounting graduates. e-Journal of Business Education & Scholarship of Teaching, 10(1), 36-57. https://files.eric.ed.gov/fulltext/EJ1167364.pdf NBTE (National Board for Technical Education). (2004). Office Technology and Management: Higher National Diploma (HND) Curriculum and Course Specifications. https://www.academia.edu/27383038/Office_Technology_and_Management_ Higher_National_Diploma_HND_Curriculum_and_Course_Specifications Okolocha, C. C., & Nwadiani, C. O. (2015). Assessment of utilization of ICT resources in teaching among tertiary institution business educators in south Nigeria. Journal of Education and Learning, 4(1), 1-10. https://doi.org/10.5539/jel.v4n1p1 Omoniyi, O. J., & Elemure, B. C. (2014). Challenges of curriculum development in office technology and management in tertiary institutions. International Journal of Technology and Inclusive Education, 1(3), 475-484.

Otamiri, S. (2014). Business and technical education delivery in the 21 st century: The challenges of quality and functional skills. Journal of Economics and Sustainable Development, 5(11), 187-196.

Suarta, I. M., Suwintana, I. K., Sudhana, F. P., & Hariyanti, N. K. D. (2017). Employability skills required by the 21st-century workplace: A literature review of labour market demand. Advances in Social Science, Education and Humanities Research, 102, 337-342. https://doi.org/10.2991/ictvt-17.2017.58 Tomlinson, J., Baird, M., Berg, P., & Cooper, R. (2018). Flexible careers across the life course: Advancing theory, research and practice. Human Relations, 71(1), 4-22.

Ubulom, W. J., Enyekit, E. O., & Egwe, O. C. (2010). Instructional material usage and students’ academic achievement in business education. Rivers Business Educational Journal, 1(1), 1-8.

Udo, M. P. (2014). Attitude of vocational business education students towards acquiring maximum vocational business skills and competencies for sustainable development in Nigeria. Asian Journal of Social Sciences & Humanities, 3(4), 113123.

Villares, E., & Brigman, G. (2018). College/career success skills: Helping students experience postsecondary success. Professional School Counseling, 22, 1-8. https://doi.org/10.1177/2156759X19834444 Wakil, K., Muhamad, D., Sardar, K., & Jalal, S. (2017). The impact of teaching ICT for developing education systems. International Journal of Advanced Research, 5(7), 873879. http://dx.doi.org/10.21474/IJAR01/4793 Widjaja, A., & Saragih, E. J. (2018). Analysis on the effect of hard skills, intrapersonal and interpersonal skills toward the performance of nurses: A case study on the alumni of Husada Hospital Nursing Academy, Jakarta, Indonesia. Journal of Research in Business and Management, 6(5), 31-38.

http://ijlter.org/index.php/ijlter

304

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 304-329, February 2021 https://doi.org/10.26803/ijlter.20.2.17

Information Technology Integration Perception on Ghanaian Distance Higher Education: A Comparative Analysis Albert Arthur Qua-Enoo University of Education, Winneba, Ghana https://orcid.org/0000-0003-2254-188X Brandford Bervell, Paul Nyagorme, Valentina Arkorful and John K. E. Edumadze University of Cape Coast, Cape Coast, Ghana https://orcid.org/0000-0002-3925-6877 https://orcid.org/0000-0002-8941-7413 https://orcid.org/0000-0001-6658-3067 https://orcid.org/0000-0003-2422-4909

Abstract. This study examined students’ perception on the integration of Information Technology (IT) in Ghanaian distance Education, and provided a comparative analysis of the levels of integration among public universities in Ghana. The research was conducted at the distance learning centres of the 4 major public universities, namely: University of Ghana (UG), University of Cape Coast (UCC), University of Education, Winneba (UEW) and Kwame Nkrumah University of Science and Technology (KNUST). Questionnaires were used to collect data from a sample of 297 students, using a cluster sampling technique and the data were analysed using One-Way ANOVA and chi-square statistics. The study showed that although computer literacy among distance learners was high, technology integration was low and varied among the institutions. Smart phones were the major internet access devices. The online learning tools for self-efficacy were vital to IT integration. Based on the gathered results, recommendations such as implementing a robust national IT infrastructure, among others, were provided for reflection by higher education institutions offering distance education in Ghana.

Keywords: distance education; Technology; integration; Ghana

higher

education;

Information

1. Introduction The buzz word for reaching large number of students from diverse socio-cultural background simultaneously was open and distance education (DE). Revolution and recent advancements in computing and internet technologies had spawned ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

305

savvy distance learning opportunities compatible with different types of learners.

Wang and Sun (2001) traced the history of distance education to 1833 in Sweden through the “Composition through the medium of the Post.” Although, global statistics on DE was often rare, institutional, regional and national data on DE enrolment and graduation continued to grow with an ever-increasing pace.

According to Allen and Seaman (2015) total online enrolments in 2002 came in just under ten percent of total student population in the United States. Data from research institutions showed incremental growth in distance education.

Ghana started DE towards the end of the 20th century to make quality education at all levels more accessible and relevant to meet the learning needs of Ghanaians, so as to enhance their performance and improve the quality of their lives (Mensah & Owusu-Mensah, 2002). Ghana had been experiencing massive growth in distance learning in recent decades. University of Education of Winneba (UEW) started DE in 1996 whiles University of Ghana (UG) and Cape Coast (UCC) started DE in 1996 with diploma programs by distance in 2001/2002 academic year (Hope & Guiton, 2005). Distance enrolment had seen a 39.4% increase in enrolment from 2012 to 2014 (Ankomah-Asare et al., 2016). They also noted that 75% of total enrolment between 2008 and 2015 were done by UEW and UCC. Schaffhauser (2019) also observed that although DE was increasing annually, on-campus students had been declining by five percent (about 931,317 students) between 2012 and 2015. The following figure explicitly showed the percentage of students taking distance courses from 2012 to 2015.

Figure 1. Digital Learning Compass: Distance Education Enrolment Report 2017 Percentage Of Students Taking Distance Courses - 2012-2015 40.0% 30.0% 20.0% 10.0%

12.6%

14.1%

13.9%

14.5%

13.3%

13.1%

14.2%

15.4%

2012

2013

2014

2015

Some Distance Exclusive Distance

0.0%

Actually, several definitions of DE existed. According to Digest of Education Statistics (2014) DE is primarily delivered using live, pre-recorded instructional videos, interactive audio or video-conferencing, and computer-based systems delivered online. Similarly, Radford (2012) defined distance education as any online class or degree program entirely conducted online. It did not include correspondence courses. That is to say and according to Integrated Postsecondary Education Data System (IPEDS) (2013), DE is “education that uses one or more technologies to deliver instruction to students who are separated from the instructor and to support regular and substantive interaction between the students and the instructor synchronously or asynchronously” p. 21. As for ADEA

http://ijlter.org/index.php/ijlter

306

(2002), in DE the learner is separated from the instructional base or teacher, either in space or time, for a significant portion of their learning.

The rapid growth in DE, among other benefits, can be traced to the advancement in information technology. The latter had transformed firms into global networked infrastructures, with processes extended all over the world, making virtual global distributed markets and systems (Cunha & Goran, 2007; Tanye, 2017). In parallel, society had gradually transitioned from “technology age” to “information age” giving the computer, Internet and the World Wide Web an outstanding status which had revolutionized virtually every scope of human activity with mixed reception. Accordingly, education had evolved from the traditional face-to-face system to electronic learning and self-study using wiki, blogs, etc., web-based education (or WBTs), video/audio tape, virtual class (CBTs), distance education, mobile learning, and social learning. Endless opportunities and capacities now existed to educate large populations through the technology with particular emphasis on sophistication and learnercenteredness.

1.1. Statement of the Problem The demand for DE by the large teeming youth of Ghana was increasing with young workers preferring to work while upgrading themselves academically.

Also, high graduates’ unemployment had necessitated the acquisition of new skills through DE. Setup of new tertiary institutions was bureaucratic and capital intensive for the government and private institutions in Ghana. Many prospective tertiary students were unfortunately refused admission into the existing tertiary institutions due to incessant lack of facilities and lectures. In this sense, Agbofa (2012) also identified large student-lecturers ratio due to large enrolments. DE departments have been a great avenue for public universities to provide services for distance students. However, these public entities lack adequate resources (infrastructure, ICT personnel, applications, etc.) to match with the growing demand of DE. Yearly complaints from both fresh and continuing DE students were cyclical emanating from delays in supplying reading materials, registration issues, and quiz delays to the disaffection of students with institutional LMS. Some DE departments had instituted blended learning with varying success.

Therefore, this study undertook a comparative analysis on integrating ICT into distance learning program by DE students in four Ghanaian public universities.

1.2. Research Questions and Hypotheses The study sought to address the following research questions: 1. What is the mode of instructional delivery in distance education institutions?

2. What is the level of IT infrastructure integration in education institutions in the four Ghanaian public universities?

3. What is the self-efficacy of distance educational students towards ICT integration?

Based on the above stated research questions, the following research hypotheses were put forward: H01: There is no statistical difference in instructional delivery mode among the four distance education institutions.

http://ijlter.org/index.php/ijlter

307

H02: There is no statistical difference in level of IT integration infrastructure among the distance education institutions.

H03: There is no statistical difference in the self-efficacy of distance education students towards the use of online learning tools in the selected universities.

2. Literature Review This session involved reviewing literature relevant to distance education. It included theoretical framework, benefits of integrating information technology, overview of Ghana’s distance education, and finally challenges associated with distance education.

2.1.

Theoretical Framework There were several theories and theoretical frameworks proposed on successful implementation of distance education. Theories of DE attrition and persistence models had been proposed to explain why there was high dropout of DE student.

These ranges from Tinto’s student integration model in 1993, Bean and Metzner’s student attrition model in 1985, Kember’s longitudinal process model of dropout distance education in 1989 to Rovai’s composite persistence model in 2003. These models had been imperative to the enhancement of distance education globally (Wladis et al., 2015). Picciano (2017) identified online educational learning theories such as Community of Inquiry (Garrison et al., 2000), Anderson’s Online Learning Model in 2011, Blending with Pedagogical Purpose Model, and Multimodal Model for Online Education.

The successful setup of any DE was partly modelled after learning theories of traditional educational system. There were many frameworks, models and concepts deemed as vital elements of DE theories. Several DE theories in the 20th century had been explored by Lee (2004). The prominent theories include behaviourism, cognitivism, and social Constructivism. These theories had and continued to influence DE designs and practices. AECT (2001) identified 6 key elements of DE, namely: separation of teacher and learner; influence of an educational organization; use of media to link teacher and learner; two-way exchange of communication; learners as individuals rather than grouped; and educators as an industrialized form. According to Garrison (2000), theory in distance education must evolve to reflect current and emerging innovative practices of designing and delivering education at a distance. He further stressed on DE design adaptability through affordability and highly interactive communications technology.

In the same vein, ADEA (2002) identified (a) efficiency and effectiveness in administration and management; (b) curriculum design; (c) course production; (d) quality assurance; (e) learner support; (f) use of information and communication technologies; (g) continental, national and regional associations that promote ODL; (h) current research being conducted; initiatives supporting ODL developments; and (i) availability of national and regional databases as critical to the success of DE implementation in Sub Saharan Africa (SSH). Again, Moore and Kearsley (2012) proposed components of a working distance education system to include: (i) a source of content knowledge and teaching; (ii) a course design subsystem to structure this into materials and activities for students; (iii) a

http://ijlter.org/index.php/ijlter

308

subsystem that delivers the courses to learners through media and technology; (iv) a subsystem that delivers the courses to learners through media and technology; (v) instructors and support personnel who interact with learners as they use these materials; (vi) learners in their different environments; (vii) and a management subsystem to organize policy, needs assessment, and resource allocation to evaluate outcomes and to coordinate other subsystems.

2.2. Distance Education in Ghana Distance education had gained prominence in developing SSA countries. The history of DE had been comprehensively described by Mensah and OwusuMensah (2002). The demand for DE in SSA and Ghana in particular was increasing at a very fast rate (Ankomah-Asare et al., 2016; Kumi-Yeboah et al., 2013; Betchoo, 2015). Schaffhauser (2019) noted that on-campus enrolment was shrinking while online continued its ascent. SSA relied mostly on print media, simulated lecture video recording and actual tutored lectures at designated learning centres nationwide (Mnyanyi & Mbwette, 2009; Ankomah-Asare et al., 2016)). A sevenyear period analysis of DE in Ghana showed a total of 374,017 in total admissions with an annual rate of increment of 7984 students (Ankomah-Asare et al., 2016).

They also noted that out of the 374,017 enrolled during the period, less than 1% went to private institutions whiles more than 99% used public universities. This could be due to the history, experience, popularity, affordability, and more importantly the credibility of these public institutions. ADEA (2002) noted SSA countries were smartly using DE to address lifelong learning among the general adult population and to accelerate social development. Kumi-Yeboah et al. (2013) discussed the demographics of Ghana as well as Ghana’s history of distance education.

2.3. Benefits of Technology in Distance Education The advantages of technology had been vastly discussed by many researchers.

These covered: cost-effectiveness; access to knowledge; social interaction; pedagogical richness; personal agency; and ease of revision (Osguthrope & Graham, 2003). Many academic and governmental institutions included MIT Open Courseware, Carnegie Mellon University (CMU), Virtual Centre for Technology Enhanced Learning (VCTEL), University of Phoenix, Cisco Networking Academy, African Virtual University (AVU), National Institute of Information Technology (NIIT) Netvarsity, in which they were among the renowned institutions that offered electronic learning (Chandwani et al., 2010; Gerson, 2002).

Recent advancement in computer internet technologies had spawned new programs that combine email, wiki and blog groups, authoring tools, video conferencing and instant messaging via text, audio and video. Learning Management Systems (LMSs) had the amazing ability to integrate the above tools for education masses of distance students with flexibility and personalization. It promoted international, multi-disciplinary variations in educational practice and equipped faculties, students, and administrators with resources to compete in modern academia (Darkwa & Mazibuko, 2000). Embedded in LMS in managing DE were: Massive Online Open Courses (MOOC), Sharable Content Object Reference Model (SCORM), Learning Tools Interoperability (LTI), and Mobile

http://ijlter.org/index.php/ijlter

309

Learning (M-Learning). LMS was also referred to as Course Management System (CMS), Integrated Learning System (ILS) or Computer-based learning environment (CBI), and virtual learning platform. The use of computers in the teaching process, testing and evaluation process, guidance purposes, library, and school administration were cited as real life used in education, as claimed by Singh (2009). Annku (2014) reported high potential in using digital technologies to augment the traditional teaching method.

Significantly, online education continued to increase due to students’ preference and off campus constraints (Allen & Seaman, 2017). Many universities were leveraging on technology to increase their admission rate to replenish their oncampus student population. United States’ students enrolled exclusively in DE courses by institutional sector data trends from 2012 to 2016 showed percentage increase of 25 % public, 54.7% private, Non-Profit and a decline of 24.3% Private, For-Profit. Total percentage increase was 13.2%. (Taylor-Straut, 2017). DE can enhance in-service training of untrained teachers and professional upgrading in pre-tertiary education (Robinson & Latchem, 2002; Saint, 1999). African Virtual University (AVU) started with 57 learning centres in 27 African countries aimed at supporting economic development and offered many programs leading to certification in many educational programmes (African Virtual University, 2012).

Besides, the AVU helped in increasing access to university education to many SSAs through the use of ICT and multimedia (AVU, 2005).

2.4. Challenges of Distance Education In spite of the high demand for distance learning and the obvious advantages in transforming the economy of Ghana, several challenges existed. Asabere and Enguah (2012) asserted that DE programs generally required better management skills compared to traditional tertiary programs. ADEA (2002) posited that, for quality provision of distance education, a clear policy framework with clearly defined roles, responsibilities and lines of communication for DE managers was of fundamental importance. Access to DE continued to be a major challenge in Ghana although annual admission continued to grow. Leary and Berge (2007) opined that despite the continued development of information and communication technology (ICT), including videos, online training modules, and web-based training (WBT) systems, traditional DE delivery methods (printed manuals and texts) continued to prove as the most reliable, most sustainable, and most widely used. According to ADEA (2002) report, there was an apparent lack of culture of inter-institutional collaboration with regard to the establishment, maintenance and utilisation of ICT potential among DE institutions.

The report also noted that the use of ICTs in SSA was severely hampered by a lack of expertise, poor infrastructure and a largely technologically illiterate usergroup. Awareness and funding of ICT tools by institutions continued to be a challenge. Currently, the UG had deployed Sakai LMS, while KNUST and University of Professional Studies had adopted the Moodle LMS (Darko-Adjei, 2018). Other technological challenges of DE included frequent power outages; slow Internet bandwidths; limited telephone connections; low computer ownership; poor ICT infrastructure; and ICT experts (Dube, 2017; Kulshrestha & Ramswaroop, 2013; Hassan & Mirza, 2020). Globally, mainstream distance

http://ijlter.org/index.php/ijlter

310

education generally was designed without consideration for the physically disabled. Pant (2014) cited lack of the presence of a teacher; low status of DE institutes; rigidity imposed by university regulations; misconception about the role of DE departments; discrimination with the product of the DE departments; and lack of support by the faculty were actually some setbacks of DE. Kinyanjui (2000) noted the increasing pressure on African countries to provide Open and Distance learning institutions.

The abrupt transition of students from paper-based learning to an electronic one was a daunting task for DE students. Inadequate orientation and poor computer literacy skills can result in poor perception of electronic learning by students. Selfefficacy theory derived from psychology and presented a theoretical framework which accounted for human behaviour changes from diverse modes of treatment (Bandura, 1997), Darko-Adjei (2018) found that fresh DE students inadequately prepared for the Sakai LMS. In the same line of thought, most Senior High Schools (SHS) graduates got their first computer experience at the high schools’ ICT laboratory, which were inadequately furnished and often manned by poorly trained ICT staff, coupled with high student-to-computer ratio, and limited timetable period for practice (Nyagorme et al., 2017). Lack of computer selfefficacy or confidence posed a significant barrier to engaging in DE (Tagoe, 2012; Lee et al., 2005; Acheampong, 2016). Adequate self-efficacy in using online tools

by DE students was one of the significant determiners of persistence and achievement in DE (Joo et al., 2000; Zhang et al., 2001). Kuo (2010) identified Internet self-efficacy and self-regulation as typically the sole predictors of student satisfaction from previous studies.

3. Methodology The research design, instrumentation, data collection and data analysis were covered under this session.

3.1.

Research Design and Participants The survey research design was used to collect quantitative data for a quantitative analysis using descriptive and inferential statistics. A survey design was an approach that collected data through sampling from the population and used statistical analysis to make inferences about the population (Curtis & Curtis, 2011). The DE undergraduate students of the 4 major public universities, namely: UG, UCC, UEW and KNUST were used as the target population for the study.

These chosen universities were pioneers of DE in Ghana and also accounts for more than 90% of DE students in the country.

3.2.

Instrumentation The instrumentation for the study was used to broadly measure the demographics of the participants, the modes of instructional delivery, the level of IT infrastructure integration, and the online learning tools self-efficacy of distance educational students. Ethical principles such as informed consent, voluntary participation and confidentiality were factored into the preparation and administration of the questionnaires. Questionnaires consisted of 21 open and closed ended questions as well as Likert scale items. Both manual and online versions of the research instrument were employed. The collected data included

http://ijlter.org/index.php/ijlter

311

categorical, 4-point and 5-point Likert scales, and scale variables. It was modified to include manual as supplementary option since some DE students were less comfortable with the online option. The Cronbach’s alpha scores for modes of instructional delivery, level of IT infrastructure integration and online learning tools self-efficacy were 0.72, 0.68 and 0.85, respectively.

3.3. Procedure Permission was sought from the 4 university authorities and granted before administering the questionnaires. The study used quantitative data from a survey questionnaire consisting of participants’ regional distribution, age, gender, and educational level. Also, their skills of internet usage, mode of instructional delivery and assessment methods, and knowledge of institutional e-learning / learning management systems were also collected. Modes of internet access, bandwidth, hour per week and amount (in Ghana cedis) spent on the internet were also identified. The purposes for using the internet, frequency of usage and the confidence in using it were also explored. The researchers created an online survey using Google forms for the pilot study and administered to 51 participants from UCC and UEW public universities. The online questionnaire was reviewed to include a manual version as a complementary option for the less tech savvy participants.

Questionnaires were used to elicit information from a sample of 297 respondents from the 4 public universities in the descriptive survey. The sample size included 26.2% (n = 77) from KNUST, 20.7% (n = 61) from UCC, 25.2% (n = 74) from UEW, and 27.9% (n = 82) from UG, as shown in Table 1. The cluster sampling method was employed due to economy, reduced variability, and feasibility as the DE study centres were many and scattered across the country (Creswell, 2009). Out of ten regions, only 4 were selected. Sever study centres were then selected from Accra, Kumasi and Sogakope for the survey. Simple random sampling was then used to sample the participants for this study. Data analysis and presentation were done using the Statistical Package for Social Sciences (SPSS) version 21, and both descriptive and inferential statistics were done. The descriptive test involved cross-tabulations, bar charts, and pie-charts. The One-Way analysis of variance (ANOVA) and chi-square statistics were used to test the hypotheses of the study.

The tukey test for post-hoc analysis was employed to determine if the relationships between the sets of data were statistically significant.

4. Results The study analysed comparative analysis of public universities in Ghana in integrating IT into DE as perceived by the participants.

Table 1. Participants’ Demographic Information Demographic Variables Region Ashanti Central Greater Accra Volta Total

KNUST 0 (0.0) 64 (26.3) 13 (40.6) 0 (0.0) 77 (26.2)

http://ijlter.org/index.php/ijlter

UCC 1 (100.0) 42 (17.3) 18 (56.3) 0 (0.0) 61 (20.7)

UEW 0 (0.0) 55 (22.6) 1 (3.1) 18 (100.0) 74 (25.2)

UG 0 (0.0) 82 (33.7) 0 (0.0) 0 (0.0) 82 (27.9)

Total 1 (0.3) 243 (82.7) 32 (10.9) 18 (6.1) 294 (100.0)

312

Gender

Female Male Total

Age

Up to 30 years 31 - 35 years 36 - 40 years 41 - 45 years 46 - 50 years 51 and above Total Students’ 1st year Level 2nd year 3rd year 4th year Masters Total

15 (12.3) 61(37.4) 76 (26.7) 32 (17.3) 31 (46.3) 12(50.0) 1(20.0) 0 (0.0) 0 (0.0) 76 (26.8) 10 (58.8) 8 (7.7) 23 (38.3) 11 (14.3) 22 (95.7) 74 (26.3)

38 (31.1) 24 (14.7) 62 (21.8) 40 (21.6) 18 (26.9) 1 (4.2) 1 (20.0) 0 (0.0) 0 (0.0) 60 (21.1) 1 (5.9) 21 (20.2) 6 (10.0) 32 (41.6) 0 (0.0) 60 (21.4)

33 (27.0) 37(22.7) 70 (24.6) 44 (23.8) 11 (16.4) 9 (37.5) 3 (60.0) 0 (0.0) 0 (0.0) 67 (23.6) 4 (23.5) 25 (24.0) 11(18.3) 26 (33.8) 1 (4.3) 67 (23.8)

36 (29.5) 41(25.2) 77 (27.0) 69 (37.3) 7 (10.4) 2 (8.3) 0 (0.0) 1 (100.0) 2 (100.0) 81 (28.5) 2 (11.8) 50 (48.1) 20 (33.3) 8 (10.4) 0 (0.0) 80 (28.5)

122 (42.8) 163 (57.2) 285 (100.0) 185 (65.1) 67 (23.6) 24 (8.5) 5 (1.8) 1 (0.3) 2 (0.7) 284 (100.0) 17 (6.0) 104 (37.0) 60 (21.4) 77 (27.4) 23 (8.2) 281 (100.0)

Table 1 also showed the regional distribution, gender, age, and students’ classification of the DE students in the 4 public universities in Ghana. Out of the 297 participants, 1(0.3%) was from Ashanti region, 18 (6.1%) were from Volta region, 32 (10.9%) were from Greater Accra region, and the majority (82.7%) were from central region. The gathered data reflected that 122 (42.8%) were females while 163 (57.2%) were males. Besides, the table also displayed research informants’ different ages, in which most of the participants 185 (65.1%) were up to 30 years, 67 (23.6%) were between 31 - 35 years, 24 (8.5%) were between 36 - 40 years, 5 (1.8%) were between 41 - 45 years, 1 (0.3%) were between 46 - 50 years, and finally 2 (0.7%) were 51 years and above. As for students’ classification, 17 (6.0%) were in first year, 104 (37.0%) were in second year, 60 (21.4%) were in third year, 77 (27.4%) were in final year, and 23 (8.2%) were in graduate school.

Table 2. Instruction Received through Technology

No Yes Total

KNUST N % 20 14.5 57 36.5 77 26.2

Distance Education Institution UCC UEW UG N % N % N % 52 37.7 45 32.6 21 15.2 11 7.1 28 17.9 60 38.5 63 21.4 73 24.8 81 27.6

Total N % 138 46.9 156 53.1 294 100

As shown from Table 2, the majority of the participants (53.1%) stated the institutional use of educational technology, while 46.9% stated no.

http://ijlter.org/index.php/ijlter

313

Yes

KNUST 37%

38%

UCC UEW UG

18%

7%

Figure 2. Instructional Technology Usage by DE Institution

In details, 60 (38.5%) of UG students stated yes, followed by 57 (36.5%) KNUST students, then 28 (17.9%) UEW students, and finally 11 (7.1%) UCC students as revealed in Figure 2. The homogeneous test consisted of KNUST and UG in a group with UCC and UEW in other separate groups using Tukey HSDa,b test (See Table 7).

Table 3. Institutional Instructional Media Distance Education Institution

None Printed Media Television Computer software CD ROMs Video-conferencing Audio/Video tapes Total

KNUST 28(17.8%) 4(12.9%) 1(33.3%) 28 (35.4%) 0(0.0%) 6(66.7%) 10(66.7%) 77(26.1%)

UCC 54(34.4%) 7(22.6%) 1(33.3%) 1(1.3%) 0(0.0%) 0(0.0%) 0(0.0%) 63(21.4%)

UEW 45(28.7%) 10(32.3%) 0(0.0%) 19(24.1%) 0(0.0%) 0(0.0%) 0(0.0%) 74(25.1%)

Total

UG 30(19.1%) 157(53.2%) 10(32.3%) 31(10.5%) 1(33.3%) 3(1.0%) 31(39.2%) 79(26.8%) 1(100%) 1(0.3%) 3(33.3%) 9(3.1%) 5(33.3%) 15(5.1%) 81(27.5%) 295 (100.0%)

Table 3 reflected the instructional media used by DE institutions. A larger proportion of participants 157 (53.2%) did not state any instructional media.

However, out of the 138 (46.8%) who specified instructional media mode, 31(10.5%), 3(1.0%), 79(26.8%), 1(0.3%), 9(3.1%), 15(5.1%) stated printed materials, television, computer software, CD ROMs, Audio/Video-conferencing, and Audio/Video tapes, respectively. Both UEW and UG students reported the highest score (32.3%) in the use of printed material. Television, Audio /Videoconferencing, Audio/Video tapes, and CD ROMs were not a popular educational technology for all the institution reporting less than 12 participants in the universities. Computer softwares were more than a third (>33.3%) of instructional technology according to KNUST and UG students. KNUST and UG had virtually similar responses compared to UCC and UEW which were also in separate groups as table 7 showed.

http://ijlter.org/index.php/ijlter

314

Table 4. Mode of Assessment (quizzes, tests, etc.) Conducted

Manual Online Mixed Total

Distance Education institution.

KNUST UCC UEW UG Total Freq. % Freq. % Freq. % Freq. % Freq. % 53 28.5 58 31.2 69 37.1 6 3.2 186 65.3 1 2.6 0 0.0 0 0.0 38 97.4 39 13.7 22 36.7 3 5.0 1 1.7 34 56.7 60 21.1 76 26.7 61 21.4 70 24.6 78 27.4 285 100.0

The popular mode of assessment (quizzes, tests, group work) reported by students was paper-based 168 (65.3%), with 39 (13.7%) stating online mode, while 60 (21.1%) reporting both the manual and online modes. On institutional basis, KNUST, UCC and UEW were heavily dependent on paper-made assignments, whiles UG focused more on online assignments and to a less extend mixed modes as Table 4 displayed. Also, UEW and UCC were found in the same homogeneous subset and varied differently from KNUST and UG (See Table 7).

Figure 3. Mode of End-of-Semester Examination

Figure 3 reflected on the fact that all the participating institutions focused on paper-based end of semester examinations 259 (89.0%).

Online mode was lowest 10 (3.4%) with the rest 22 (7.6%) stating the mixed method. UEW, UCC and KNUST were found in the same homogeneous subset with UG being different according to what Table 7 covered.

Table 5. Immediate Feedback on Assessments KNUST

No Yes Total

Freq.

53 14 67

Distance Education Institution UCC UEW UG

% Freq.

26.9 60 20.0 0 25.1 60

http://ijlter.org/index.php/ijlter

% Freq.

30.5 60 0.0 4 22.5 64

% Freq.

30.5 24 5.7 52 24.0 76

Total % Freq.

12.2 197 74.3 70 28.5 267

% 73.8 26.2 100.0

315

138 (48.6%) of the participants stated no immediate feedback on formative assessments was conducted by lecturers, while 146 (51.4%) had immediate feedback as revealed from table 5. For the yes group, UG and UCC recorded relative higher values with 47(32.2%) and 38 (26.0%) respectively compared to KNUST 35 (24.0%) and UEW 26 (17.8%).

Table 6. Feedback Mode

Face-to-face Online Phone Total

Distance Education Institution KNUST UCC UEW UG Total Freq. % Freq. % Freq. % Freq. % Freq. % 29 28.2 38 36.9 28 27.2 8 7.8 103 67.3 5 10.6 1 2.1 0 0.0 41 87.2 47 30.7 1 33.3 0 0.0 1 33.3 1 33.3 3 2.0 35 22.9 39 25.5 29 19.0 50 32.7 153 100.0

Table 6 above showed the three main options available to students in communicating with lecturers. Out of the 297 participants, 158 (53.2%) interacted with their lecturers on assignments. Face-to-face feedback was the major option 103 (67.3%) with 47 (30.7%) using online options and 3(2.0%) using the mobile phone.

Table 7. Tukey HSD Tests Institution mode of Instruction 1 2 3 UEW .38 KNUST .74 UG .74 UCC .17

Forms of technology 1 2 6.57 5.00 5.37

Assignments Mode 3 1 2 3 1.03 1.59 2.36 7.59 1.10

End-of-Sem Mode 1 2 1.00 1.12 1.55 1.03

Feedback Mode 1 2 1.07 1.20 1.86 1.03

Homogeneity test of the variables above showed interesting results among the four DE universities according. There were 3 subgroups using instructional technology with UCC in 1 group, UEW in the next group, with KNUST and UG in the third group. A similar pattern was observed among the institutions with the forms of technology. Assignment mode also had 3 different subgroups with UEW and UCC in a single group with KNUST and UG in different groups. End of semester mode and feedback mode had two subgroups each with similar groupings: UEW, KNUST and UCC in the first group and UG in the second group.

4.1. IT Infrastructure Integration in Distance Educational Institutions This session dealt with the level of IT integration with the services offered by distance educational institutions in available wireless-fidelity (Wi-Fi), e-learning, and LMS.

http://ijlter.org/index.php/ijlter

316

Table 8. Reliable Wi-Fi Internet Connectivity at the Learning Centre for Effective Studies Distance Education Institution KNUST UCC UEW UG Freq. % Freq. % Freq. % Freq.

No 53 26.9 60 30.5 60 30.5 24 Yes 14 20.0 0 0.0 4 5.7 52 Total 67 25.1 60 22.5 64 24.0 76

Total % Freq. % 12.2 197 73.8 74.3 70 26.2 28.5 267 100.0

Table 8 covered the presence of reliable Wi-Fi internet facility at the learning centres for effective studies. As shown, 267 out of 297 participants answered this question item, in which 197 (73.8%) stated the absence of Wi-Fi internet, whereas 70 (26.2%) stating its presence. Also, only UG participants responded more yes (68.6%) than no (31.6%).

Table 9. Internet Speed Internet Speed Very Fast Fast Manageable Slow Very Slow Total

KNUST Freq. % 5 23.8 38 31.7 28 32.9 6 12.2 0 0.0 77 27.1

Distance Education Institution UCC UEW UG Freq. % Freq. % Freq. % 6 28.6 4 19.0 6 28.6 27 22.5 28 23.3 27 22.5 18 21.2 13 15.3 26 30.6 10 20.4 16 32.7 17 34.7 0 0.0 5 55.6 4 44.4 61 21.5 66 23.2 80 28.2

Total Freq. % 21 7.4 120 42.3 85 29.9 49 17.3 9 3.1 284 100.0

As far as DE participants’ view of the internet bandwidth for using online resources are concerned, 21 (7.4%) indicated it as very fast, 120 (42.3%) stated fast, while 85 (29.9%) noted it as manageable, 49 (17.3%) stated slow, and 9 (3.1%) declared very slow. Generally, out of the 226 participants who were comfortable with the internet speed, 71 (31.4%) were from KNUST, 51 (22.7%) were from UCC, 45 (19.9%) were from UEW and 59 (26.0%) were from UG. Conversely, participants with unstable and low bandwidth were 6 (10.3%) from KNUST, 10 (17.2%) from UCC, 21 (36.2%) from UEW, and 21 (36.2%) from UG. Table 10. E-learning Platform Used by Academic Institutions Unanswered None Not Sure Internet V CLASS Schoology National Program for Technology Enhanced Learning Sakai Total

Frequency 182 2 7 13 27 1 2 63 297

Percent (%) 61.3 .7 2.4 4.4 9.1 .3 .7 21.1 100.0

When investigating the types of e-learning services offered by DE institutions, different data were gathered. As shown in Table 10, 182 (61.3%) participants did not state any e-learning platform, 9 (3.0%) stated either none or not sure. As for

http://ijlter.org/index.php/ijlter

317

the remaining research informants 106 (35.7%), 63 stated Sakai LMS, 27 stated virtual (V) class, 2 stated NPTEL, 1 stated Schoology LMS, and 13 stated just the Internet.

The use of LMSs by DE institutions was investigated and data were gathered from participants as found in Figure 4.

Instistution Use of LMS 150

No

100

Yes

50

Not Sure

0

KNUST

UCC

UEW

UG

Figure 4. Engage in LMS in your Institution

From Figure 4, 234 (78.8%) participants answered this out of 63 (21.2%). Also, 90 (38.5%) students said no, 115 (49.1%) said they were not sure, and 29 (12.4%) stated yes. This implied that 209 (87.6%) of DE students had no engagement with LMSs. Only UG students had more yes 20 (69.0%) responses than no 13 (14.4%) responses.

Table 11. State any LMS Used by your Academic Institution Unanswered Lamass Internet Not Sure Sakai Total

Frequency 272 2 4 4 16 298

Percent (%) 91.3 0.7 1.3 1.3 5.4 100.0

The types of LMSs used by the 4 public universities were shown in Table 11. The majority of participants (91.3%) did not answer this question item compared to participants who did (8.7%), and (7.4%) who actually stated the particular LMSs. Findings also revealed that 16 (5.4%) stated Sakai LMSs, 2 (0.7%) mentioned Lamass ,and 4 (1.3%) stated the internet.

4.2. DE Students’ Self Efficacy towards Online Learning Tools The self-efficacy towards online learning tools by DE participants was centred on the average weekly time spent on browsing the internet, internet bandwidth, and confidence in using online resources.

http://ijlter.org/index.php/ijlter

318

Table 12. Average Weekly Internet Usage Hours / week

KNUST Freq. % 9 18.4 27 29.3 12 29.3 13 43.3 3 12.5 11 27.5 75 27.2

0-1 2-4 5–6 7-9 10 – 20 Above 20 Total

Distance Education Institution UCC UEW UG Freq. % Freq. % Freq. % 9 18.4 18 26.7 13 26.5 18 19.6 27 29.3 20 21.7 14 34.1 5 12.2 10 24.4 6 20.0 6 20.0 5 16.7 5 20.8 5 20.8 11 45.7 8 20.0 6 15.0 15 37.5 60 21.7 67 24.3 74 26.8

Total Freq. % 49 17.7 92 33.3 41 14.9 30 10.9 24 8.7 40 14.5 276 100.0

Table 12 displayed average weekly usage spent on the internet. On weekly basis, 49 (17.7%) participants spent about 0 – 1 hour, 92 (33.3%) spent 2 – 4 hours, 41 (14.9%) spent 5 – 6 hours, 30 (10.9%) spent 7 - 8 hours, 24 (8.7%) spent 10 – 20 hours, and 40 (14.5%) spent over 20 hours. Comparatively, all the DE participants mostly spent 2 – 4 hours weekly on the internet.

Table 13. Confidence in Using the Computer for Education

Very Confident Quite confident Confident Not confident Don't Know Total

Distance Education Institution KNUST UCC UEW UG 46(34.8%) 26(19.7%) 30(22.7%) 30(22.7%) 15(17.6%) 18(21.2%) 26(30.6%) 26(30.6%) 13(26.0%) 12(24.0%) 10(20.0%) 15(30.0%) 0(0.0%) 3(27.3%) 3(27.3%) 5(45.5%) 0(0.0%) 0(0.0%) 0(0.0%) 2(100.0%) 74(26.4%) 59(21.1%) 69(24.9%) 78(27.9%)

Total 132(47.1%) 85(30.4%) 50(17.9%) 11(3.9%) 2(0.7%) 280(100.0%)

The confidence of DE students in using computers for educational purpose was identified, in which 132 (47.1%) were very confident, and 85 (30.4%) were quite confident. While 50 research informants (17.9%) were confident and 11 (3.9%) were not confident, 2 (0.7%) did not know. Therefore, it is safe to conclude that data on the confidence in using the computer for education in each university was significant to a great extent, wherein 74 (27.7%), 56 (21.0%), 66 (24.7%), and 71 (26.6%) were respectively recorded in KNUST, UCC, UEW, and UG public universities, Table 14. Confidence in Using a Web-Browser

Very Confident Quite Confident Confident Not Confident Don't Know Total

Distance Education Institution KNUST UCC UEW UG 52(33.5%) 31(20.0%) 32(20.6%) 40(25.8%) 11(17.2%) 16(25.0%) 14(21.9%) 23(35.9%) 10(22.2%) 12(26.7%) 12(26.7%) 11(24.4%) 0(0.0%) 0(0.0%) 11(84.6%) 2(15.4%) 0(0.0%) 1(33.3%) 0(0.0%) 2(66.7%) 73(26.1%) 60(21.4%) 69(24.6%) 78(27.9%)

Total 155(55.4%) 64(22.9%) 45(16.1%) 13(4.6%) 3(1.0%) 280(100.0%)

Table 14 showed participants’ confidence in using a web-browser. Generally, 155 (55.4%) were very confident, 64 (22.9%) were quite confident, 45 (16.1%) were confident, 13(4.6%) were not confident, and 3(1.0%) did not know. 264 (94.3%)

http://ijlter.org/index.php/ijlter

319

were in the confidence group while 16 (5.7%) were in the non-confidence group.

Institutionally the confidence group had 73 (27.7%) from KNUST, 59 (22.3%) from UCC, 58 (22.0%) from UEW and 74 (28.0%) from UG participants.

Table 15. Confidence in Using Wikipedia KNUST 31(32.6%) 17(22.7%) 13(26.0%) 5(20.0%) 7(22.6%) 73(26.4%)

Very Confident Quite confident Confident Not confident Don't Know Total

Distance Education Institution UCC UEW 21(22.1%) 18(18.9%) 13(17.3%) 17(22.7) 12(24.0%) 12(24.0%) 3(12.0%) 11(44.0%) 7(22.6%) 11(35.5%) 56(20.3%) 69(25.0%)

Total UG 25(26.3%) 95(34.4%) 28(37.3%) 75(27.2%) 13(26.0%) 50(18.1%) 6(24.0%) 25(9.1%) 6(19.4%) 31(11.2%) 78(28.3%) 276(100.0%)

As far as students’ confidence level in using Wikipedia was concerned, the following data were collected. 95(34.4%) were very confident, 75(27.2%) were quite confident, 50(18.1%) were confident, 25(9.1%) were not confident, and 31(11.2%) did not know. The sum of the confidence group was 220 (79.7%) and that of the non-confidence group was 56 (20.3%). Also, the confidence group had 61 (27.7%) for KNUST, 46 (21.0%) UCC, 66 (24.7%) for UEW, and 71 (26.6%) for UG. Table 16. Confidence in Downloading from the Internet

Very Confident Quite confident Confident Not confident Don't Know Total

KNUST 46(34.1%) 11(16.7%) 16(27.1%) 1(7.1%) 0(0.0%) 74(26.4%)

Distance Education Institution UCC UEW 28(20.7%) 30(22.2%) 14(21.2%) 16(24.2%) 9(15.3%) 16(27.1%) 8(57.1%) 4(28.6%) 1(16.7%) 3(50.0%) 60(21.4%) 69(24.6%)

Total UG 31(23.0%) 25(37.9%) 18(30.5%) 1(7.1%) 2(33.3%) 77(27.5%)

135(48.2%) 66(23.6%) 59(21.1%) 14(5.0%) 6(2.1%) 280(100.0%)

Confidence of DE students in downloading information from the internet was investigated and summarised as reflected in the above table. 135 (48.2%) indicated that they were very confident, 66 (23.6%) indicated quite confident, and 59 (21.1%) indicated confident summing up to 260 (92.9%). Also 14 (5.0%) stated the fact that they were not confident and 6 (2.1%) opted for the last alternative tallying 20 (7.1%). Institutionally, 73 (28.1%) were from KNUST, 51 (19.6%) were from UCC, 62 (23.8%) were from UEW and 74 (28.5%) were from UG. Table 17. Confidence in Uploading in the Internet

Very Confident Quite confident Confident Not confident Don't Know Total

Distance Education Institution KNUST UCC UEW UG 31(30.4%) 25(24.5%) 18(17.6%) 28(27.5%) 15(23.4%) 11(17.2%) 19(29.7%) 19(29.7%) 23(34.3%) 8(11.2%) 16(23.9%) 20(29.9%) 2(6.9%) 10(34.5%) 10(34.5%) 7(24.1%) 3(25.0%) 1(8.3%) 4(33.3%) 4(33.3%) 74(27.0%) 55(21.1%) 67(24.5%) 78(28.5%)

http://ijlter.org/index.php/ijlter

Total 102(37.2%) 64(23.4%) 67(24.5%) 29(10.6%) 12(4.4%) 274(100.0%)

320

Table 17 recorded the confidence of participants in uploading data in the internet.

As displayed, 102 (37.2%) were very confident, while 64 (23.4%) and 67 (24.5%) were quite confident and confident, respectively. Surprisingly, 29 (10.6%) were not confident and 12 (4.4%) noted that they don’t know. More significantly, 233 (85.0%) indicated some degree of confidence and 41(15.0%) noted the lack of confidence in uploading contents in the internet. From the former group, 69 (29.6%) were from KNUST, 44 (18.9%) were from UCC, 53 (22.7%) were from UEW, and 67 (28.8%) were from UG.

Hypothesis One H01: There is no statistical difference in instructional delivery mode among the four distance education institutions Table 18. One Way ANOVA Table of Mode of Instructional Delivery Receive instruction through technology Forms of technology used to receive instructions Mode of assignments (quizzes, tests, etc.) conducted Mode of end-of-semester examinations conducted Immediate feedback on the assessment from lecturers Feedback Mode

DF 3 3 3 3 3 3

F 28.171 23.246 71.899 19.348 4.350 49.005

Sig. .000 .000 .000 .000 .005 .000

The One-Way ANOVA for each variable under the mode of instructional delivery system of the 4 public institutions was statistically different with p< 0.05 according to Table 18. With the df=3, instruction through technology (F=28.171, p=0.000); forms of instructional technology (F=23.246, p=0.000); mode of assignments (F=71.899, p=0.000); mode of semester exams (F=19.348, p=0.000); immediate feedback (F=4.350, p=0.005); and feedback mode (F=49.005, p=0.000) showed different and important data. The gathered data implied that each DE institution had different policies and practices. In other terms, each DE institution was autonomous in the implementation of DE. Therefore, it was safe to reject hypothesis 1 (H01).

Hypothesis Two H02: There is no statistical difference in level of IT integration infrastructure among the distance education institutions Table 19. One Way ANOVA for Internet Speed at Learning Centre for Effective Studies Sum of Squares 8.883 248.339 257.222

Between Groups Within Groups Total

DF 3 280 283

Mean Square F 2.961 3.338 .887

Table 20. Tukey Test for Internet Speed N UEW KNUST UG UCC

http://ijlter.org/index.php/ijlter

66 77 80 61

Subset for alpha = 0.05 1 2.85 2.45 2.83 2.52

Sig. .020

321

Tables 19 and 20 represented one-way ANOVA of the internet speed across the learning institutions and the Tukey homogeneity test, respectively. The value of p<0.05 in table 19 implied statistically significant difference in the internet speed for using online resources learning centres (F=3.338, df=3, p=0.020). Hypothesis 2 was therefore rejected. However, all the different institutions were in one homogeneous group according to the Tukey’s test.

Table 21. Chi Square Test of Presence of Wi-Fi Connectivity at Learning Centres Value DF Asymp. Sig. (2-sided) 105.472a 3 .000 Pearson Chi-Square Likelihood Ratio 113.814 3 .000 44.176 1 .000 Linear-by-Linear Association 267 N of Valid Cases a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 15.73.

According to Table 21, there was a statistically significant difference among DE institutions offering Wi-Fi connectivity at the learning centres with χ2 (2, N=267) = 105.472, p=.000. Hence, hypothesis 2 was also rejected.

Null Hypothesis Test Sig. Decision The distribution of ‘Engage One-Sample .000 Reject the in LMS in your institution is Kolmogorovnull normal’ with mean 1.107 and Smirnov Test hypothesis standard deviation 0.93 Figure 5. Chi Square test of Institutional LMS Asymptotic significance is displayed. The level is 0.05 Table 22. Chi Square Test of Presence of Institutional LMS Total N

234

Most Extreme Difference Absolute .323

Positive .267

Test Statistic

Negative -.323

Asymptotic Sig. (2-sided test)

4.934

.000

Both Figure 5 and Table 22 showed that there was a statistical significant difference in the institutional utilization of LMS among the DE institutions χ2(2, N=234) = 4.934, p=.000. Hypothesis 2 was hence rejected.

Hypothesis Three H03: There is no statistical difference in the self-efficacy of distance education students towards the use of online learning tools in the selected universities.

Table 23. One-Way ANOVA for Self-Efficacy in Online Learning Tools Confident in using the computer for education Confident in using opening a web-browser (Firefox, Chrome, etc.)

Confident in using Wikipedia Confident in using search engine Confident in downloading on the internet Confident in uploading on the internet

http://ijlter.org/index.php/ijlter

DF 3 3

F 3.364 4.875

Sig. .019 .003

3 3 3 3

4.085 2.731 2.461 1.412

.075 .198 .063 .240

322

Table 23 covered the One-Way ANOVA for the self-efficacy of DE students in using online learning tools. There were significant statistical differences for educational use of computer with F (3, N=280) = 3.364, p=.019, and using web browsers with F (3, N=280) = 4.875, p=.003 among the different institutions.

However, there were no statistical differences among the participants in the use of Wikipedia F (3, N=276) =4.085, p=.075; search engine F (3, N=277) = 2.731, p=.198; downloading F (3, N=280) =2.461, p=.063); and uploading F (3, N=274) = 1.412, p=.240.

Table 24. Tukey HSDa,b Test

UEW KNUST UG UCC

Confident Confident Confident Confident N computer for In web in in education browser Wikipedia search engine 1 2 1 2 1 1 64 1.80 1.80 2.03 2.71 2.41 67 1.55 1.42 2.18 2.00 76 2.01 1.76 1.76 2.23 2.35 60 1.86 1.86 1.73 1.73 2.32 2.40

Confident Confident in in download upload 1 1 2.04 2.45 1. 62 2.07 1.94 2.23 2.00 2.11

The post hoc results in Table 24 showed that confidence in computer use for education had 2 subgroups with UEW, KNUST and UCC in one group with UEW, UG and UCC in the other group. Similarly, the confidence in using web browsers yielded 2 subgroups with KNUST, UG and UCC in one group and UEW, UG and UCC in the other. The homogeneity test for confidence in using Wikipedia, search engines, download and upload were felt in single groups.

5. Discussion This study sought to compare the level of IT integration into distance learning programs of 4 public universities in Ghana. The main variables investigated were the demographics of the participants, mode of instructional delivery in distance education institutions, level of IT integration infrastructure, and the online learning tools self-efficacy of distance educational students. Accordingly, the following paragraphs covered an in depth discussion of the collected data for the set research questions.

Research Question 1 What is the mode of instructional delivery in distance education institutions?

The mode of instructional delivery in distance educational institutions was predominantly manual or paper-based popularly known as face-to-face tutorials.

Textbooks and photocopied items served as the main medium for knowledge transfer. Although the majority of the participants responded yes (53.1%) to instructional use of technology, table 3 showed that 53.2% were unable to list specific technologies. Zaina et al., (2001) analysed the main aspects of 5 web-based distance education system used by DE institutions. Despite the numerous, robust, inexpensive and open source learning management systems available for efficient management of DE, only 2 of the participating institutions functionally used them for their students in a blended mode. UCC and UEW were yet to utilize LMS to enhance their activities. The findings met in this study did not align with what Raja and Nagasubramani (2018) claimed. That is to say, they regard technology as

http://ijlter.org/index.php/ijlter

323

making the teaching and learning process more enjoyable. Research was ambivalent on the impact of technology (virtual worlds) on students’ academic performance. While a positive correlation was found among them (Sun et al., 2008; Gregory & Gregory, 2011) the reverse phenomenon was observed in other studies (Strasburger et al., 2010; Dalgarno et al., 2009).

The high percentage for paper-based formative (65.3%) and summative (89.0%) assessments the DE programs of the 4 public universities raised numerous underlying issues in relation to ICT integration. These underlying factors included lecturers and tutors’ adoption, perception, training and usage of ICT for DE, adequate instructional/course designer; prospective students’ awareness, preparation, prior IT literacy skills, perception, motivation, demand and relevance of ICT for DE, institutional and national policies for ICT implementation in DE, and ease of accessibility of the online platforms were critical to transform traditional mode to current modern trend. Darko-Adjei (2018) noted involuntary attitude of UG students in using the Sakai platform. The high variability of the mode of instructional delivery among the 4 major DE institutions as shown by ANOVA table 18 reflected the lack of standardization and collaboration of programs (Elameer & Idrus, 2011). Also, lack of a national examination body implied that DE graduates from each of the public universities completed their programs with different skill sets although they engaged in similar educational programs. In this vein, effective partnership between government and academic institutions is needed (Kinyanjui, 2000).

Research Question 2 What is the level of IT infrastructure integration in education institutions in the four Ghanaian public universities?

The study showed that students enrolling in DE in Ghana had internet experience, evidenced by 283 (96.3%) of the participants. Also, portable devices, especially mobile phone (53.4%), were the main internet accessing devices implying high penetration among tertiary students from the study. With the majority of participants reporting absence of Wi-Fi internet (73.8%), DE institutions’ quest for seamless ICT integration needed to be reviewed. 153 (53.9%) participants were unaware or not sure that their universities used e-learning and 191(64.4%) could not state e-learning platform. This, again, was attested by the fact that 205 (87.6%) participants either said no or were not sure whether their DE institution used LMS, and 288 (92.6%) could not state institutional LMS. Caruth and Caruth (2013) noted USA enrolment rate exceeded total rate in higher education. AnkomahAsare et al. (2016) also confirmed this finding. University of Ghana deployed Sakai LMS in 2012 (Oheneba-Sakyi & Amponsah, 2018). KNUST used Virtual Classroom (V class) for its students (Ofosu-Asare, 2017).

Both the ANOVA and the Chi square tests for the level of IT integration infrastructure in the DE institutions underscored the vast disparity of IT infrastructure and the challenges, both institutions and DE students have integrated educational technology. Nyerere et al. (2012), Özen (2012), and Usluel, Mumcu and Demirarslan (2007) reported similar challenges with IT integration.

Huge investment in technology and training by the government was needed to provide an integrated and uniformly distributed DE and IT infrastructure. In this

http://ijlter.org/index.php/ijlter

324

concern, Mensah and Owusu-Mensah (2002) recommended networking all learning centres to facilitate its integration.

Research Question 3 What is the self-efficacy of distance educational students towards ICT integration?

The time spent online for educational studies by the 76.8% of participants was less than 10 hours per week due to the predominantly traditional nature of instructional delivery in DE. Blended and pure e-learning modes can significantly increase the online hours of DE students to improve their self-efficacy. There were similarities, among all the DE students, in the self-efficacy of using Wikipedia, search engines, downloading and uploading data with their p-values greater than 0.05 from the ANOVA table. However, self-efficacy of the different DE students for both computers for education and web browsers were statistically significant.

Exposure to different educational programs and ease of use or popularity of particular web browsers could account for these variations. In addition, the selfefficacy of the distance education students was adequate and varied for engagement with online distance learning. These results resonated to that of Kuo (2010), Zhang et al. (2001), and Joo et al. (2000).

6. Conclusion The distance education sector of Ghana continued to evolve from the manual and labour intensiveness to current and internationally technology – driven standards.

The public universities needed to harness the immerse potential of information and communication technology to offer online programs both nationally and globally. Old barriers to education such as distance, accommodation, and cost and internet coverage had been substantially minimized or eliminated to provide both tertiary and lifelong learning to potential learners of Ghana and beyond. A paradigm shift is needed to transform the traditional DE to a fully technologically integrated one. Registration of new applicants, managing of finance and student records has been IT integration. However, provision of full online academic programs continues to be a challenge to public universities distance education faculties. More focus and investments need to be channelled into these areas to enable them offer services to more local and international distance learners.

Furthermore, the lack of adequate manual instructional materials for learners coupled with periodic delays in posting of assessment results would be immensely minimized.

7. Limitations There were some limitations in this study. The study was confined to distance education undergraduates of 4 public universities in Ghana. Secondly, no more than 2 learning centres, out of the many, were chosen from each of the universities for data collection. Finally, only students’ views were captured in this study without considering that of the administrators and students’ instructors.

8. Recommendations for Policy and Practice The demand for distance education was increasing due to the large number of students graduating from the senior high school yearly without proportionate expansion of high educational facilities. Information technology had the

http://ijlter.org/index.php/ijlter

325

capability to absorb this huge future demand to enable countries adequately prepare its future workforce. A robust national IT infrastructure designed for distance learning was capable for effectively maximizing the human inefficiencies associated with the institutional delivery of distance education. This will enable the universities to focus on their functions to deliver quality education without being saddled with the underlying technology. If IT infrastructure was seamlessly integrated into distance education, current and potential DE students will improve their self-efficacy in online learning tools to enjoy the many benefits of DE. Travelling long distances to learning centres will be reduced and students will be encouraged to upgrade their computer skills. The transformation of academic provision of DE institutions would enable the National Tertiary Council (NTC) to streamline, monitor and accurately certify the new DE programs and prospective institutions for quality national education. DE can also be effectively modified to comprehensively include learners with disability issues.

9. Recommendations for Future Research Future studies could investigate a mediation relationship between DE students’ internet data cost, frequent online resources use, perception and peer studies to establish its existence or otherwise. The relationship between online hours and internet data cost by distance education students can also be explored. Future research could also cover a comparative study of frequency of online resources usage among distance education students. Furthermore, the perception of the effectiveness of online peer studies among distance education students and their online resources usage categories could also be investigated.

10. References Acheampong, E. K. (2016). Students’ use of electronic resources in University of Professional Studies, Accra [Master’s thesis, University of Ghana].

African Virtual University. (2012). Open distance and e-learning (ODeL) centres, Nairobi. http://www. avu.org/AVU-Multi-national-support-project/ avu-capacityenhancement-program-acep.html.

Agbofa, F. J. (2012). Challenges and prospects facing students of distance education in Ghanaian public universities: The case of University of Cape Coast study centre. [MA dissertation, University of Cape Coast.] https://erl.ucc.edu.gh/jspui/bitstream/123456789/2686/1/AGBOFA%202012. pdf Allen, I. E., & Seaman, J. (2015). Grade level: Tracking online education in the United States. https://files.eric.ed.gov/fulltext/ ED572778.pdf Allen, I. E., & Seaman, J. (2017). Digital learning compass: Distance education enrolment report 2017. https://www.onlinelearningsurvey.com/reports/digtiallearningcompassenroll ment2017.pdf Anderson, T. (2011). The theory and practice of online learning (2nd ed.). Edmonton, AB: AU Press.https://www.aupress.ca/app/uploads/120146_99Z_Anderson_2008Theory _and_ Practice_of_Online_Learning.pdf Ankomah-Asare, E. T., Larkai, A.T., & Nsowah-Nuamah, N. N. N. (2016). Distance education in Ghana: An overview of enrolment and programme patterns.

http://ijlter.org/index.php/ijlter

326

Annku, E. (2014). Technology-enhanced teaching and learning: A case of the faculty of art In Kwame Nkrumah university of science and technology [Master’s thesis, Kwame Nkrumah University of Science and Technology].

Association for the Development of Education in Africa, ADEA. (2002). Distance learning in sub-Saharan Africa: A literature survey on policy and practice. The Commonwealth of Learning. http://oasis.col.org/bitstream/handle/11599/183/02DEinSSA_LiteratureSurve y.pdf?sequence=1&isAllowed=y Association for Educational Communications and Technology (AECT). (2001). What is the Knowledge Base? Association for Educational Communications and Technology. http://www.aect.org/standards/knowledgebase.html Asabere, N. Y., & Enguah, S. E. (2012). Development of an information and communication technology business model for electronic learning. International Journal of Engineering Science & Advanced Technology, 2(5), 1178–1191.

Asunka, S. (2008). Online learning in higher education in Sub-Saharan Africa: Ghanaian university students' experiences and perceptions. International Review of Research in Open and Distance Learning, 9(3).

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman. http://doi.org/10.5860/choice.35-1826 Betchoo, N. K. (2015). Sub-Saharan Africa’s perspective of distance learning. International Letters of Social and Humanistic Sciences, 48, 185-191. http://doi.org/10.18052/www.scipress.com/ILSHS.48.185 Caruth, D. L., & Caruth, G. D. (2013). Distance education in the United States: From correspondence courses to the internet. Turkish Online Journal of Distance Education, 14 (2), 141-149. https://files.eric.ed.gov/fulltext/EJ1013772.pdf Chandwani, A., Lihitkar, S., & Anilkumar, S. (2010). E-learning initiatives in India [Conference Paper]. Conference on Modern Practices in Library and Information Services, Nagpur, India. http://hdl.handle.net/10760/15721 Clinefelter, D. L., & Aslanian, C. B. (2015). Online college students 2015: Comprehensive data on demands and preferences. Louisville, KY: The Learning House, Inc. http://www.learninghouse.com/wpcontent/uploads/2015/07/OnlineCollegeS tudents 2015.pdf Creswell, J. W. (2009). Research design: Qualitative, quantitative and mixed approaches (3rd ed.). Los Angeles: SAGE publications.

Cunha, M. M., & Goran, D. P. (2007). A changed economy with unchanged universities?

A contribution to the university of the future. International Journal of Distance Education Technologies, 5(4), 5-25. http://doi.org/10.4018/jdet.2007100102 Curtis, B. & Curtis, C. (2011). Social Research: A Practical Introduction. London: Sage.

Dalgarno, B., Bishop, A., Adlong, W., & Bedgood, D. (2009). Effectiveness of a virtual laboratory as a preparatory resource for distance education chemistry students.

Computers and Education, 53(3), 853-865. https://doi.org/101016/jcompedu.2009.05.005 Darko-Adjei, N. (2018). Students’ perceptions and use of the Sakai learning management system in the university of Ghana. http://ugspace.ug.edu.gh Darkwa, O., & Mazibuko, F. (2000). Creating virtual learning communities in Africa: Challenges and prospects.

First Monday, 5(5). http://www.firstmonday.dk/issues/issue5_5/darkwa/index.html Digest of Education Statistics. (2014). National postsecondary student aid study. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2016006

http://ijlter.org/index.php/ijlter

327

Dube, S. (2017). The 21st century students’ educational ICT preferences. International Robotics & Automation Journal, 3(5). http://medcraveonline.com/IRATJ/IRATJ-03 -00069.pdf Elameer, A. S., & Idrus, R. M. (2011). National e-learning strategy to enhance and enrich the Iraqi universities. https://files.eric.ed.gov/fulltext/ED526803.pdf Garrison, D. R. (2000). Theoretical challenges for distance education in the 21st century: A shift from structural to transactional issues. International Review of Research in Open and Distance Learning.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education, 2(2-3), 87-105. https://doi.org/10.1016/S1096-7516(00)00016-6 Gerson, B. (2002). University helping to improve education for thousands in India. http://www.cmu.edu/cmnews/020411/020411_ helpindia.html Gregory, S., & Gregory, B. (2011, November 28-30). Do virtual worlds have a role in increasing student engagement as measured by their higher academic grades? CreateWorld, Brisbane, Australia.

Hassan, M. M., & Mirza, T. (2020). Information and communication technology (ICT) in the distance education system: An overview. IOSR Journal of Research & Method in Education (IOSR-JRME), 10(6), 38-42. http://doi.org/10.9790/7388-1006053842 Hope, A., & Guiton, P. (2005). Strategies for sustainable open and distance learning. New York: Routledge/Falmer Press.

Joo, Y., Bong, M., & Choi, H. (2000). Self-efficacy for self-regulated learning, academic selfefficacy, and internet self-efficacy in Web-based instruction. Educational Technology Research and Development, 48(2), 5-17. http://doi.org/10.1007/BF02313398 Kinyanjui, P. E. (2000). Future trends in open and distance education, in Huria. Journal of the Open University of Tanzania, (3),1-10.

Kulshrestha, T., & Ramswaroop, S. (2013). Benefits of learning management system (LMS) in Indian education. International Journal of Computer Science & Engineering Technology, 4(8), 1153-1164. http://www.ijcset.com/docs/IJCSET13-04-08036.pdf Kumi-Yeboah, A., Young, W., & Boadu, K. (2013). 21st century distance learning in subSaharan Africa: Distance and blended learning in Ghana. https://www.researchgate.net/publication/290572198 Kuo, Y. C. (2010). Interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in distance education courses. htps://digitalcommons.usu.edu/etd/741 Leary, J., & Berge, Z. (2007). Successful distance education programs in sub-Saharan Africa.

Turkish Online Journal of Distance Education, 8(12). http://tojde.anadolu.edu.tr/tojde26/pdf/article_12.pdf Lee, M. K. O., Cheung, C. M. K., & Chen, Z. (2005). Acceptance of internet-based learning medium: The role of extrinsic and intrinsic motivation. Information and Management, 42(8), 1095-1104. http://doi.org/10.1016/j.im.2003.10.007 Lee, R. (2004). E-Learning prospects for the developing world. In K. Stanoevska-Slabeva (Ed.), The digital economy. Anspruch and Wirklichkeit, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17032-4_25 Mensah, S. K. E., & Owusu-Mensah, F. (2002). Priorities and strategies for capacity building in tertiary distance education for human resources development in Ghana. http://siteresources.worldbank.org/EDUCATION/Resources/2782001099079877269/547664-1099079956815/Ghana_capacity_building_En02.pdf

http://ijlter.org/index.php/ijlter

328

Mnyanyi, C. B., & Mbwette, T. S. A. (2009). Open and distance learning in professional development in third world countries [Conference presentation]. Maastricht‘s 23rd International Conference on Distance Education.

Moore, M., & Kearsley, G. (2012). Distance education: A systems view. New York: Wadsworth Publishing Company.

National Centre for Education Statistics. (2003). Distance education at degree-granting postsecondary institutions: 2000-2001. http://nces.ed.gov/pubs 2003/2003017.pdf Nyagorme, P., Qua-Enoo, A. A., Bervell, B., & Arkorful, V. (2017). The awareness and use of electronic learning platforms: A case of a developing country. World Journal of Computer Application and Technology, 5(2), 13-23. http://doi.org/10.13189/wjcat.2017.050201 Nyerere, J. K. A., Gravenir, F. Q., & Mse, G. S. (2012). Delivery of open, distance, and elearning in Kenya. The International Review of Research in Open and Distance Learning, 13(3), 185-205. http://doi.org/10.19173/irrodl.v13i3.1120 Ofosu-Asare, Y. (2017). Impact of aesthetics and design on e-learning usability and credibility: The case of KNUST virtual classroom graphic user interface [Master’s dissertation, Kwame Nkrumah University of Science and Technology].

Oheneba-Sakyi, Y., & Amponsah, G. K. (2018). Still offline and behind in this digital world?

Join in the Sakai LMS experience. Paper presented at the meeting of an Inter-college lecture series, University of Ghana. https://www.ug.edu.gh/pad/sites/pad/files/u6/paddocs/2017 2018%20INTER%20COLLEGE%20LECTURE%20SERIES.pdf Osguthrope, R. T., & Graham, C. R. (2003). BL systems: Definitions and directions.

Quarterly Review of Distance Education, 4(3), 227-234. https://eric.ed.gov/?id=EJ678078 Özen, R. (2012). Distance education for professional development in ICT integration: A study with primary school teachers in turkey. International Journal of Business and Social Science, 3(3), 185–195. http://www.ijbssnet.com/journals/Vol_3_No_3_February_2012/19.pdf Pant, A. (2014). Distance Learning: History, Problems and Solutions. Advances in Computer Science and Information Technology (ACSIT), 1(2), 65-70.

Picciano, A. G. (2017). Theories and frameworks for online education: Seeking an integrated model.

Online Learning, 21(3), 166-190. http://doi.org/10.24059/olj.v21i3.1225 Radford, A. W. (2012). Learning at a Distance Undergraduate Enrollment in Distance Education Courses and Degree Programs. National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

Raja, R., & Nagasubramani, P. C. (2018). Impact of modern technology in education.

Journal of Applied and Advanced Research, 3(1), 33. http://doi:10.21839/jaar2018.v3iS1.165 Robinson, B., & Latchem, C. (2002). Teacher education through open and distance learning.

London: Routledge.

Saint, W. (1999). Tertiary distance education and technology in Sub-Saharan Africa.

Washington, DC: The World Bank.

Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. http://trendsandissues.org/node/413 Singh, K. D. (2009).

Computers in education. http://wikieducator.org/index.php?title=Computers_in_Education&oldid=292 994 Schaffhauser, D. (2019).

A personalized learning backlash. http://thejournal.com/article/2019/personalized-learning-backlash.aspx

http://ijlter.org/index.php/ijlter

329

Strasburger, V. C., Jordan, A. B., & Donnerstein, E. (2010). Health effects of media on children and adolescents.

Pediatrics, 125(4), 756-767. https://doi.org/10.1542/peds.2009-2563.

Sun, K. T., Lin, Y. C., & Yu, C. J. (2008). A study on learning effect among different learning styles in a Web-based lab of science for elementary school students. Computers & Education, 50(4), 1411-1422. https://doi.org/10.1016/j.compedu.2007.01.003 Tagoe, M. (2012). Students’ perceptions on incorporating e-learning into teaching and learning at the University of Ghana. International Journal of Education and Development using Information and Communication Technology, 8(1), 91-103.

Tanye, H. A. (2017). Quality e-learning in distance learning: Benefits and implications for national e-learning policy in Ghana. International Journal of Multicultural and Multireligious Understanding, 4(3), 1-11. http://dx.doi.org/10.18415/ijmmu.v4i3.73 Taylor-Straut, T. (2017). Distance education enrolment growth—Major differences persist among sectors.

The Association for Educational Communications and Technology, AECT. (2001). History of distance education. http://members.aect.org/edtech/ed1/13/13-02.html Usluel, K. Y., Mumcu, K. F., & Demiraslan, Y. (2007). ICT in the learning-teaching process: Teachers’ views on the integration and obstacles. H.U. Journal of Education, 32, 164179.

Wang, Y., & Sun, C. (2001). Internet-based real time language education: Towards a fourth generation distance education.

CALICO Journal, 18(3), 539-561. https://doi.org/10.1558/cj.v18i3.539-561 Wladis, C., Conway, K. M., & Hachey, A. C. (2015). The online STEM classroom—Who succeeds? An exploration of the impact of ethnicity, gender, and non-traditional student characteristics in the community college context. Community College Review, 43(2), 142-164. https://doi.org/10.1177/009155211557172 Zaina, L. A. M., Bressan, G., Silveira, R. M., Stiubiener. I., & Ruggiero, W. V. (2001).

Analysis and comparison of distance education environments [Conference presentation]. International Conference on Engineering Education, Oslo, Norway.

Zhang, J., Li, F., Duan, C., & Wu, G. (2001). Research on self-efficacy of distance learning and its influence to learners’ attainments. In C. H. Lee (Ed.). Proceedings of the International Conference on Computers in Education (ICCE) (pp. 1510-1517).

http://ijlter.org/index.php/ijlter

International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 330-348, February 2021 https://doi.org/10.26803/ijlter.20.2.18

ESL Teachers’ Challenges in Implementing E-learning during COVID-19 Brenda Anak Lukas and Melor Md Yunus Universiti Kebangsaan Malaysia, Bangi Selangor, Malaysia https://orcid.org/0000-0003-2892-8217 https://orcid.org/0000-0001-7504-7143 Abstract. Education sector in Malaysia had put emphasis on the use of online learning or e-learning with technology and devices as a mediator of communication to replace face-to-face learning during the COVID-19 pandemic. Subsequently, with the improvement of learning technologies, English as a Second Language (ESL) teachers faced various challenges in language teaching. In this regard, this study aimed to investigate the challenges ESL teachers faced in implementing elearning during COVID-19. Using a qualitative approach, this research was a case study which involved 20 primary ESL teachers. Data collection was done through an in-depth interview to explore teachers’ e-learning experiences and challenges in teaching the English language.

Based on the findings, the majority of teachers expressed that the use of e-learning was effective with various limitations such as teachers’ readiness to adopt e-learning, accessibility to mobile phones and Internet connectivity, classroom management in term of low students’ participation and assessment. Thus, the researcher recommended more studies to highlight teachers’ insight regarding the significance of elearning. ESL teachers, stakeholders, policy makers, and institutions can benefit from the results of the study and come out with practical strategies to utilize online sources for education in the pandemic situation.

Keywords: E-learning; English Language; ESL Teachers; COVID-19; Challenges

1. Introduction Across the globe, the spread of the COVID-19 pandemic had led to profound changes, which affected social interaction and organization in general, and the education sector in particular. The Malaysian government had enforced Movement Control Order (MCO) and introduced pandemic precautions for all which was called "social distancing" (Kamarudin, 2020). These actions were taken to reduce close contact and minimise any community transmission that could rapidly develop in crowded places like the university, as mentioned by

©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).

331

Weeden and Cornwell (2020). Therefore, the government together with the Ministry of Education (ME), came up with several measures to close all schools and turn to e-learning initiative, marking the fast transition of the standard face to face classes to virtual learning (Kamarudin, 2020). Online learning or elearning had been popular and flexibly used by teachers in teaching and learning due the fast escalation of the Internet and accessible technological tools (Huang et al., 2020). Other studies concerning e-learning during Covid-19 also had been carried out by several reseachers; Verawardina et al. (2020), Basilaia and Kvavadze (2020), Almaiah et al. (2020), and Dai and Xia (2020). Malaysian researchers also conducted studies regarding e-learning mainly in tertiary levels of education such as universities (Yusuf & Ahmad, 2020; Ismail et al., 2020; Nassr et al., 2020). However, there was limited research focused on primary schools and primary teachers’ practices in e-learning, though they are also shifted to e-learning during this pandemic.

According to Mohammad (2018), technology had an essential role in improving various learning tasks. It would help teachers improve their pedagogical skill, and learners who participated in e-learning were more likely to perform better than students who were learning traditionally, especially in teaching and learning English. Mutambik (2018) stated that the popularity of technology adoption in the domain of education had increased, especially since the emergence of the World Wide Web (WWW). He added, some e-learning applications permitted students learning English to readily access beneficial language resources and communicate directly with native English speakers. It is important to note that English is well known and spoken by many worldwide and known as lingua franca (Yen & Mohamad, 2020), but in some countries, English is the first or second language of its citizen (Breene, 2016). According to Nishanthi (2018), about 1 billion people worldwide speak English, 67 countries have English as their official language, and there are 27 countries have English as their secondary official language. As supported by the English Proficiency Index (EPI), English was taught and learned around the world and had a high proficiency as a second language (Breene, 2016). Accordingly, in 2020, Netherland scored the highest (652), followed by Denmark (632) and Finland (631), while Malaysia made its spot at top 30.

Although many teachers believed how new technology can effectively benefit the teaching and learning process, they did not fully embed it in their practices.

This is the initial stage how teachers should plan and prepare for their teaching methodology in the classroom. In terms of language teaching, teachers need to create a supportive environment where learners will have comfort if the language is taught with the communicative language teaching approach (AlSobhi & Preece, 2018). During this pandemic period, students who have been struggling learning English in face to face meeting will face a more challenging situation when they learned it virtually. Thus, this situation caused delay in learning English. During the COVID-19 outbreak, all educational levels such as schools, colleges, universities have been shifted to e-learning. Unfortunately, institutions that had less or no experience with e-learning were subject to failure when implementing it. Add to this, teachers are having difficulties to handle the

http://ijlter.org/index.php/ijlter

332

online applications (Zaharah et al., 2020). These challenges range from ICT skills, lack of motivation and training, and poor access to the Internet, resources and materials, to name just a few. Research regarding education during the covid-19 pandemic, particularly in English language teaching was lacking and had not been conducted thoroughly. Accordingly, more research is needed to identify individuals' challenges about integrating technology into the teaching and learning process especially in English language teaching. Therefore, this study investigated the challenges faced by primary ESL teachers while implementing e-learning during COVID-19. This study also sought to discover ESL teachers’ experiences of e-learning effectiveness during COVID-19.

2. Literature Review 2.1. E-learning According to Verawardina et al. (2020), online learning or e-learning used Internet technology that allowed teachers and students to conduct teaching and learning in or out of the classroom. Teaching and learning materials presented using this medium are visual, word, animation, video or audio-visuals so that learning can occur flexibly anytime and anywhere. Aparicio et al. (2016) mentioned that the e-learning was a concept of the use of computerised systems to aid and help teachers and students with the learning process. They determined other terms that were in the same concept such as online learning, virtual learning, distance education, and other learning management systems. In the same vein, Rodrigues et al. (2019) defined e-learning as an innovative webbased system invented based the use of digital technologies and various choices of learning resources and materials which aimed to give a conducive, learnercentered, and interactive environment to students.

This online learning skill was one of the 21st century skills that every person needed to master to succeed in this challenging age. In fully-online education, learners and teachers only meet via virtual environments using the Internet and its technologies. Hybrid teaching or the so-called blended learning took place when there is a face-to-face interaction between educators and learners in online learning. The web-assisted method of e-learning used synchronous welldesigned tools that are meant to enhance teaching and make learning effective.

Web-assisted learning included online discussions and interactions between the learners and instructors, and varied information can be analysed, to improve teaching quality (Gonzalez et al., 2020). Various online applications can be accessed using mobile phones or laptops in this modern-day such as Zoom, Google Classroom, Microsoft Teams, WhatsApp, Telegram, and Edmodo.

According to Amry (2014), WhatsApp is considered as cost-effective, easy to administer as it has low uploading and downloading rate. Thus, using WhatsApp can lighten teachers’ burden and enable students to access in a short time. It has been found out that WhatsApp was unique in promoting good student-teacher relationships and a positive classroom environment (Hershkovitz et al., 2019).

http://ijlter.org/index.php/ijlter

333

2.2. Benefit of E-learning Bailey and Lee (2020) stated some benefits of e-learning as compared to the traditional modes of classroom learning, namely: (a) accommodated everyone's needs, (b) learners can take lectures any time, (c) low cost, (d) offered access to updated content, and (e) little environmental impact. Recently, there are many alternatives which can suit the students' learning styles in language learning. By instilling the interactivity in language learning, students will feel more attracted to learn the language as their affective filter is lowered. E-learning improved teachers’ pedagogical skills, offered varieties of instruction methods, relevant curriculum design, and language learning tools (Mynbayeva et al., 2017).

Merç (2015) claimed that teachers’ beliefs about the usefulness of technology in the second language classroom influenced their pedagogical beliefs, strategies, procedures, and choice of materials to be utilised in their teaching. This aspect provided us to better understand their actual practices. In this respect, Shifflet and Weilbacher (2015) stated that teachers' beliefs about the technology for teaching and learning might be the strongest factor in the classroom. Besides, Halim and Hashim (2019) mentioned that teachers' way of engaging the lesson can be improved, they would build self-confidence, and had an effective interaction in the classroom. Students’ motivation can also be improved if they were involved in e-learning activities that can attract and sustain their attention (Morat et al., 2017). As supported by Pazilah et al. (2019) and Morat et al. (2017), the use of authentic learning materials such as videos from YouTube can make the online teaching and learning experience meaningful and purposeful.

2.3. E-learning in ESL Classroom Various studies have been conducted to evaluate how e-learning impacted its users (Mahlangu, 2018; Kintu et al., 2017). As for English language learning and teaching, the widespread use of technology and the e-learning environment has impacted the pace of learners’ second language learning. The rapid developments and innovations in technology provided opportunities and new features in language learning. For example, a combination of audio, visual and animation effects made the e-learning more appealing and effective (Wijaya & Helmi, 2018). Moreover, e-learning is also considered a cost-efficient and timesaving tool in learning a language. It also provided massive forms of sources to practice the language such as online educational applications.

Learning a second language is said to be challenging especially in semi-urban and rural areas (Endriyati, 2019). Fortunately, in this modern technology, it served second language learners in many ways. According to Foti (2020), elearning became an easy and affordable source for learners to access and interact with others by sharing knowledge and information. It is claimed by many researchers that advanced technological methods in teaching can spark interest and boost the motivation of learners to learn the target language (Nadeak, 2020).

Despite its advantages in teaching and learning, many teachers did not fully maximise the use of this technology in their classes due to the lack of facilities, time, and poor ICT knowledge (Ghavifekr et al., 2016).

http://ijlter.org/index.php/ijlter

334

In modern times with easy access to smartphones and the Internet, students are likely to become more open to using technology and online resources as tools to learn a language. Nowadays, many students have offline or online dictionaries downloaded in their smartphones which can help them find the meanings of the new words. Apart from that, ESL teachers also find those applications quite useful when designing particular tasks for their students. As asserted by Mohamad et al. (2017), electronic dictionaries were handy and time-efficient for learners to search for the meaning of new words that they wanted to know. This advanced application has promoted students' self-directed esteem to become independent learners and take responsibility in their learning as they controlled their own learning pace (Mohamad et al., 2017). Other online educational platforms such as blogs, social media applications, and smartphones can alleviate and enhance both teachers’ and students' language teaching and learning.

Learning English as a Second Language (ESL) had always never been easy especially for the remedial students who always needed teachers to monitor and guide them during lessons. Some studies mentioned the difficulty to teaching language skills, namely listening, speaking, reading, and writing during this COVID-19. For example, teachers and students found it difficult to teach and learn writing during the pandemic period (Yunus et al., 2019). Sayuti et al. (2020) stated that the lack of confidence in speaking during the virtual class made students uncomfortable compared to face-to-face meeting in the classroom (Pazilah et al., 2019). Face-to-face teaching is different than online learning, where teachers can address and give attention to almost all students' doubts and queries (Dai & Xia, 2020). Furthermore, students were all different in terms of learning styles, interest, and proficiency levels especially in learning English as a second language is bound to be moderated by a wide array of linguistic, contextual and individual difference (Pawlak, 2017).

Moreover, in Malaysia, ESL pupils also found it hard to learn during the pandemic period (Yen & Mohamad, 2020). In the online classroom, peer interaction during speaking class rarely happened making which resulted in making students feel shy to practise compared to the traditional classroom (Pazilah et al., 2019). Furthermore, in a study that was conducted in Hong Kong

by Moorhouse and Beaumont (2020), they mentioned that learners with mixedabilities were having problems in catching up with the rest of the class as teachers were giving standardized-tasks to all of them. Although there was slightly improvement in language learning using Learning Language Strategies (LLS), their effectiveness was still not fully applied (Lestari & Wahyudin, 2020).

2.4. Challenges of E-learning Even though e-learning was rapidly being used worldwide, it persisted as a challenging problem for various levels of education. As asserted by Eltahir (2019), universities’ readiness was one of the challenges in adopting e-learning that led to failure in the system. He also mentioned several problems of elearning adoption; technology barriers and accessibility, self-competencies, norm/cultural challenges, and course challenges. However, these challenges

http://ijlter.org/index.php/ijlter

335

were very different for each country regarding their geographical location, norm, and users' awareness. Ghavifekr et al. (2016) also stated that poor network connection and the lack in the main development of infrastructure were the significant challenges of e-learning system adoption in developing countries.

Another study also presented system features, Internet connection, and computer skill that hindered the successfulness of the e-learning system. Despite these efforts, none of these studies had investigated the actual challenges users face during the e-learning system. However and according to Al-Khasawneh and Obeidallah (2019), the e-learning challenges still exist, and that cause learners to become reluctant to use it in Jordan.

Similarly, many researchers had stated few challenges in English language teaching. During the COVID-19, the rush to use e-learning had caused teachers and students to adopt the shift. Shahzad et al. (2020) mentioned that e-learning environment had impacted teachers’ instructional methods and caused failure in teaching and learning. According to Ko and Rossen (2017), the successfulness of teaching activities is affected by teachers’ ICT skills and experiences, which was considered as one of the challenges they face. Furthermore, Aliyyah et al. (2020) mentioned other challenges of e-learning related to teachers’ lack of readiness to tackle students to fully participate and complete their tasks. Senior teachers found it difficult to adapt but those who master the applications and technology tools approached it significant which can be implemented effectively and smoothly especially during the COVID-19 (Reimers et al., 2020). Furthermore, the learners showed less effort and responses during teaching and learning session in switching off their video and audio throughout the whole online lesson, thus effective engagement and interaction between students and teachers could not be highly produced (Mohmmed et al., 2020). Alternatively, teachers will need to find other alternatives to deliver the lessons using other available platforms which in turn caused students to be delayed in learning (Kaden, 2020).

It is indeed that e-learning had plenty of benefits to education, however, its challenges hindered the smoothness and success for learning. Students’ engagements can be distracted by other online content, and thus can cause the lesson to become less meaningful (Yunus et al., 2019). Besides, technical issues were there to arise, such as audio and video problems (Halim & Hashim, 2019).

Also, in terms of language learning, authentic communication was hardly existing and less opportunities during e-learning lesson (Pazilah et al., 2019).

Furthermore, “one size fits all” approach is not suitable to use in e-learning situation as it could restrict students with individual differences to participate in the lesson (Gillett, 2017). Add to this, instructors who are not competent about online delivery could make course design and planning useless and problematic.

Teachers should really plan and prepare their lessons beforehand so that objectives of lessons can be achieved and aligned with the curriculum framework. Despite the many challenges related e-learning with the rapid widespread of COVID-19, teachers and other parties will have to create and design innovative strategies that best suit educational needs so that no student would be left behind.

http://ijlter.org/index.php/ijlter

336

3. Methodology 3.1. Research Design Using a qualitative approach, this research was a case study which involved 20 primary ESL teachers. It aimed to investigate their experiences and challenges of e-learning implementation during COVID-19 pandemic. Creswell (2009) mentioned that a case study is used to get a rich data from groups of different people in a particular setting or scenario. The aim of using case study was to identify a case in a thorough and comprehensive manner.

3.2. Research Participants Table 1 presented a summary of the participants' demographic profile for this study (n=20). The majority of the participants were females, n = 13 (65%). The participants clustered towards the youngest group, at 23 to 30 years old (n = 11, 55%). The participants' school setting clustered towards rural (n = 12, 60%).

Participants were all degree holder in teaching English as Second Language (TESL) and they taught one to three English classes a day from primary one till primary six, respectively.

Table 1: Demographic Profile of Participants Demographic Profile Gender Male Female Age 23 - 30 31- 40 41 - 50 51 – 60 School Setting Urban Suburban Rural

N

Percent (%)

7 13

35 65

11 4 3 2

55 20 15 10

3 5 12

15 25 60

3.3. Research Instrument This study employed an interview to obtain and achieve the study objectives.

Questions in the interview generally involved open-ended questions, semistructured, and closed-ended questions. Besides, the interview served as a powerful and a useful data collection instrument to reach participants. In this research work, the interview was used because it was suitable and relevant to obtain data on teachers’ experiences in implementing e-learning and the challenges they faced. They answered the interview questions completely through online by means of WhatsApp application, wherein each interview lasted for 10-15 minutes through audio and video calls. The interview consisted of 11 questions adapted from Lie et al. (2020), and is considered as an important tool for teachers to include their views and opinions about online teaching and the use of technology during the hard circumstances like the present crisis.

http://ijlter.org/index.php/ijlter

337

3.4. Research Procedures and Analysis The data for the present study were collected from 28 November to 18 December 2020 through WhatsApp interviews with all the selected teachers who taught English in primary schools. After describing the objectives and purposes of the study, the teachers were then required to sign the consent form if they agreed to participate in the interview. The latter was conducted via audio and video calls with all the participants. Conducting an interview using WhatsApp was best method to collect data as the pandemic was still actively spreading around the world. Five-step data analysis model was used in the present research.

Figure 1: Five Steps Data Analysis Model

Based on Figure 1, the researcher transcribed and filtered the original data from the interview. This was intentionally done to omit unnecessary data and retrieve important details. The answers given by the interviewees were categorised into several patterns or sub-themes using the Nodes function. Next, the list of themes was then classified into few main themes. Lastly, the researchers proceeded with the report by writing the interview results.

4. Findings and Discussion

Figure 2: Thematic Map

http://ijlter.org/index.php/ijlter

338

From Figure 2, 11 nodes (themes) were created during the interview analysis; Internet connection, family background, e-learning acceptance among teachers, availability of mobile phones in students’ families, students’ attitudes and behaviour, methods of teaching delivery, types of multimedia that suit students’ learning styles, working parents, teachers’ ICT skills, different proficiency levels among students, and assessment and feedback. The nodes were then classified into 4 nodes classifications (themes) depending on the answer to the research questions. The themes were classified according to the number below: 1. Teachers’ experiences of e-learning effectiveness during COVID-19 2. Teaching methods for effective lesson 3. Challenges of e-learning implementation faced by teachers 4. Students’ participation and assessment 4.1. Teachers’ Experience of E-learning Effectiveness during COVID-19 Based on teachers’ e-learning experiences, they looked at online ESL teaching from different degrees of agreement. The participating teachers had almost similar experience of e-learning over COVID-19 when all of them were mainly using WhatsApp application for their online teaching. Interestingly, WhatsApp (75%) became particularly practical and convenient for their online teaching during the closure of schools as they were able to reach their students. Other online applications that they utilised were: Google Classroom (15%), Zoom (10%), and Quizzes (25%). Figure 2 below represented teachers’ modes of online applications.

Figure 2: Modes of Online Applications

The following were some excerpts manifested by some interviewees: “I prefer to use WhatsApp for online teaching and learning. It is easy to use and nowadays all parents who own smartphones has WhatsApp application.” (Teacher 2)

http://ijlter.org/index.php/ijlter

339

“Using WhatsApp is easy to upload and download materials as it does not really require excellent Internet connection.” (Teacher 10) According to Amry (2014), WhatsApp was cost-effective, easy to administer as it had low uploading and downloading rate. Thus, using WhatsApp can lighten teachers’ burden and enable students to access in a short time. Hershkovitz et al.

(2019) also supported this finding in their study mentioning that WhatsApp was an application that facilitates and promotes good students-teacher relationship and also creates an optimistic learning condition.

In terms of lesson plan preparation and implementation, 15 teachers prepared their lesson plans according to the syllabus and Scheme of Work (SoW) and implemented them accordingly during their online teaching. Another five teachers otherwise expressed their uncertainty of their lesson plans implementation as e-learning sometimes faced inconsistency in delivering the intended knowledge. The factor to this problem was related to teachers’ unexpected situation during online teaching such as poor internet connection.

This finding also existed in other studies which presented system features, internet connection, and computer skill that hindered the successfulness of the elearning system (Almaiah et al., 2020). Poor internet connection can be the main factor behind the failure of e-learning which would result in frustration of both teachers and students.

4.2. Teaching Methods for Effective Lesson Despite having time to adapt to online teaching and learning, and at the same time adhering to government initiatives to shift from the traditional classroom teaching to e-learning, teachers still have to ensure that learning can take place.

There were various methods to utilise online teaching such as lectures, gamebased, task-based, problem-based learning projects, and discussion. In this study, 13 teachers gave worksheets to their students through WhatsApp, and the others mentioned project-based, inquiry-based, online quizzes, and educational videos.

“I find it easier to send worksheet or task through WhatsApp, parents easily can understand my instruction and let students complete the worksheet given. Plus, I can easily send any related materials(notes) I find in the Internet for the English lesson.” (Teacher 14) “Giving worksheets is the best online teaching I can do for my students.

I list out the learning steps in a simple language and attach the worksheet. Giving them time and ask them to send the pictures of their work through WhatsApp.” (Teacher 15) “I implement my online teaching sometimes using online quizzes. The quizzes are related to their learning, and through its students become more active and attracted to the leaning.” (Teacher 5) Some of them also mentioned that they used various methods to reach to their students including giving worksheet through WhatsApp, and sending pictures or videos as extra support materials. For example, a combination of audio, visual

http://ijlter.org/index.php/ijlter

340

and animation effects made e-learning more appealing and effective (Wijaya & Helmi, 2018). Their choices of methods were strongly related to students’ home area which had poor internet connection. This finding was in line with several studies that presented technology barriers and accessibility which were approached to be the main obstacles in adopting to online teaching (Eltahir, 2019; Ghavifekr et al., 2016).

Notwithstanding teachers had to utilise various methods to teaching online during Covid-19, some others liked the ideas of it as they can access to plenty of English materials to share with the students on the spot compared to the traditional classroom teaching. Based on this assumption and according to Foti (2020), online teaching and learning became an easy and an affordable source for the teachers and learners to access and interact with others by sharing knowledge and information in the modern era. It provided plenty forms of sources to practise the language significantly through online educational applications.

“As time passes by, I find online teaching is effective as it has various relevant materials and sources for teaching and learning English.

Teacher can adopt and adapt teaching materials based on students’ proficiency.” (Teacher 8) “One thing that excites my students is the use of videos or animations even in the face-to-face meeting, so in this period of online teaching I try my best to get them attracted to my lesson by using pictures and videos even though we cannot interact using the real time application like Microsoft teams or Google Classroom.” (Teacher 12) As supported by Pazilah et al. (2019), the use of authentic learning materials like videos from YouTube can make the online teaching and learning experience meaningful and purposeful. These optimistic views showed that teachers accept the use of online teaching as it benefited their teaching and their students. According to Merc (2015), teachers’ beliefs about the usefulness of technology in the second language classroom influenced their pedagogical strategies, procedures, and materials, and provided a better understanding about their actual practices. Below statements showed teachers’ view on how online teaching and learning was effective for their teaching practices.

“The advantages of online learning are making the learning to be possible in everywhere and every time as long as the facilities are well adequate as it is convenience and fun.” (Teacher 16) “Students are able to explore more knowledge with Internet as students are trained to look for information independently. Also, teachers get to explore lots of strategies in teaching.” (Teacher 5) 4.3. Challenges of E-learning Implementation Faced by Teachers In spite of their optimistic views about e-learning, teachers also mentioned few of its limitations such as readiness to adapt to e-learning, technological barriers in terms of internet connection and devices, and classroom management in term of students’ participation and assessment. The participating teachers stated their

http://ijlter.org/index.php/ijlter

341

anxiety and concern during the process of e-learning. Teachers' readiness in term of technological experience and competence was one of the challenges mentioned in the interview. Ko and Rossen (2017) also supported this by saying that teachers who were lacking ICT skills found it challenging to implement elearning, and it determined whether the lesson can be delivered successfully or not. They found it not convenient for them to conduct e-learning as they were less competent with technology. This problem mostly impacted senior teachers in which they regarded it difficult to run the advanced applications. When this happened, teachers feel burdened, and online teaching and learning was not optimal. However, it was different for young and those who master the applications and technological tools, in which e-learning can be implemented effectively and smoothly during COVID-19 (Reimers et al., 2020). In the interviews, few of them mentioned their readiness in adopting online teaching during the pandemic period.

“At first, I was so worry about what platform should I utilise and what if my students do not understand the lesson. I am still anxious about how my online teaching will be even until now, not so sure about its effectiveness.” (Teacher 18) “I have never taught online before, and I find it difficult, not confident to catch up with the technology. But I use WhatsApp, which is doable and easier than other applications.” (Teacher 4) “I feel that e-learning is not easy to implement, especially now, and even the parents face the same problems as mine.” (Teacher 6) “I find it difficult to prepare my teaching materials for the online teaching as I am not experienced and need longer time to adapt to this.”

(Teacher 19) “I am teaching three classes; I am still struggling to adapt to the new norm. I'm having difficulty in organising my classes' tasks, but I try to put more effort.” (Teacher 9) The transformation in the education to make use of e-learning had influenced teachers’ working progression and education quality. Teachers who participated in this study believed that e-learning needed to be effective and practical to implement, and all related parties should cooperate and have the same goals that align with the objectives of the government ensuring the notion that education must take place no matter what situation we are in. Regarding teaching strategies using e-learning, teachers emphasised that technological readiness and skills, whether online or offline applications was essential for the "new normal" approach, and it was necessary to support online learning success.

Dai and Xia (2020) mentioned that educators are recommended to attend regular professional development workshops to utilise e-learning and high access to the curriculum resources.

The second main challenge was technological barriers in term of accessibility to internet and devices like mobile phones and laptops. The low-income family

http://ijlter.org/index.php/ijlter

342

background where parents did not have access to the electronic devices and poor network and also those who lived in the suburbs and rural area are subject to be unsuccessful when in e-learning situation. It had been confirmed that learning a second language can be challenging especially in semi-urban and rural areas (Endriyati, 2019). Eltahir (2019) mentioned several problems of elearning adoption; technology barriers and accessibility, self-competencies, the norm cultural challenges, and course challenges. These challenges determined the successfulness of the instructional activities to be taken place. Also, student’s participation was low, and learning was not optimal. These obstacles caused some students to be left behind the lesson and led to procrastination and delay in submitting their tasks. These technological impediments often faced by students with poor access to internet who came from low economic level and live rural areas as previously highlighted. The following statements covered what had been claimed by the participating teachers: “Poor Internet access and not all of my students have mobile phones to follow e-learning.” (Teacher 1) “The Internet coverage in my students' area is not supportive of them to use e-learning as most of my students live in rural areas.” (Teacher 7) “More screen time which may cause eye strain and worsen my eye health condition.” (Teacher 20) “Pupils use their parents' device. Sometimes, parents can only guide or make time for online learning after working hours.” (Teacher 17) In the implementation of online learning during COVID-19, teachers and students must have enough internet quota and accessibility towards various electronic devices. Owning this privilege will help students to follow online learning smoothly. However, some teachers visited students at home, sending homework to make sure that all students received the same materials.

4.4. Students’ Participation and Assessment Some students were not familiar with online learning as they were so used to face-to-face interaction with their teachers. Moreover, students struggled a great deal with gadgets such as mobile phones as most of them did not own one.

Learners from the rural area which had poor or no internet connection and lowincome family could not access to online learning. Teachers will need to find other alternatives to deliver the lessons using other available platforms which again caused students to be delayed in learning (Kaden, 2020). Due to these factors, students were gradually losing interest and motivation in joining online classes. This is also felt with some interviewees as shown below: “Remedial students that need guidance are sometimes left behind as the good students conquer the lessons. I find it hard to monitor and guide struggling students and that somehow demotivate them.” (Teacher 2) “One of the disadvantages of online learning is I feel there is a wall between me and my students that I couldn’t really feel their presence, unlike in classroom. I might don’t know what is their problem because

http://ijlter.org/index.php/ijlter

343

in the screen I cannot see their body language and mimic for instance if they apply display picture on their screen. Some of the students also might lie to their teacher, by only joining in but they are doing something else.” (Teacher 3) “I cannot run from the fact that face-to-face interaction is still imperative to be able to keep track of students’ progress (tasks or performance) especially in the rural areas and it is hard to assess students as their performances are inconsistent.” (Teacher 10) “Parents were answering the quizzes instead of students which in turn teachers can’t assess students’ performance.” (Teacher 13) “Some students did not do their homework on their own, as some answers seem dishonest. Plus, it is difficult to provide feedback individually.” (Teacher 11) Teacher can plan and prepare virtual classroom that both interesting and effective at the same time. In order to appeal to students' enthusiasm and motivation in online learning, teachers had to be creative to attract students’ attention. Teachers were required to maximise the use of varied instructional methods to encourage more enthusiastic participation in online learning.

Involving students in a fun and engaging learning activities can also increase their motivation to learn (Morat et al., 2017). During learning from home, several teachers expressed that they were having hard time in getting students to be interested and maintain students’ attention and motivation for learning. At the beginning of online learning, students were excited about immerging themselves with the "new normal" of learning, but the spirit was fading after a few months, in which students became uninterested and less bothered to learn. Moorhouse and Beaumont (2020) stated that learners with mixed-abilities were having problems in catching up with the rest of the class as teachers were giving standardised-tasks to all of them. Thus, feedback and assessment were not consistent as pupils perform way better than at school. This condition was highly due to major helps came from parents as they were the ones who did and completed the task instead of their children.

All the aspects above have affected the achievement of instructional objectives and teachers’ performance in delivering knowledge to students online, however and according to Moorhouse and Beaumont (2020), challenges still existed. The most significant obstacle teachers encountered when conducting e-learning was students’ poor internet connection and little to no access to mobile phones or laptops. Influential parties such as parents, teachers, and the government should give unconditional support to ensure effective implementation of e-learning.

Collaboration from those parties will nevertheless help teachers to deliver the knowledge successfully and motivate students at the same time. The findings of this study indicated that educators and curriculum designers ought to prepare online learning frameworks that catered to few conditions such as teachers' perception of e-learning, accessibility to the technologies, students economic background, and needs in implementing online learning during the COVID-19

http://ijlter.org/index.php/ijlter

344

pandemic. Thus, for successful e-learning implementation, several factors must be considered. Importantly, teachers’ technology readiness must be polished and improved so that fun, meaningful and purposeful lesson can be created.

5. Conclusion In the findings on this study, the majority of teachers expressed that the use of online teaching was effective but with various limitations. There were limitations connected to teachers’ readiness to adopt online teaching, accessibility to mobile phones and internet connectivity, classroom management in terms of low students’ participation and assessment. The study found that it was important for instructors to be prepared and flexible in utilising e-learning to increase learning effectiveness. Hopefully, findings of this study will help teachers, students, parents, and stakeholders to have a more depth understanding of the online teaching efficacy and digital divide caused by the pandemic in Malaysia. Collaboration and innovation from essential parties such as government, teachers, parents, and schools were crucial to optimise the efficacy and practicality of e-learning during this critical period. Based on the current limitations of this research that covered a small number of ESL teachers, future research needed a thorough and comprehensive research on the issue with large population. Teachers’ personal experiences such as years of teaching experience, age, and education level may lead to differences in their view about e-learning in teaching English.

6. Implication COVID-19 pandemic has caused a big crisis in all educational systems worldwide. Many countries have come out with alternatives to make sure the learning is still taking place. Eventually, the ministry of education put great emphasis on e-learning implementation. Along with the shift, alterations were necessary to employ effective instructional pedagogy, teachers' technological readiness to conduct e-learning and giving constructive support to all needed.

This was to maximise e-learning potential, and no student would be left behind.

7. References Aliyyah, R. R., Rachmadtullah, R., Samsudin, A., Syaodih, E., Nurtanto, M., & Tambunan, A. R. S. (2020). The perceptions of primary school teachers of online learning during the COVID-19 pandemic period: A case study in Indonesia.

Journal of Ethnic and Cultural Studies, 7(2), 90-109. https://doi.org/10.29333/ejecs/388 Al-Khasawneh, A. M., & Obeidallah, R. (2019). E-learning in the Hashemite University: Success factors for implementation in Jordan. In Advanced Online Education and Training Technologies, 135-145. https://doi.org/10.4018/978-1-5225-7010-3.ch008 Almaiah, M.A., Al-Khasawneh, A., & Altunibat, A. (2020). Exploring the critical challenges and factors influencing the E-learning system usage during COVID19 pandemic.

Education and Information Technologies. https://doi.org/10.1007/s10639-020-10219-y Al-Sobhi, B. M. S., & Preece, A. S. (2018). Teaching English speaking skills to the Arab students in the Saudi school in Kuala Lumpur: Problems and solutions.

http://ijlter.org/index.php/ijlter

345

International Journal of Education and Literacy Studies, 6(1), 1-11. https://doi.org/10.7575/aiac.ijels.v.6n.1p.1 Amry, A. B. (2014). The impact of WhatsApp mobile social learning on the achievement and attitudes of female students compared with face to face learning in the classroom.

European Scientific Journal, 10(22), 116–136. https://doi.org/10.19044/ESJ.2014.V10N22P%25P Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-Learning theoretical framework.

Educational Technology & Society, 19(1), 292–307. https://www.researchgate.net/publication/290086485_An_eLearning_Theoretical_Framework Bailey, D. R., & Lee. A. R. (2020). Learning from experience in the midst of COVID-19: Benefits, challenges, and strategies in online teaching. Computer-Assisted Language Learning Electronic Journal, 21(2), 178-198. https://www.researchgate.net/publication/343219357_Learning_from_experie nce_in_the_midst_of_COVID19_Benefits_challenges_and_strategies_in_online_teaching Basilaia, G., & Kvavadze, D. (2020). Transition to online education in schools during a SARS-CoV-2 Corona virus (COVID-19) pandemic in Georgia. Pedagogical Research, 5, 1-9. https://doi.org/10.29333/pr/7937.

Breene, K. (2016). Which countries are best at English as second language?. World Economic Forum. https://www.weforum.org/agenda/2016/11/whichcountries-are-best-at-english-as-a-second-language-4d24c8c8-6cf6-4067-a7534c82b4bc865b/ Creswell, J. W. (2009). Research design: Qualitative, quantitative and mixed method approaches (3rd ed.). Los Angeles: SAGE Publications.

Dai, D., & Xia, X. (2020). Whether the school self-developed e-learning platform is more conducive to learning during the COVID-19 pandemic?. Best Evid Chin Edu., 5(1), 569-580. https://doi.org/10.15354/bece.20.ar030 Eltahir, M. E. (2019). E-learning in developing countries: Is it a panacea? A case study of Sudan. IEEE Access, 7, 97784–97792. doi:10.1109/ACCESS.2019.2930411 Endriyati, P. A. A. (2019). Challenges in teaching English at rural and urban schools and their solutions. International Journal of Scientific & Technology Research, 8(10), 37063710.

Foti, P. (2020). Research in distance learning in Greek kindergarten schools during the pandemic of covid-19: Possibilities, dilemmas, limitations. European Journal of Open Education and E-learning, 5(1), 20-39. https://doi.org/10.5281/zenodo.3839063 Gillett, S. J. (2017). The challenges of online learning supporting and engaging the isolated learner.

Journal of Learning Design, 10(1), 20-30. https://doi.org/10.5204/jld.v9i3.

Ghavifekr, S., Kunjappan, T., Ramasamy, L., & Anthony, A. (2016). Teaching and learning with ICT tools: Issues and challenges from teachers’ perceptions.

Malaysian Online Journal of Educational Technology, 4(2), 38-57. https://files.eric.ed.gov/fulltext/EJ1096028.pdf Gonzalez, T., Rubia, M. A., Hincz, K. P., Lopez, M. C., Subirats, L., Fort, S., & Sacha, G. M. (2020). Influence of COVID-19 confinement in students’ performance in higher education.

Universidad Autónoma de Madrid.

1-25. https://doi.org/10.1371/journal.pone.0239490 Halim, M., & Hashim, H. (2019). Integrating web 2.0 technology in ESL classroom: A review on the benefits and barriers. Journal of Counseling and Educational Technology, 2(1), 19-26. https://doi.org/10.32698/0381

http://ijlter.org/index.php/ijlter

346

Hershkovitz, A., Elhija, M., & Zedan, D. (2019). WhatsApp is the message: Out-of-class communication, student-teacher relationship, and classroom environment.

Journal of Information Technology Education: Research, 18, 073-095. https://doi.org/18. 073-095. 10.28945/4183 Huang, R., Tlili, A., Yang, J., Chang, T., Wang, H., Zhuang, R., & Liu, D. (2020). Handbook on facilitating flexible learning during educational disruption: The Chinese experience in maintaining undisrupted learning in COVID-19 outbreak. https://iite.unesco.org/wp-content/uploads/2020/03/Handbook-onFacilitating-Flexible-Learning-in-COVID-19-OutbreakSLIBNU_V2.0_20200324.pdf Ismail, N., Mazlina, N., Wafa, S., Syed, W., Wafa, S., & Thompson, J. (2020). Online learning challenges during pandemic COVID-19 in Malaysian higher learning institution.

Universal Journal of Educational Research,8, 7151-7159. https://doi.org/10.13189/ujer.2020.081282 Kaden, U. (2020). COVID-19 school closure-related changes to the professional life of a K–12 teacher.

Education Sciences, 10(6), 165. https://doi.org/10.3390/educsci10060165 Kamarudin, K. (2020). Not all students can benefit from online teaching. Bernama. https://www.bernama.com/en/features/news.php?id=1827297 Ko, S., & Rossen, S. (2017). Teaching online: A practical guide. Routledge.

Kintu, M.J., Zhu, C., & Kagambe, E. (2017). Blended learning effectiveness: The relationship between student characteristics, design features and outcomes. Int J Educ Technol High Educ, 14(7). https://doi.org/10.1186/s41239-017-0043-4 Lestari, M., & Wahyudin, A. Y. (2020). Language learning strategies of undergraduate EFL students. Journal of English Language Teaching and Learning, 1(1), 25-30. https://doi.org/10.33365/jeltl.v1i1.242 Lie, A., Tamah, S., Gozali, I., Triwidayati, K., Utami, T., & Jemadi, F. (2020). Secondary school language teachers’ online learning engagement during the Covid-19 pandemic in Indonesia. Journal of Information Technology Education Research, 19. https://doi.org/10.28945/4626 Mahlangu, V. P. (2018). The good, the bad, and the ugly of distance learning in higher education, trends in E-learning.

Intech Open.

17-29. https://doi.org/10.5772/intechopen.75702 Merç, A. (2015). Using technology in the classroom: A study with Turkish pre-service EFL teachers. The Turkish Online Journal of Educational Technology, 14(2), 229–240. https://www.researchgate.net/publication/283231199_Using_technology_in_th e_classroom_A_study_with_Turkish_pre-service_EFL_teachers Mohamad, M., Rashid, N., & Wan, M. W. M. (2017). The advantages and disadvantages of Edictionaries to enhance vocabulary learning of ESL learners. Paper presented at the Asian Conference on Education & International Development 2017 Official Conference Proceedings. http://papers.iafor.org/wpcontent/uploads/papers/aceid2017/ACEID2017_34731.pdf Mohamad, R. A. (2018). The use of technology in English language learning: A literature review. International Journal of Research in English Education, 3(2), 116-125. https://doi.org/10.29252/ijree.3.2.115 Mohmmed, A. O., Khidhir, B. A., Nazeer, A., & Vijayan, V. J. (2020). Emergency remote teaching during coronavirus pandemic: The current trend and future directive at middle east college Oman. Innov.

Infrastruct.

Solut, 5, 72. https://doi.org/10.1007/s41062-020-00326-7

http://ijlter.org/index.php/ijlter

347

Moorhouse, B. L., & Beaumont, A. M. (2020). Utilizing video conferencing software to teach young language learners in Hong Kong during the COVID‐19 class suspensions. TESOL Journal, 11(3). https://doi.org/10.1002/tesj.545 Morat, N., Shaari, A., Abidin, M. J. Z., & Abdullah, A. (2017). YouTube within ESL classroom: Exploring an instructor’s and her learners’ experiences concerning the authenticity of language and technology use. Malaysian Journal of Learning and Instruction: Special Issues 2017, 173-196. https://doi.org/10.32890/mjli.2017.7802 Mutambik, I. (2018). The role of e-learning in studying English as a foreign language in Saudi Arabia: Students’ and teachers’ perspectives. English Language Teaching, 11(5). https://doi.org/10.5539/elt.v11n5p74 Mynbayeva, A., Sadvakassova, Z., & Akshalova, B. (2017). Pedagogy of the twenty-first century: Innovative teaching methods. Contributions of Research in Education. https://doi.org/10.5772/intechopen.72341 Nadeak, B. (2020). The effectiveness of distance learning using social media during the pandemic period of COVID-19: A case in Universitas Kristen Indonesia.

International Journal of Advanced Science and Technology, 29(7), 1764-1772. https://www.researchgate.net/publication/341669112_The_Effectiveness_of_D istance_Learning_Using_Social_Media_during_the_Pandemic_Period_of_COVI D-19_A_Case_in_Universitas_Kristen_Indonesia Nassr, R. M., Aborujilah, A., Aldossary, D. A., & Aldossary, A. A. A. (2020).

Understanding education difficulty during COVID-19 lockdown: Reports on Malaysian university students’ experience. Institute of Electrical and Electronics Engineers. 8, 186939-186950. https://doi.org/10.1109/ACCESS.2020.3029967 Nishanthi, R. (2018). Important of learning English in today world. International Journal of Trend in Scientific Research and Development, 3. https://doi.org/10.31142/ijtsrd19061 Pawlak, M. (2017). How teachers deal with individual differences in the language classroom: Results of a study.

Neofilolog, 179-195. https://doi.org/10.14746/n.2019.52.1.13 Pazilah, F. N., Hashim, H., & Yunus, M. M. (2019). Using technology in ESL classroom: Highlights and challenges.

Creative Education, 10, 3205-3212. https://doi.org/10.4236/ce.2019.1012244 Reimers, F., Schleicher, A., Saavedra, A., & Tuominen, S. (2020). Supporting the continuation of teaching and learning during the COVID-19 pandemic annotated resources for online learning.

OECD. http://www.oecd.org/education/Supporting-the-continuation-of-teachingand-learning-during-the-COVID-19-pandemic.pdf Rodrigues, H., Almeida, F., Figueiredo, V., & Lopes, S.L. (2019). Tracking e-learning through published papers: A systematic review. Comput. Educ., 136, 87–98. https://doi.org/10.1016/j.compedu.2019.03.007 Sayuti, H. A. M., Ann, T. G., Saimi, W. M. S. A., Bakar, M. A., Bakar, A., Dawawi, S.N. A., & Mohamad, M. (2020). Using gold standard project based learning for intermediate year three pupils to enhance English speaking skill: A conceptual paper.

Creative Education, 11(10), 1873-1889. https://doi.org/10.4236/ce.2020.1110137 Shahzad, S. K., Hussain, J., Sadaf, N., Sarwat, S., Ghani, U., & Saleem, R. (2020). Impact of virtual teaching on ESL learners' attitudes under COVID-19 circumstances at post graduate level in Pakistan. English Language Teaching, 13(9), 1-9. https://doi.org/10.5539/elt.v13n9p1

http://ijlter.org/index.php/ijlter

348

Shifflet, R., & Weilbacher, G. (2015). Teacher beliefs and their influence on technology use: A case study. Contemporary Issues in Technology and Teacher Education, 15(3), 368–394. https://citejournal.org/volume-15/issue-3-15/social-studies/teacherbeliefs-and-their-influence-on-technology-use-a-case-study/ Verawardina, U., Asnur, L., Lubis, A. L., Hendriyani, Y., Ramadhani, D., Dewi, I. P., Darni, R., Betri, T. J., Susanti, W., & Sriwahyuni, T. (2020). Reviewing online learning facing the covid-19 outbreak. Journal of Talent Development & Excellence, 12(3), 385-343. https://www.iratde.com/index.php/jtde/article/view/281 Weeden, K., & Cornwell, B. (2020). The small-world network of college classes: Implications for epidemic spread on a university campus. Sociological Science,7(9), 222-241. https://doi.org/10.15195/v7.a9 Wijaya, H., & Helmi, R. (2018). Animation effectiveness for E-learning with progressive web APP approach: A narrative review. International Journal of Engineering and Technology (UAE), 7, 112-120. https://doi.org/10.14419/ijet.v7i4.11.20785 Yen, E. L. Y., & Mohamad, M. (2020). Utilising e-learning to assist primary school ESL pupils in learning to spell during COVID-19 pandemic: A literature review.

Creative Education, 11(8), 1223-1230. https://doi.org/10.4236/ce.2020.118091 Yunus, M. M., Salehi, H., & Chenzi, C. (2019). Integrating social networking tools into ESL writing classroom: Strengths and weaknesses. English Language Teaching, 5(8), 42-48. https://doi.org/10.5539/elt.v5n8p42 Yusuf, M, B. N., & Ahmad, J. (2020). Are we prepared enough? A case study of challenges in online learning in a private higher learning institution during the covid-19 outbreaks. Advances in Social Sciences Research Journal, 7(5), 205-212. https://doi.org/10.14738/assrj.75.8211 Zaharah, Kirilova, G., & Windarti, A. (2020). Impact of Corona virus outbreak towards teaching and learning activities in Indonesia. SALAM: Jurnal Sosial dan Budaya Syar-i, 7(3), 269-282. https://doi.org/10.15408/sjsbs.v7i3.15104

Appendix 1 Interview Questions 1. What online applications (s) do you utilise for your e-learning?

2. Do you follow the lesson plans that you had prepared?

3. What methods do you use?

4. How do you provide assignment and assessment after each lesson?

5. How do you provide feedback to you students?

6. How would you describe your students’ response and participation to elearning?

7. Are your students able to use your e-learning platform? Justify your answer.

8. According to you, what are the benefits and drawbacks of e-learning?

9. How effective is your e-learning?

10. What have you learned about your teaching pedagogy during this pandemic?

11. What are other challenges you face in implementing e-learning?

http://ijlter.org/index.php/ijlter

PUBLISHER Society for Research and Knowledge Management Port Louis Republic of Mauritius www.ijlter.org

International Journal of Learning, Teaching and Educational Research

The International Journal of Learning, Teaching and Educational Research is an open-access journal which has been established for the disChief Editor Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, edEditorial Board ucators, teachers, trainers and other practitionDr. Fatima Zohra Belkhir-Benmostefa ers on all aspects of education to publish high Dr. Giorgio Poletti quality peer-reviewed papers. Papers for publiDr. Chi Man Tsui cation in the International Journal of Learning, Dr. Fitri Suraya Mohamad Teaching and Educational Research are selected Dr. Hernando Lintag Berna through precise peer-review to ensure quality, Dr. Charanjit Kaur Swaran Singh originality, appropriateness, significance and Dr. Abu Bakar readability. Authors are solicited to contribute Dr. Eglantina Hysa to this journal by submitting articles that illusDr. Mo'en Salman Alnasraween trate research results, projects, original surveys Dr. Hermayawati Hermayawati and case studies that describe significant adDr. Selma Kara vances in the fields of education, training, eDr. Michael B. Cahapay learning, etc. Authors are invited to submit paDr. Bunmi Isaiah Omodan pers to this journal through the ONLINE submisDr. Vassiliki Pliogou sion system. Submissions must be original and Dr. Meera Subramanian should not have been published previously or Dr. Muhammad Kristiawan be under consideration for publication while Dr. Wahyu Widada being evaluated by IJLTER. Dr. Som Pal Baliyan Dr. Reem Khalid Abu-Shawish Dr. Froilan Delute Mobo Dr. Mohamed Ali Elkot Dr. Anabelie Villa Valdez Mr. Teody Lester Verdeflor Panela