Tempus BLATT course review report by Anita Jankovic

2014 Pilot Course Reviews

University of Priština, Kosovska Mitrovica

BLATT Pilot Courses Self-evaluation reports and expert reviews Contents: Aleksandar Jovanovic – Endocrinology Self-evaluation Expert review

page page

4 10

Aleksandar Jovanovic – Project Proposals Self-evaluation Expert review

page page

15 19

Biljana Vitosevic – Biology of Exercise Self-evaluation Expert review

page page

24 31

Jelena Djokic – IT in Environmental Protection Self-evaluation Expert review

page page

36 38

Snezana Markovic Jovanovic – Pediatrics Self-evaluation Expert review

page page

39 44

Vladan Peric – Basics of ECG Self-evaluation Expert review

page page

50 57

Anita Jankovic – Contemporary English Language 1 Self-evaluation Expert review

page page

63 71

The Blended Course Peer Review Form used in this report is part of the Blended Learning Toolkit prepared by the University of Central Florida (UCF) and the American Association of State Colleges and Universities (AASCU) with funding from the Next Generation Learning Challenges (NGLC). It is provided as an open educational resource under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

1. Endocrinology – Aleksandar Jovanovic Self evaluation Course name Short name Instructor Moodle

Endocrinology OCE Aleksandar Jovanovic http://elearning.rcub.bg.ac.rs/moodle/enrol/index.php?id=215

1. Course design The course is based on real case studies that we have had over many years of working in the department of endocrinology and intensive care unit of the Department, of course, respecting the privacy of patient data. During the problem-based learning sessions, students would face demands, responsibilities and concerns that doctors and staff had had in a real-time situations 2. Course objectives and learning outcomes Course objective The aim of this course is addressing of students‟ diagnostic, therapeutic and differential – diagnostic capabilities in the field of endocrinology, thereby further developing their basic clinical skills and gaining practical knowledge in an efficient way. Learning outcomes: After the successful completion of the course students will:        

Perform dynamic endocrine testing according to the indications Recognise and analyse and 4ecognize radiographic and echosonografic findings Analyse and interpret radiographic procedures Perform and interpret ultrasonographic imaging procedures Carry out the diagnostic conclusions based on the clinical, laboratory, imaging, electrographic examination results Differentiate endocrine disorders during the case studies Implement and adjust therapy of the endocrine diseases Apply different types and regimes of insulin therapy

ď&#x201A;§

Conduct the medical and supportive treatment of the urgent metabolical disturbances (hypoglycemia, diabetic ketoacydosis, hyperosmolar non-ketotic state)

3. Teaching staff Aleksandar Jovanovic, MD, Professor, Medical Faculty, University of Pristina/K.Mitrovica. Full resume can be viewed here: http://bit.ly/1mOD9lq 4. Facilities and learning resources The Faculty of Medicine of the University of Pristina has two computer-equipped classrooms, one of which is in the library of the Faculty of Medicine, as well as room for the clinical simulation, which is also partially computer- equipped. Unfortunately the internet at studentsâ&#x20AC;&#x; dormitories does not work during the major part of the day, except in the early morning hours and the late evening. Although we took steps to equip at least one or two buildings with wireless internet with satisfactory flow, the opportunities for Internet access would vary individually from student to student depending on where they live during the school year and where they reside when the school year is over; additionally, most of the students who stay in the private accommodation in the town do not have internet access at their homes. Equipment owned by the Faculty of Medicine is mostly obsolete, particularly the computers in the faculty library. One of the classrooms has two servers that allow setting up the Facultyown Moodle platform which is used mainly for the courses in Medical Statistics and Hygiene. However, for the updating and developing of this software, the faculty relies on two former teachers of Informatics / Statistics, who had, in the meantime, left the faculty and find the new employment at the Belgrade University, Serbia; the possibilities of contact with them and the intervention in case of failures are therefore limited. Computers and servers in this classroom are old and the flow rate in the computer network is small and although access to online content is still possible, sometimes it takes minutes to open a single web-page, especially when all the computers in the classroom are on. Although the teachers participating in the BLATT project from the Department of Internal Medicine, personally bought two routers for the Clinic of Internal Medicine in K. Mitrovica, but the connection in the classroom used for the practice learning is not enabled, because the classroom is located in the cellar of the building where no signal is accessible. Therefore, we use the above-mentioned classroom with the servers and computers for our courses, sharing and adjusting terms with other teachers who also use this classroom. Despite the promises given by the University management, the necessary servers, software and computers in the project were not procured yet, although the funds were transferred to the University in due time; other options for modernizing and upgrading the existing equipment are not available. 5

5. Study process and assessment The course was provided for the students during the two semesters in 2013. The course was organized and implemented with some specifics, due to the following factors: 1. In our experience, students at the Faculty of Medicine in K.Mitrovica, rarely apply for any extracurricular activities (for example informal courses, internships) other than language courses they find necessary for the careers abroad. This was also true for the two online mini – courses we attempted to organize via the web site and Facebook twothree years ago, the response was minimal. The reason is very traditionally organized teaching of medicine, with arbitrary defined ECTS credits which does not reflect the real students workload. In reality, by the internal regulation of the students are obliged to attend to all the classes, both theoretical and practical , during the morning hours, and greater part of the afternoon, every day, sometimes with additional classes during the Saturday. Despite this, the students still must learn over 60 % of the materials necessary to pass the exam at their homes. Most students live in other cities and travel over the weekend, so for them it is an absolute priority to finish compulsory part of the studies and shorten the time of studying, which at our faculties takes approximately 8 years. 2. The hybrid courses that we started in the framework of the BLATT project at the Faculty of Medicine are not mandatory but rather complementary learning activity. Although formally recognized as “Other Forms of Teaching” by the faculty and awarded by a small number of ECT credits, in reality these classes are completely based on the efforts of individual teachers and not in any manner recognized by the managements, not just at the faculty level, but also at departments. Students,therefore, have no obligation to participate in these courses out of hours. Therefore, the only way to create a critical mass of students was that the hybrid course is done within theclasses scheduled for practical exercises. The good side of it was that I was able to personally assist students in the registration process on the Moodle platform, explain the principles of operation of the platform, what are teachers‟ expectations, and to clarify certain ambiguities in the course where they appeared. At the same time, this is why the online (video and/or written) instructions on accessing and registering for the courses were omitted from the Moodle platform at this time. In general, there my role as a teacher ended, except when needed to clarify some data, and of course, in evaluation of responses and essays at the end of the course. So far, our experience during two semesters showed that by organizing students to resolve diagnostic and therapeutic problems in small groups were very stimulating and that the results achieved by this groups become fully compatible with diagnostic and therapeutic skills showed by the individual physicians from the health centers participating during the course of continuous medical education; the fourth-year students groups and individual physicians solved the same 6

problems within the two courses, so we were able to compare achievements. All groups of students showed at least satisfactory results in solving complicated clinical studies presented online (the worst result was 70% of the case study resolved, average 85%). The quality largely relied on the group interaction, and partly the capability of the leaders of the groups. Generally, with the better interaction (with the active participation of all group members),the resolution of the tasks and clinical problems was more efficient and the final evaluation grade was better. The assessment was done for the groups of students, not for the individual students, in attempt to promote group work which is bitterly underestimated at the faculty. In the first semester the assessment was based on the scale of 5-10 , but in the second semester were reformulated as a percentage of the completion of the study case. The results in the second semester were presented in the form of the brief reports to the Chief of Cathedra of Internal Medicine, in attempt to include these results in the final grade of the Internal Medicine final exam, but no feedback was received. The reports on the studentsâ&#x20AC;&#x; results in the previous semester are attached to this report. Since the most of the teacher- students as well as students â&#x20AC;&#x201C; students interactions took place in the classroom, alive, they have not been documented on the blog at this stage of the course implementation. There is a strong need, however, to preserve and document the diagnostic conclusions, studentsâ&#x20AC;&#x; comments and opinions online, either by the existing Moodle tools, or with the Google Docs. In this regards the Google Docs are preferable, since we had many problems with the Moodle Platform. On the occasions of upgrading the software (which happened two or three times in last two semesters) many of the users, responses and essays simply disappeared. The process of registration of the students to the course was difficult and took a lot of time during the classes. The return messages with links often went in the spam folders. Usernames were often not accepted or interfered with other users from other universities. The captcha was often unreadable and the instructions for the insertion of the numbers and letters were unclear; the students sometimes wasted a lot of time on this step. In the last semester we had a lot of power outages that have restricted the classes. This problem was partially solved in the past years, but as the equipment became outdated, it re-occurred. 3. Institutional management and support Generally, in terms of implementation of hybrid and electronic teaching, the situation with the management is not satisfactory. Although there is some formal, initial support to the project implementation at the level of university and faculty managements, the real interest in the project by the management is negligible. Unfortunately, although the project provides opportunities for enhancing learning, for striving towards output-based education, for independent learning, for stimulating group work, the acquisition of additional clinical skills, capacity and faculty 7

involvement in future trends in education prior to competing universities, is not recognized by either the administration or by colleagues. Colleagues who showed interest in the project do that mainly out of personal interests, and less because of the desire to modernize the teaching process. As part of its constant striving to maintain the status quo ante bellum, the Faculty and the University miss the great opportunity for development. In a situation where it is clear that the faculty won‟t get any serious funding for the improvement of its scientific research level any time soon, and that the current research funding for the University is absolutely insufficient, and that such situation should certainly be the same for at least the next five years, the development in terms of modern educational trends could improve the rating faculty and expand its capacities. Unfortunately, the chances are that the e-learning and hybrid learning at the university will be accepted only after all private and public higher education institutions in the country largely begin to apply these types of educational activities. So,the faculty will almost certainly retain its retrograde position in relation to other higher education institutions in Serbia. 4. Internal QA     

The quality assurance of the course rely on Teachers self – evaluation Students‟ evaluation (not performed yet) Partner institutions‟ evaluation Regular reporting to the department management

5. Student evaluation results Since the course is not fully developed yet, and since we had different kinds of problems following almost every session on the Faculty of Medicine (starting with the difficulties in internet access to the latest power outages) I was reluctant to perform students‟ evaluation in this semester, because the problems largely influenced students‟ experience during the course. In fact, the first sessions in the first semester were with the least problems, while later we had more and more technical issues. In cooperation with my colleagues I took steps to overcome most of the issues we had, so the coming students‟ evaluation focus more on quality of course itself and on the quality of learning than on the peripheral problems. Regardless, some of the conclusions (although they may be subjective) may be drawn from the experience so far:

Results show that the course exceeds the level of fulfillment of the learning outcomes on the traditional courses; on average, the groups exceed 80 % of the tasks required, while on the traditional course of the Internal medicine the average marks are somewhat above 7 on the 5-10 scale. The students here had the rare opportunity (at least in our study program) to participate in a complementary problem based learning; they showed significant and enviable diagnostic skills, ability to interpret laboratory analyses results, radiological and (to a lesser extent) ECG findings, and the capability of planning and preparation conservative therapy in Internal Medicine. At the same time, these skills of their largely underestimated in the traditional courses at our university, which mainly insist on the theoretical knowledge acquired during lectures and the knowledge gained from text books. The course showed the importance of problem teaching at the Faculty as an alternative to traditional academic ex â&#x20AC;&#x201C; cathedra teaching. Despite initial confusion a certain number of students showed, not used to freely express their own opinions and occasional quarrels among the students who competed among themselves, the sessions were generally well accepted, interesting and the engagement of students was very good. The concept of problem teaching that have been tested during the course, I partially included in the traditional lectures in the field of endocrinology. BACK

Course Review - Endocrinology Course Name: Reviewer Names: % F2F:

Endocrinology

Alastair Creelman, Izidor Golob 80

Course Instructor Name: Review Date:

Aleksandar Jovanovic

% Online:

4 April 2014

Key Exemplary = a model implementation for this criterion Accomplished = excellent implantation, comparable to others Promising = good implementation but lacking in some aspects Incomplete = good start but only partial implementation Missing = no evidence of this criterion Not Appropriate = this criterion is not relevant for the course reviewed

Course Expectations E A P I M NA Criteria Instructions are provided to students on how to start the x course Some possible options: • In-class orientation session • Online orientation materials (e.g., screencasts, diagrams, documents, etc.) The relationship between online and face-to-face x expectations/activities is stated clearly to students Protocols for course communications between students and x instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) Protocols for course communications between students and x instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) All materials required for the course are delineated for x students (e.g., texts, equipment, software, web access, etc.). Notes: Instructions were given in-person (as described in self evaluation report) and are not online and therefore it is difficult to assign grades based only on what is visible in Moodle. There are reports of difficulties with registration process, related to reused usernames and non-readable catcha. In spite of technical and/or managerial difficulties, as reported, students should be given clear course expectations and information on the relationship

between the online material assembled in Moodle and the F2F workshop.

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

Learning objectives/outcomes for each sub-division of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording

Learning objectives/outcomes align with multiple levels of Bloomâ&#x20AC;&#x;s Taxonomy of Educational Objectives as appropriate

Notes: No evidence in Moodle although these issues were addressed F2F as described in the self assessment. No sign that case studies in Moodle are linked to learning outcomes of course or linked to multiple levels in Bloomâ&#x20AC;&#x;s taxonomy. In spite of technical and/or managerial difficulties, as reported, students should be given clear learning objectives of the course. A full description of the module, its learning outcomes and how these will be measured should be at the top of the Moodle page. An introduction video from the instructor explaining the benefits of the module would also provide students with teacher contact before the F2F workshop.

Learning Activities / Content Criteria Learning activities align with stated learning objectives/outcomes Learning activities contain clear, detailed instructions for students Learning activities promote student-to-student interaction Learning activities promote student-to-instructor interaction Learning activities promote student-to-content interaction Content presentations support learning activities Content presentations align with learning objectives/outcomes All web links and embedded media elements included in

x x x x x x x x 11

content presentations are functional Notes: Interaction was F2F. Moodle only used to present case studies but no interaction or instructions there. Learning activities are not seen. There are no web links and/or embedded media elements included in the course.

Learning Assessment E A P I M NA Criteria A variety of learning assessments is provided to students x Learning assessments align with stated learning x objectives/outcomes Course documents state the contributions toward the final x course grade of each graded learning assessment Written scoring criteria are provided to students for each x human-scored learning assessment Care is taken to avoid a â&#x20AC;&#x153;high stakesâ&#x20AC;? testing environment x Course documents include statements detailing when and x where grades will be posted Course documents include statements detailing when and x where grades will be posted Informal assessments are incorporated when appropriate x Notes: Very little evidence of these criteria in the limited content available in Moodle. Since this was a very short module consisting of only one classroom meeting it is hard to apply course criteria. Information on Learning Assessment is not given to the students.

Technology Tools E A P I M Criteria All required technology tools (within Course Management x System or from other sources) have a clearly stated purpose related to course goals All required technology tools have clear student usage x instructions Technical support contact information is provided for all x required technology tools Notes: As reported, technical support was missing, and the necessary servers, software and computers in the project were not procured yet.

No instructions in Moodle about how the material is to be used. All instructions were given F2F.

Ethical / Legal Compliance E A P I M NA Criteria Course materials provide information about how students x with disabilities may receive accommodations. Course materials/activities show evidence of universal design x principles (e.g., video captioning online/face-to-face; “clicker buddies” pairing two students with one personal response device; alternative text for images in online materials, etc.) Course materials contain statements clarifying ownership and x usage rights where appropriate (e.g., “…used with permission…;” “…falls within Fair Use guidelines…;” “…used under the terms of a Creative Commons Attribution 3.0 license…;” etc.) Instructor takes steps to protect students‟ educational x records/privacy rights (e.g., no personal information used in public posting of student grades; encouraging student aliases in online public interactions; etc.) Course materials provide written definitions of and x consequences for student behaviors that constitute plagiarism and/or academic misconduct. Notes: No evidence of any of the above. If no university material is available online on plagiarism issues then link to another university‟s resources. Students with disabilities should be considered in course design. At least offer subtitles to videos, consider use of fonts and colour contrasts in documents and web material and also provide alternative formats, eg if you have a video also provide a low bandwidth text explaining the same points. All external material should be labeled with appropriate CC license (try to use OER as much as possible). Copyright resources need permission to be used or simply link to them

x x x 13

Instructor is responsive to student questions/concerns x Instructor works to maintain a consistent integration between x f2f and online contexts Instructor adapts design of the course to meet emergent x needs as appropriate Students are engaged appropriately in both f2f and online x contexts Notes: Instructor was clearly very active in F2F but no evidence in Moodle of teacher involvement A film of the classroom session was made available showing students working in groups with the online case studies. Due to the evolving status of the course, the teacher was reluctant to perform students' evaluation in this semester, because the problems largely influenced students' experience during the course. Due to missing information, it is not possible to give marks to the questions related to f2f and flexibility of the course design. General notes: This case is really a F2F workshop with additional information available on Moodle and cannot be reviewed as a full blended learning course. Due to considerable technical problems most of this course was F2F with only case studies available in Moodle. These were mostly information and questions that the students could discuss and even if the teacher‟s self evaluation gives promising results from the F2F student interaction there is no evidence of this in Moodle. There is considerable scope for improvement in this course. The classroom interaction could be enhanced and extended by using Moodle more effectively with all information communication based there. Here are some proposals that may be considered:  Students should be given clear information in Moodle on course expectations, learning objectives and assessment.  Adopt a flipped classroom approach by putting pre-workshop material (video, instructions, sample case study etc) on Moodle for students to study before the F2F session to raise motivation.  Encourage students to write their workshop notes in a common digital document (Titanpad or Google Docs) and this can be linked into Moodle immediately after for follow-up. This was done to some extent but the resulting notes were not placed on Moodle as a concrete result of the classroom work.  Provide links to extra material in Moodle including good reasons why they should look at them.  Use students‟ notes/solutions with the following group. After they have completed the workshop they can see how the previous group answered.  Pre-course information should explain that to gain full benefit from the module the pre- and post-classroom work is essential and that all course information and interaction will be made available via Moodle (one-stop shop). BACK

2. Project proposals – Aleksandar Jovanovic Self-evaluation

Course name Short name Instructor Moodle

Project Proposals OPWC Aleksandar Jovanovic http://elearning.rcub.bg.ac.rs/moodle/enrol/index.php?id=226

1. Course design The Project Proposal Writing course is the intensive course for young people who intend to apply their creative ideas in different areas by composing projects which can be submitted for by with appropriate foundations and institutions. Learning outcomes Upon successful completion of this course, participants will be able to: • Perform a SWOT analysis • Define the title and objectives of the project • Compose and write the introductory part of the • Define the results and activities of the project in accordance with its objectives • Become familiar with the types of indicators and define indicators of the project in accordance with the activities • Assess risks and bring out assumptions • Create a strategy for the sustainability of the project • Define resources • Compose the project budget • Become familiar with the most important calls , initiatives , foundations and institutions for project application • Perform an oral and visual presentation of the application 2. Teaching staff Aleksandar Jovanovic, MD, Professor, Medical Faculty, University of Pristina/K. Mitrovica. Full resume can be viewed here: http://bit.ly/1mOD9lqFacilities and learning resources.

The course was held in the premises and the offices of the NGO Center for Training and Development (CTD), located in the CRYM Coalition building in Kosovska Mitrovica . At first, the course was held in the premises of the University Library, but the equipment and the internet were not satisfactory. In the last semester we used the teachersâ&#x20AC;&#x; personal network connection through the mobile phone, which provides a solid internet approach. Students use their own lap-tops, along with the two provided by CTD. Maintaining of the Moodle platform was secured by the cooperation with RCUB, Belgrade. Study process and assessment. This is quite an ambitious project with the purpose to train students and graduates in project applications writing. The interest rate for a course was solid, about 20-30 attendees per semester, although a good portion (at least 10 per semester ) drop out at the very beginning of course, when they find out that they do not learn about writing projects, but have to develop their own project . So far, in two semesters, four groups (of 5-6 people) have been certified, while a group of 7 people remained without certificate for composing the low-quality and uncompleted project; another group of 5 participants dropped out during the midst of the second semester training, but the reasons were not clear, although CTD contacted them in several occasions. All four groups received certificates for the very well done projects with excellent budgets. However, only two of these groups were willing to continue their preparations and submit projects to the appropriate calls. Both projects were rejected, in one case because the part of the project was sent after the call deadline, in the second case because of inattentive reading of the propositions and not finding a suitable partner.These experiences have shown that it is not enough to learn only the technology of writing logical framework and budget, but it is necessary to familiarize the participants with the application forms and the demands placed upon them in the application.

Unlike the medical course in endocrinology, in preparing this course attendees needed much more help from the lecturers. Particularly significant number of complaints was related to the example of the construction of railway in Northern Kosovo given in the Moodle platform, and it seems that this example was not the brightest one and needs to change. In addition, this example encourages all students to define similar types of results and activities such as mentioned in the example. Maybe it would not be bad if the example of building railroads was completely removed, so that the students started preparing the results and activities with more creativity.

Instructions during the course must be expressed with more clarity. Students rely more on the advice of lecturers than their own creative forces, seemingly fearing mistakes, and in the following improvements to the course that fact must be addressed. At the very face to face classes we practice writing projects within the Moodle platform, and the actual proposal is done from home in a collaborative online tool Titan Pad. For the part about writing a budget CTD engaged the external expert. While this was a good solution at the time, it was not to be permanent, however, because the high price of hiring experts and then a limited number of working hours we can afford to the budget writing. Efforts are being made that the teaching of budget writing take CTD staff, which would simultaneously allow permanent contact and consultation between students and teachers . In conclusion, this is a new course that still needs a lot of improving to become fully functional. At the end of the course the evaluation of projects is done by the three-member commission of CTD. During the last semester the budgets proposals were evaluated by the external expert. The projects that were written in a satisfactory manner, and which contained all the necessary chapters are awarded with certificates. The most promising projects are recommended for application with the certain calls and for implementation. 3. Institutional management and support At first, CTD firmly collaborated with the University Career Centre for training in projects writing, however, it appeared that there were too many obstacles blocking that cooperation. For that reason, the CTD constituted itself as a non-governmental organization, but still cooperate with the University in terms of certification. The course management is simple, because there are only three members organizing and implementing this course, one assist in the administrative section, another is the lecturer in the first part of the course, and the third member is prepared to take the second part of the course - the training in budget writing. 4. Internal QA Evaluation of the Project Proposal Writing Course will be done on the basis of self-evaluation reports and evaluation by the partner institutions.

However, the most important measure in ensuring the quality remains the number of project applications submitted and the number of projects accepted for funding. So far we have only initial results, which are far from satisfactory. 5. Student evaluation results Not performed so far.

BACK

Course review – Project proposals Course Name: Reviewer Names: % F2F:

Project proposals Metka Zorič Venuti

Course Instructor Name: Review Date:

A. Jovanović 28 March 2014

% Online:

Course Expectations E A P I M NA Criteria Instructions are provided to students on how to start the √ course Some possible options: • In-class orientation session • Online orientation materials (e.g., screencasts, diagrams, documents, etc.) The relationship between online and face-to-face √ expectations/activities is stated clearly to students Protocols for course communications between students and √ instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) All materials required for the course are delineated for √ students (e.g., texts, equipment, software, web access, etc.). Notes: There is no detailed course description: keywords, level/pre-knowledge, aims, teaching and learning plan, structure of course regarding learning activities and assessment,...

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

√ √

Learning objectives/outcomes for each sub-division of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording Learning objectives/outcomes align with multiple levels of Bloom‟s Taxonomy of Educational Objectives as appropriate

√

Notes: Knowledge about different funding programmes could have been included. Knowledge about the different scope of these programmes could have helped to prevent students to design similar projects

Learning Activities / Content E A P I M NA Criteria Learning activities align with stated learning √ objectives/outcomes Learning activities contain clear, detailed instructions for √ students Learning activities promote student-to-student interaction √ Learning activities promote student-to-instructor interaction √ Learning activities promote student-to-content interaction √ Content presentations support learning activities √ Content presentations align with learning √ objectives/outcomes All web links and embedded media elements included in √ content presentations are functional Notes: Whole course is inside ONE Moodle activity type "lesson" ...you can‟t implement clear and good communication, collaboration, activities, assessment. Basic structure as general is good, promising a lot, good base for do it better :-)

You could have included a Facebook group. This usually intensifies communication among students. But it is important that everybody has access to Facebook and is willing to use Facebook as an additional tool. Google+ is useful for this as well. As most funding programmes have international aspects, it would be great to include co20

operation with another University or other external parties. This could be done e.g. via video conferenes (E.g. Adobe Connect, Google Hangout) A first step could have been to include an international guest speaker through an online conference.

Learning Assessment E A P I M NA Criteria A variety of learning assessments is provided to students √ Learning assessments align with stated learning √ objectives/outcomes Course documents state the contributions toward the final √ course grade of each graded learning assessment Written scoring criteria are provided to students for each √ human-scored learning assessment Care is taken to avoid a “high stakes” testing environment Course documents include statements detailing when and √ where grades will be posted Informal assessments are incorporated when appropriate √ Notes: I saw just one assessment at the end –final presentation of project proposal. Inside lesson are parts where students need to write something, but I don't see what teacher do/write. Moodle has many ways to assess knowledge, but first need to be put down precisely what you'll assess and how students prove they achieve it.

Technology Tools E Criteria All required technology tools (within Course Management System or from other sources) have a clearly stated purpose related to course goals All required technology tools have clear student usage instructions Technical support contact information is provided for all required technology tools Notes: Main rule: choose technology user can handle without problems. Next time you'll do it better.

√ √ √

A great tool to collaborate on project proposals is Google Docs/Drive. Everybody can see all the changes immediately. You do not have to send Word files with the result of confusion which version is the actual version. 21

Ethical / Legal Compliance E A P I M NA Criteria Course materials provide information about how students √ with disabilities may receive accommodations. Course materials/activities show evidence of universal design √ principles (e.g., video captioning online/face-to-face; “clicker buddies” pairing two students with one personal response device; alternative text for images in online materials, etc.) Course materials contain statements clarifying ownership and √ usage rights where appropriate (e.g., “…used with permission…;” “…falls within Fair Use guidelines…;” “…used under the terms of a Creative Commons Attribution 3.0 license…;” etc.) Instructor takes steps to protect students‟ educational √ records/privacy rights (e.g., no personal information used in public posting of student grades; encouraging student aliases in online public interactions; etc.) Course materials provide written definitions of and √ consequences for student behaviors that constitute plagiarism and/or academic misconduct. Notes: Course doesn't have a lot of digital sources. Moodle activity "Assignment" has declaration about plagiarism - students need to click it before they send their assignment (file) to teacher.

Implementation of Blended Course Criteria Instructor is active in guiding students through the course (e.g., issuing reminders, clarifying instructions, etc.) Instructor ensures her/his accessibility by students in both f2f and online contexts Instructor solicits feedback from students Instructor is responsive to student questions/concerns Instructor works to maintain a consistent integration between f2f and online contexts Instructor adapts design of the course to meet emergent

√ √ √ √ √

needs as appropriate Students are engaged appropriately in both f2f and online â&#x2C6;&#x161; contexts Notes: Good start. With more work can be very good blended course. Congratulations! Would have liked to have synchronous tools (video conferences, collaborative real-time text editing for e.g. brainstorming) included. General notes: 1. Course needs detailed description of structure (f-2-f, on-line), content, activities, assessment. 2. Course in Moodle has to be prepared in separate topic units, each with content, communication, ways of collaboration, assignment and assessment. 3.More digital sources, list of additional learning sources 4. Clear criteria about evaluation of student final work. 5. Perhaps different measure of quality (accepted application for funding is VERY hard to achieve). It is already quite a success to submit a proposal in due time! BACK

3. Biology of exercise â&#x20AC;&#x201C; Biljana Vitosevic Self-evaluation Course name Short name Instructor Moodle

Biology of Exercise BIOS Biljana Vitosevic http://elearning.rcub.bg.ac.rs/moodle/course/view.php?id=222

1. Course design The course was a combination of face to face teaching and web components. At the beginning instructions were provided to students on how to enroll in the course and printed instructions were distributed. On the home page of the course were set curriculum, learning outcomes, exam questions and literature. The course was divided into 10 teaching units (topics). The number of hours in the classroom was 4, online component of 2 hours per week. Face to face teaching was in the form of a theoretical introduction to the topic, which was underpinned with anecdotal sports stories, case studies and other discovery learning procedures that explore higher order thinking about real life issues and situations. Web component served as a supplement that contained everything that is explaining better the theory through the additional documents such as Word or PDF processed documents, Power Point presentations, external media-sharing websites (e.g. You-tube). Lessons on the Web were released two days before the face-to face classes to help students understand better the topic or issue (as pre-class preparation). There were 74 students who participated in the course from various towns all over Serbia. Online discussion forum helped a lot in order to stay in touch with students after the face-to-face class with the possibility to discuss different topics and issues related to the course. In addition to the forum, a large number of private messages were also sent to the students when they needed a consultation or advice not only for the course, but also in relation to sports nutrition, supplements, sports injuries etc. Online component has also served for completion of course assignments. For the assessment of knowledge there were the tests made on the basis of questions bank and an essay. The aims of essay are to develop both the skills of scientific thinking and problem solving and to find the relevant literature for the research topic. Students could write the essay alone or in a group and receive feedback on their work. Activity of students in the course has been satisfactory, but when the deadlines for assignments were approaching it was higher. At the end of the course, students were given a questionnaire on course to anonymously give their opinion.

2. Course objectives and learning outcomes The aim of the course is to understand the biological processes in the body and their identification of adaptation to physical exercise of various intensities and duration and positive application of acquired knowledge to the training process and in professional competitions of athletes. Learning outcomes: After successful completion of this course, students should be able to: - Explain the systematic place of a man in nature and understand its evolutionary path - Define internal and external factors of human development and explain it individually through the examples - Specify and distinguish stimulants that are commonly used in sports doping, define and describe the functional characteristics of the hormones used in doping, analyze their impact and negative consequences - Assess the impact of exercise on basic physical factors (height, weight, width and volume of the body) - Consider the impact of exercise on the basic functional systems, possible morphological and functional changes in the improper application of force and intensity of exercise and suggest measures for their treatment - Formulate differences in the thermoregulatory response of children and adults in exercise and explain the mechanisms of acclimation - Adapt exercise to different periods of human development - Compare the morphological, functional and physical abilities of women and men, to recognize problems in female athletes (female athlete triad) and suggest approaches for their prevention and treatment, indicate recommendations for physical exercise during menstruation and pregnancy - Assess the role of exercise as nonpharmacological treatment in various pathological conditions - Describe cell biology, cellular and molecular basis of inheritance and examine the impact of genetic advances on the physical capabilities of man, defines gene doping using appropriate examples of potential candidate genes, evaluating the ethical aspects of gene manipulation

3. Teaching staff / 4. Facilities and learning resources At the faculty there are no proper computers for students in the classroom. 5. Study process and assessment The study process is based on a tentative integration of the theory and practice (web materials). Web documents are not just a collection of files and resources, but the trigger that initiates the process of connecting facts, the processes that drive research, questions, discussion and finally answers. All this together (face to face classes and web component) allows the students to understand the topic through teaching and learning activities (lectures, exercises, discussions), and by assessment (tests, essays) adequately achieve the above-mentioned learning outcomes. Assessment and grading for student achievement: Course activity-10 points, test 1-5 points, test 2-5 points, final test-10 points, essay-20 points, oral exam-50 points. Figure 1.Total activity in the course for three months

(The students were on course skiing one week)

Figure 2.Messages in the course for three months

(The students were on course skiing one week)

6. Institutional management and support / 7. Internal QA - Internal questionnaire for students - Consultation with peers - My teaching experience 8. Student evaluation results In the section your comments, on the question what would you change in the course, students all answered that nothing should change and that everything was great (except for one response to changing professor). On the question what did you lack in this course,students answered that they lacked nothing, everything was available (except one answer that they lack a 6 (grade)).

E - LEARNING PLATFORM (MOODLE) Teacher regularly responds to messages Moodle platform was easy to use Each module (theme) had the clearly defined objectives Course was divided into modules (themes) Assignment deadlines were clearly marked Syllabus was displayed on the platform and was alwaysâ&#x20AC;Ś 0 no

yes

number of students

COURSE QUALITY Course requirements were given in advance Difficulty level was appropriate Tasks were aligned with the learning outcomes Learning was made easier by course tasks and activities Instructions were clear and diverse Course organisation was logical and easy to follow 0

completely disagree

disagree

neutral

30 40 number of students

agree

strongly agree

YOUR EXPERIENCE Course objectives were achieved This course offered me more opportunites for interaction and communication with my colleagues compared toâ&#x20AC;Ś I was more active during this course compared to other course I have attended this semester I understood course materials better in this course compared to others I have attended this semester Course materials were more accessible in this course compared to others I have attended this semester 0 strongly agree

agree

neutral

10 disagree

30 40 50 number of students

completely disagree

Ser.1-for recreational; Ser.2-professional athletes, members of a sport club

100% 98% 96% 94% 92% 90% 88% Category 1 Series 1

Series 2

In my opinion, a blended learning is a good choice for students who are professionally engaged in sports as they can follow the course and complete their assignments in addition to sporting commitments. Students have confirmed this in their questionnaires. BACK

Course Review – Biology of exercise Course Name: Reviewer Names: % F2F:

Biology of exercise Umberto Cao, Peter Diedrichs

Course Instructor Name: Review Date:

Biljana Vitosevic 25 March 2014

% Online:

Course Expectations E Criteria Instructions are provided to students on how to start the course Some possible options: • In-class orientation session • Online orientation materials (e.g., screencasts, diagrams, documents, etc.) The relationship between online and face-to-face expectations/activities is stated clearly to students Protocols for course communications between students and instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) Protocols for course communications between students and instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) All materials required for the course are delineated for students (e.g., texts, equipment, software, web access, etc.). Notes:

x x

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

Learning objectives/outcomes for each sub-division of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording

Learning objectives/outcomes align with multiple levels of Bloomâ&#x20AC;&#x;s Taxonomy of Educational Objectives as appropriate

Notes:

x x x x x x x x

Learning Assessment Criteria A variety of learning assessments is provided to students Learning assessments align with stated learning objectives/outcomes Course documents state the contributions toward the final course grade of each graded learning assessment

x x x

Written scoring criteria are provided to students for each human-scored learning assessment Care is taken to avoid a “high stakes” testing environment Course documents include statements detailing when and where grades will be posted Informal assessments are incorporated when appropriate Notes:

x x x x

Technology Tools Criteria All required technology tools (within Course Management System or from other sources) have a clearly stated purpose related to course goals All required technology tools have clear student usage instructions Technical support contact information is provided for all required technology tools Notes:

x x

Ethical / Legal Compliance E Criteria Course materials provide information about how students with disabilities may receive accommodations. Course materials/activities show evidence of universal design principles (e.g., video captioning online/face-to-face; “clicker buddies” pairing two students with one personal response device; alternative text for images in online materials, etc.) Course materials contain statements clarifying ownership and usage rights where appropriate (e.g., “…used with permission…;” “…falls within Fair Use guidelines…;” “…used under the terms of a Creative Commons Attribution 3.0 license…;” etc.) Instructor takes steps to protect students‟ educational records/privacy rights (e.g., no personal information used in public posting of student grades; encouraging student aliases in online public interactions; etc.) Course materials provide written definitions of and consequences for student behaviors that constitute plagiarism and/or academic misconduct.

x x

Notes:

x x x x x x x

General notes: First of all, it has to be mentioned that the whole evaluation herein provided is mainly based on the Teacher's self-evaluation and on the course's platform accessed as guest user. The evaluation therefore incurs in the following major limitations: 1 The evaluator has almost no means to assess most of the aspects declared on teacher's self-evaluation 2 The evaluator had no opportunity to observe f2f and online settings during the course term 3 Several areas and activities on the platform were not accessible to the evaluators (ex. forums, tests) 4 The language barrier, being all learning materials and resources on the platform in Serbian. This said, the course generally results very well structured and with a strong coherence between the declared learning objectives, the contents provided and the learning activities. Concerning the on-line Platform, this looks clear and intuitive, thanks to the efficient structure of the sections (recalling the teaching units) and with a controlled number of topics per section. An appropriate use of Moodle calendar was made too, which helps students in organizing tasks and deadlines. The declared main purpose of the on-line Platform was to provide resources and 34

documents aimed to get students interested in the topics and eventually raise related discussions at f2f classes. The use of the Platform results absolutely coherent with that, being several contents and references provided on it (ex. videos on Youtube) potentially very interesting for students (especially athletes), therefore promoting and facilitating peer interactions and exchanges. However, a greater space to group activities and co-working spaces might have been dedicated within the organization of the course and on the Platform. Basically, the Platform of this Course was mainly used an on-line repository, where to store documents, resources, links and host some activities (like tests): it was made a definitely good and effective use in this sense and this has to be considered as an achieved result. What it might be imagined as a further development of that, it is a step forward from the actual "Content management system" to a "Learning management system": this would especially mean to try having some activities running live on-line on the Platform, in some cases with the teacher involved too (ex. webinars, Questions & Answers live sessions,) and not last, a social area connected to the course, based on the major social networks (ex. Facebook, Twitter, Linkedin, Youtube dedicated channel, etc.) It is also appreciable the three-stages internal QA process implemented by the teacher and it would be interesting to know how this supported the teacher and with which outcomes. Users' evaluation system seems good and effective too: it a very high which showed very high levels of satisfaction for the Course. Finally, a key element to be taken in due consideration for further developments of the course is the involvement of students in devising new actions/sections/initiatives (on-line, but not only), asking them â&#x20AC;&#x201C; for instance - what they would like to find on the Platform, what kind of resources, what kind of social solutions, what kind of activities they would like to carry out on-line, etc. And of course, a clear and effective support from the University and the Faculty establishment is a fundamental condition for a positive setting-up and development of any blended learning process.

BACK

4. Informatics in Environmental Protection â&#x20AC;&#x201C; Jelena Djokic Self evaluation Course name Short name Instructor Moodle

Informatics in Environmental Protection ITZS Jelena Djokic, PhD http://elearning.rcub.bg.ac.rs/moodle/course/view.php?id=235

1. Course design The MSc course consists of an induction week, then eight taught modules, a group project, and an individual research project. The early part of the Informatics in Environmental Protection course is structured around a series of taught modules consisting of short lectures, on-line demonstrations and practical exercises. Each module forms the sole unit of study for a period of two weeks. An opportunity to undertake a project in the style of a consultancy project is offered program. The third period of the course is devoted to the individual thesis project. The individual modules are linked through case studies and practical work so that the different aspects of the technologies utilized are integrated. The group project is an applied multi-disciplinary activity, providing student teams with the opportunity, whilst working under academic supervision, to gain an intimate knowledge of practical applied informatics technologies and systems, working within agreed objectives, deadlines and budgets. The project involves the application and integration of component informatics technologies and methodologies (such as visualization, GIS, modeling, software design, and geo-statistical and statistical analytics) to produce quality-assured innovative solutions. The individual project involves the application and integration of the approaches and component technologies learned on the course. It provides students with the opportunity to demonstrate their ability to carry out independent research, and to think and work in an original way, contribute to knowledge, and overcome genuine problems in environmental informatics.

2. Course objectives and learning outcomes The students will get key skills and techniques such as:    

data mining, statistical interpretation, visualization and predictive analytics for handling such data collected by environmental monitoring.

The students will have competences in modeling and simulation of environmental situation by applying environmental management tools in order to perform feasibility study for the future objects and processes or Environmental Impact Analysis for the existing production processes and objects. 3. Teaching staff Jelena Djokic,PhD 4. Facilities and learning resources There is a classroom equipped with desk computers and good internet connection 5. Study process and assessment The study process is based on an integration of the theory and practice (web materials). Web downloads are used for practicing using different software to estimate, evaluate and simulate different processes, mechanisms and kinetics of abiotic ecosystems through the process of collecting data, research, discussion and conclusions. All this together (face to face classes and web component) allows the students to understand the topic through teaching and learning activities (lectures, exercises, discussions), and by assessment (tests, essays) adequately achieve the above-mentioned learning outcomes. Assessment and grading for student achievement: 10% Practical work 20% Assignment 40% Project paper 30% Project presentation 6. Institutional management and support / 37

7. Internal QA This year, the course of Informatics in Environmental Protection is new course, elective in the third semester of Master study program in Environmental Engineering, and compulsory in the fourth semester of Master study program in Occupational Safety Engineering. This October, the course has started, but just two of the students have taken this course as elective. After a short period of time, they have been engaged in some business activities, and chosen to freeze the semester, until next year. That made my course useless in the winter semester, and, as the students didn‟t take any exercises I have stopped uploading the material. This was out of my control, but this could be avoided, if more students had decided to take this elective. Now, in the summer semester, there is another chance, but, even if this is mandatory course, the group of master students is very small, just three of them. With proper approach, I hope to have better outcome. The student survey cannot be used as a tool for course improvement. The second tool for evaluation, peer review was also not applicable, and I hope to have some input in the summer semester. The third tool, students‟ success, is bad, as they didn‟t even try to do the project, or to sit the exam. My conclusion is that I should have taken some other course for blended learning, as I also teach Recycling of Waste Materials, the compulsory course that is popular among the students. BACK

Course review – IT in environmental protection This course could not be reviewed since it did not take place as planned. A new course is planned during spring 2014 and that can be reviewed at a later date. BACK

5. Pediatrics - Snezana Markovic Jovanovic Self evaluation

Course name Short name Instructor Moodle

Pediatrics PVP Snezana Markovic Jovanovic http://elearning.rcub.bg.ac.rs/moodle/enrol/index.php?id=218 Guest access: pedinadaljinu

1. Course design The course is designed for students of Medicine and Health Care study programs on the Faculty of Medicine in K. Mitrovica. The course is the part of the subjects of Pediatric on the fifth study year of Medicine study program and the second year of Health Care study program. Also, this course may be useful for medical doctors and pediatricians in a sense of reviewing or supplement their knowledge. The students have the opportunity to discuss the issues with the professor and get quick answers to their questions. On the other hand, students who live in places away from the Faculty locations and are unable to attend studentsâ&#x20AC;&#x; consultation hours can do so in this way. The course is designed in the form of several individual topics in pediatrics incorporated in the studentsâ&#x20AC;&#x; practical training. The course is designed course is in the form of lessons, illustrations, video clips, links with which they can obtain further information. In addition, since students do not have textbooks for learning and preparing practical exam in Pediatrics, this course also provides them with the necessary materials for the preparation of their practical exam. During the course, the students have the opportunity to access the written and video materials and to apply the knowledge in diagnostic and therapeutic procedures; by resolving the appropriate tests at the end of the each topics they can check the acquired knowledge and skills, or return to the specific area in the case of need. The students also have the opportunity to further demonstrate learned techniques in the direct patient examination and evaluation right after the onlinesessions, during the practical class. Furthermore, during the regular classes of PediatricsI can combine the online-course topics with the materials presented during the lectures to create the studentsâ&#x20AC;&#x; discussions on specific issues and to analyze their achievements. 39

2. Course objectives and learning outcomes The aim of this course is the institution of the joint interactive activities in order to improve students medical skills and capabilities to analyze and interpret various types of data obtained by specific pediatric techniques and to resolve medical problems.

Learning outcomes Outcomes: Upon successful completion of the course, the student will be able to: Master the basic knowledge of pathological changes and laboratory findings in pediatric diseases Analyze data and distinguish between normal and pathological findings; to discuss, describe, or express in the form of essay the expected laboratory and biochemical changes To consider the differential diagnosis of the selected pediatric problems

Methods

Previous introduction with the online preparatory course materials set on the Moodle platform; Visit and practical work in the pediatric laboratory Essay on laboratory diagnostics

Previous regular lectures in Pediatrics. The course online differentialdiagnostics tests set To construct a protocol for the Work in small groups: diagnosis of pediatric diseases planning of the diagnostic protocols To conclude on the therapeutic Previous regular lectures in scheme applicable in the Pediatrics. treatment of the selected Work in small groups: pediatric diseases planning of the therapeutic protocols

Strategy and Methodology of Assessment Group discussion, evaluation and assessment ( on the 5- 10 scale) Essay assessment (5-10 scale)

Differential-diagnostic tests assessment (0-100% scale)

Oral presentation (5-10 scale )

Group discussion and assessment (5-10 scale)

3. Teaching staff

Snezana Markovic Jovanovic, full resume can be viewed here: http://bit.ly/MC9A9p 4. Facilities and learning resources: By the online connection students can access the course set on the Moodle platform from their homes or student dormitories where they live and wherever there is an adequate technical support. Students are introduced to the course within the regular practical classes, in the classrooms of the Faculty of Medicine at the premises of the Health Centre Kosovska Mitrovica provided for teaching of Pediatrics. The learning resources used for this purpose are the personal computers and students‟ mobile phones and video projectors provided by the Faculty; due to the poor internet connection in the classroom, I used my own personally created Wi-Fi network. The books used as a source material for the course: -

Nelson Textbook of Pediatric- Robert Kleigman, Behrman Richard et al., 18th edition; Part II, Growth, Development and Behavior; Part XXV – The Endocrine System Lekarski priručnik iz Dečje nefrologije- Milena popović Rolović, Amira Peco Antić, Olivera Marsenić, 3. Ispitivanje funkcije i morfologije bubrega Dečja Gastroenterologija-Dobrivoje Filipović- Ispitivanje gastrointestinalnog trakta Pedijatrija- Duško Mardešić i sar. Pog. 7: Poremećaj prometa vode, elektrolita, minerala i poremećaji acidobazne ravnoteže Rudolphs Pediatrics- Abraham Rudolph, Jullien Hoffman, Colin Rudolph- Part 23Clinical Approuch to Neurologic disease

Online links: https://www.google.rs/search?newwindow=1&q=lumbar+puncture+child&oq=lumbar+puncture +chi&gs_l=serp.1.0.0i19l4j0i22i30i19l2.294042.295660.0.298494.4.4.0.0.0.0.268.784.0j3j1.4.0.. ..0...1c.1.36.serp..0.4.784.Z3Y6MrD3brE https://www.youtube.com/watch?v=R2_0gOI8uV0&feature=player_embedded http://www.bd.com/resource.aspx?IDX=20324 http://www.youtube.com/watch?v=pnnYXqHWYoU 5. Study process and assessment

The process of the hybrid learning is new to the Medical Faculty, and the students meet with the new model providing the new type of availability of teaching material and the new type of assessment for the first time. Until now, the practical part of the course was based on the professor's presentation of study and requiring students to repeat the lessons or results The advantage of the hybrid learning methods is the availability of course materials on-line, the abundance of links or video clips for a presentation of techniques or procedures, the availability of opportunities for additional training, improvements or specializations according to personal interests... There is sufficient time for student to read material, process it, then compose and ask questions, or analyze practical study cases through blogs. My experience during this school-year shows the student's initial interest in this kind of learning, but it also shows there is still insufficient information about the possibility of this type of learning. They are interested in group work and animated by the case studies. The downside of course that there is no acceptance or recognition of the test results by the administration of the Faculty and Head of the Department, as the official part of the assessment and the exam. Therefore, I have to say that students do not have sufficient motivation for active participation in the course and for solving the tests. Methods for assessing the achievement of the learning outcomes are twofold: subjective methods of evaluating knowledge and objective methods of measuring knowledge. By the subjective method I evaluate oral responses in class, explaining and presenting seminars. This method is used for evaluation of factual knowledge and procedural knowledge of the students. Also, by the subjective methods I evaluate the student's ability to quickly adopt and apply new knowledge based on existing facts. The objective measurement of knowledge is carried out by solving tasks within the test after each lesson. Successful completion of the test should enable student to take the Childrensâ&#x20AC;&#x; History and Physical Examination part of the exam and oral part of the Exam in Pediatrics afterwards. 6. Institutional management and support By the faculty administration, there is still no official approval and acceptance of hybrid courses as an official way of learning and passing exams. The Head of the Department is aware of the content and operation of the course. So, for now, the course Practical Exercises in Pediatrics is, so far, relied exclusively on the involvement and engagement of the individual teacher. 7. Internal QA Monitoring of the efficiency of the course is conducted by the self-evaluation reporting, student evaluations at six-month and annual basis. The evaluation by the Faculty management, unfortunately I cannot get, because of yet unrecognized importance of the course for the development and changing of the higher education. 42

8. Students evaluation Student evaluation test is going to be implemented by using anonymous questionnaires that were completed by the student after six- month period. Questions were: - How did you learn about the Practical Exercises Course in Pediatrics? - Have you been engaged with this type of course before? - If you completed the Practical Exercises Course exercises in Pediatrics how would you assess it on a scale of 5 to 10? - How much this course is of use to you in your learning Pediatrics? - Are you a supporter of the methodology of hybrid learning and e-learning and what next topics would you suggest? BACK

Course Review - Pediatrics Course Name: Reviewer Names:

% F2F:

Pediatrics Peter Diedrichs peter.diedrichs@lnu.se Umberto Cao umberto.cao@unibo.it Mainly

Course Instructor Name: Review Date:

Snezana Markovic Jovanovic March-April, 2014

% Online:

Students consultation for those living far away. Course materials.

Course Expectations E A P I M NA Criteria Instructions are provided to students on how to In regular class. start the course Acknowledged, Some possible options: but not much • In-class orientation session info provided in • Online orientation materials (e.g., screencasts, SE. diagrams, documents, etc.) The relationship between online and face-to-face x expectations/activities is stated clearly to students Protocols for course communications between Not students and instructor are provided (e.g., general acknowled “Netiquette,” best venue for questions, procedures ged in SE. for conducting discussions, etc.) All materials required for the course are x delineated for students (e.g., texts, equipment, software, web access, etc.). Notes: Information for evaluation of those criteria is gathered from the self evaluation report (SE).

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

Learning objectives/outcomes for each subdivision of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording

Learning objectives/outcomes align with multiple levels of Bloomâ&#x20AC;&#x;s Taxonomy of Educational Objectives as appropriate

Notes: Information for evaluation of those criteria is gathered from the self evaluation report. The Topics information and setup on the course page could be more elaborated, to be more resembling a â&#x20AC;&#x153;study guideâ&#x20AC;?.

I M

N A

x x x x x x x Except for Lesson 45

content presentations are functional activities, se Notes. Notes: Information for evaluation of most criteria is gathered from the self evaluation report. There is one Lesson (in Topic 1) that seems to be designed as a quiz, but one cannot finish it. Lesson in Topic 5 cannot be opened (unavailable) There are three quizzes, mainly online-text questions.

Learning Assessment E A P I M Criteria A variety of learning assessments is provided to students x Learning assessments align with stated learning x objectives/outcomes Course documents state the contributions toward the final course grade of each graded learning assessment Written scoring criteria are provided to students for each human-scored learning assessment Care is taken to avoid a â&#x20AC;&#x153;high stakesâ&#x20AC;? testing environment Course documents include statements detailing when and where grades will be posted Course documents include statements detailing when and where grades will be posted Informal assessments are incorporated when appropriate Notes: Information for evaluation of most criteria is gathered from the self evaluation report.

x x x x x x

x x

P I

x x

As far as we can tell.

Course materials provide written definitions of and consequences for student behaviors that constitute plagiarism and/or academic misconduct. Notes:

I M

x x x X x x x

Notes: The online contact possibilities are not very good, with only a News forum. Recommended: Questions for teacher, Course discussions. (Se below.) In interview the teacher described that evaluation/student feedback will be asked for.

General notes: It has to be mentioned that the whole evaluation herein provided is mainly based on the Teacher's self-evaluation (SE). This evaluation therefore incurs in the following major limitations: 1. The evaluator has almost no means to assess most of the aspects declared on teacher's self-evaluation 2. The evaluator had no opportunity to observe f2f and online settings during the course term 3. The language barrier, being all learning materials and resources on the platform in a language we the evaluators do not speak or understand. Google translate has been to some help, but it has its limitations. 4. Our limited access to “behind the scenes”, with activity logs, settings and quiz results, among other things. This course is a good start, with lots of course material that seems nicely put together. And there are quizzes in the end of the course modules. A main problem seems to be that the students are not very active on the course page, and a reason for this is that it is quite static with not very much room for interaction. Also there is little acceptance of online test result among the faculty administration. But, quizzes are very effective as a way of self-evaluation, and for the teacher to point out important matters. This way the online “course room” will support the learning processes and the quality of performance in f2f assessments. The students are reported to appreciate group work and case studies, and a course online has good opportunities for such work. It is very much a matter choosing the right online tasks and proper tools, such as forums, wikis and other tools. Suggestions to develop the online course design to initiate activity and interaction: 1. More of a “course study guide” design. Use text in the topics to clarify course objectives and time frames. 2. Put all course- and subject documentation in the Moodle room, for more efficient course administration and as an incitement for students to use the room from start – or even before the course starts. 3. Use the Moodle calendar, and time activities inside as well as outside of the Moodle room. 4. Use of forums a. Use the “Announcements” forum to inform the students of changes/news. b. Open up a channel to ask the teacher questions, both practical and 48

concerning course objectives, and to encourage the use of this. Declare this to be the main channel for practical questions. (Forum: Questions for the teacher). c. Use of possibilities for online support/interaction between students (forums for discussions, peer support and “tips”), and to encourage the use of those. d. Use Q&A forums to give tasks aligned with the online material. For instance, each student is to write what he finds most difficult, or most interesting, in what he just studied. He can see what his fellow students wrote after he has submitted his own post, and he can then comment/discuss in other students posts. e. Create student groups and group forums in the online course room – same groups as f2f, if such are used there. To use with groups assignments, but this is also a good way of keeping in touch for students that do not live near campus. f. Encourage user pictures/avatars in the profile, and start with the teachers. This is a very simple but quite effective way to create a social foundation for everyone, and it makes online discussions more “real” as well as easier to follow. 5. Further development of online tests/quizzes a. Instead of ending each topic that way, consider splitting them up/ create more elaborated quizzes to align with each learning activity and the learning objectives. b. If possible use more multiple choice (auto grading), to unload grading chores from teacher (as mainly self-evaluation). c. If practical, set auto requirements for the students to be able to do the next test. 6. Adjustments along the way Designing a course is an ongoing process. Ask the students at the start and regularly during the course how they would like to use the Moodle room. It‟s mostly quite easy to apply changes and try out different methods for collaboration.

BACK

6. Basics of ECG â&#x20AC;&#x201C; Vladan Peric Self evaluation Course name Short name Instructor Moodle

Basics of ECG Vladan Peric http://elearning.rcub.bg.ac.rs/moodle/enrol/index.php?id=217 Guest access: 007

1. Course design Basics of ECG is a hybrid course that consists of on-line course (MOODLE platform) and faceto-face learning. Course Basics of the ECG is a part of practical training of Cardiology and the cardiology is a part of teaching subject Internal Medicine. The course is therefore lasted a month. It consisted of six hours of instruction per week in the classroom and the unlimited possibilities of on-line access to the course (at least 6 hours per week). In the classroom at the beginning of the course Students of acquainted with the basics of the hybrid course and the basics of using the Moodle platform. All the students are divided into written instructions on how to login to the Academic Network of Serbia and registration on the course. On-line course was divided into 10 teaching units (topics). The eleventh issue was the final test, and the twelfth was a questionnaire for student evaluation of the course. All lessons contain a lot of pictures with detailed explanations of how to identify and distinguish different ECG abnormalities. At the end of each teaching unit, there are issues (with and without ECG recordings), which was used for training. For all the lessons and issues students are able to return many times. On the home page of on-line course, students proposed that literature can be used to facilitate the learning course. Home page contains three forums. One forum was reserved for notification of all updates to the course, the other to clarify various ambiguities in the course, and the third to solve a variety of clinical problems based on the offered ECG recordings. During the course every day is answered on the students' questions and all questions were explained to. During face-to-face learning are further explained to the lessons and the students have interpreted different ECG recordings. At the end of the course there was the 11th theme with the final test that included 30 questions. Questions were imported from a bank questionnaire which contains 220 questions. On the final 50

test questions were mixed in the order with different students. In addition, the time limit for completion of the test (the test lasted for 1 hour) is reduced in the irregularity of the test and enable resolving test results, which were the result of the actual knowledge of each individual student. Students are able to solve the test only once. The course enrolled 78 students. 74 students took the final test. At the end of the course the students completed two questionnaires for the assessment of the course (one within the on-line course). 2. Course objectives and learning outcomes Considering its extensive teaching of internal medicine and cardiology, the goal of the course Basics of EKG was to facilitate medical students the content of electrocardiography. Learning outcomes: - Getting to know the way of spreading depolarization impulses through the heart. - Understanding of the technical characteristics of electrocardiographs and the importance of correct linking of electrodes. - Overcoming the exact determination of heart rate at different heart rhythms. - Identifying various atrial and ventricular arrhythmias. - Recognition of AV blocks and other disturbances of implementing pulse depolarization through the heart. - Determination of the electrical axis of the heart. - Recognition of ECG signs of hypertrophy of the atria and ventricles. - Diagnosis of different forms of coronary artery disease using the ECG recordings. - Identifying various electrolyte disturbances, digitalis effect, pulmonary embolism, WPW syndrome, pacemaker rhythm, and other disorders. - Training of students for independent clinical interpretation of ECG recordings. 3. Teaching staff / 4. Facilities and learning resources Department of Internal Medicine has a hall with a computer and projector for Power Point presentations. The Faculty of Medicine has a computer center with a small number of computers and internet connection, which is common to all of the Department of Medicine. These are insufficient resources to adequately progress of lectures and exercises.

5. Study process assessment The study process is based on a tentative integration of the theory and practice (web materials). Web documents are not just a collection of files and resources, but the trigger that initiates the process of connecting facts, the processes that drive research, questions, discussion and finally answers. All this together (face to face classes and web component) allows the students to understand the topic through teaching and learning activities (lectures, exercises, discussions), and by assessment (tests, essays) adequately achieve the above-mentioned learning outcomes. Assessment and grading for student achievement: Course activity-10 points, test 10 points, final test-40 points, oral exam-40 points.

Figure 1. Total activity in the course

6. Institutional management and support /

7. Internal QA - Internal questionnaires for students - Consultation with peers - My teaching experience 8. Student evaluation results These are the results of the student questionnaires (the questionnaire was completed by 56 students).

Detailed results of the answers to all the questions in this test are shown in Appendix 1 here: http://bit.ly/1mQphad The second questionnaire was completed by 20 students. The results are shown in the tables below.

Table 1. The quality of teaching The percentage of students who chosen the appropriate answer I completely I have no I do not Strongly I agree agree opinion on this agree disagree The organization of the course was logical and easy to follow The instructions were clear and varied There is ample opportunity to practice and apply the major concepts Tasks and activities are easier to adopt of teaching units The tasks were in line with the outcomes of the course Assignments are given the opportunity to demonstrate my knowledge Weight assignments was appropriate The teacher gave comments on the assignments timely Teacher comments were constructive Terms of successfully overcome of the course are given in advance

Table 2. Learning platform (Moodle) The percentage of students who chosen the appropriate answer Yes No The curriculum is set up on a platform and is always available The curriculum includes a list of necessary materials, and how to reach them. Deadlines for assignments are clearly indicated The course is divided into learning units (modules, lessons, themes) Each teaching unit follows the same pattern Each teaching unit has clearly defined goals Instructions for use of the platform is easily accessible Moodle platform is easy to use Technical support is available The teacher regularly responds to messages

100

90 95

10 5

100

100 95 95

0 5 5

Table 3. Your experiences The percentage of students who chosen the appropriate answer I completely I have no I do not Uopšte se I agree agree opinion on this agree ne slažem More freely ask for help in this course than in other attending this semester Teaching materials in this course are more accessible than in other courses to attend this semester I better understand the teaching material on this course than in other courses to attend this semester The materials presented on Moodle platforms have been interesting The materials presented on Moodle platforms have facilitated me to easily learn lessons I was more involved in this course than in other courses to attend this semester This course offers more opportunities to interact and communicate with colleagues than other courses they attend this semester Interaction and communication in this course is better than the other courses they attend this semester This course offers more opportunities to communicate with the teacher than the other courses they attend this semester Objectives of the course that were given / disclosed at the beginning are fully met

At the end of the questionnaire the students report that they are easier to overcome of the course contributed to a large number of images with explanations, and test questions for practice after each lesson. The largest number of students is very pleased with the content and organization of the course that they significantly facilitate learning electrocardiogram and would not change anything. The others complained about the course timing because it lasted, during the holidays 55

and that there was not enough time to solve the final test. Also, some students prefer to practice in the classroom with their teachers. All student comments are presented in Appendix 2 here: http://bit.ly/1fxDRCZ In my opinion, blended learning is a very useful method for education of students of medicine. It is especially useful way to learn ECG. A large number of images and graphics, forums, discussions, and constantly available material on the Internet allow students a large number of practical views and the opportunity to spend as much time learning as much as they need. The students through questionnaires confirmed the benefit of the blended learning. BACK

Course Review -Basics of ECG Course Name: Reviewer Names: % F2F:

Basics of ECG Alastair Creelman, Izidor Golob

Course Instructor Name: Review Date:

Vladan Peric 30 March 2014

% Online:

Course Expectations E A P I M NA Criteria Instructions are provided to students on how to start the x course Some possible options: • In-class orientation session • Online orientation materials (e.g., screencasts, diagrams, documents, etc.) The relationship between online and face-to-face x expectations/activities is stated clearly to students Protocols for course communications between students and x instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) All materials required for the course are delineated for x students (e.g., texts, equipment, software, web access, etc.). Notes: Classroom introduction should be supplemented by online resources for those who miss the F2F introduction or need to check something. University should have common online videos, FAQs and instructions for using Moodle and other online tools and these can be available on all Moodle course rooms. Common resources explaining “netiquette” and online conduct should be available on all courses rather than depending on individual teacher. Instructions were given in-person (as described in self evaluation report) and were not given on-line. Students were satisfied by the technical support (95 %).

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

Learning objectives/outcomes for each sub-division of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording Learning objectives/outcomes align with multiple levels of Bloomâ&#x20AC;&#x;s Taxonomy of Educational Objectives as appropriate

Notes: Students should be given clear learning objectives of the course. They are not available on the course page. Each unit in Moodle should be clearly introduced and linked to learning objectives. This is not present in the course. A general introduction unit with course description, objectives and what is expected of students is also for the most part missing.

Learning Activities / Content E A P I M NA Criteria Learning activities align with stated learning x objectives/outcomes Learning activities contain clear, detailed instructions for x students Learning activities promote student-to-student interaction x Learning activities promote student-to-instructor interaction x Learning activities promote student-to-content interaction x Content presentations support learning activities x Content presentations align with learning x objectives/outcomes All web links and embedded media elements included in x content presentations are functional Notes: Learning activities for the course are not seen on LMS. There are no embedded media elements included in the course. The 3 forums in Moodle are clearly defined but unfortunately not so well used by students

and was mostly used for information from the instructor. There are several arenas for student interaction, both in the Moodle forum and even more in the Facebook group. Linking to external resources (ECG quiz etc) is a good way of extending the classroom and the link to the international ECG Facebook group allows students to interact with a global network which is excellent. Student evaluations show that they were very satisfied with the course and in nearly all categories preferred the blended approach to a traditional F2F course.

Learning Assessment E A P I M NA Criteria A variety of learning assessments is provided to students x Learning assessments align with stated learning x objectives/outcomes Course documents state the contributions toward the final x course grade of each graded learning assessment Written scoring criteria are provided to students for each x human-scored learning assessment Care is taken to avoid a â&#x20AC;&#x153;high stakesâ&#x20AC;? testing environment x Course documents include statements detailing when and x where grades will be posted Informal assessments are incorporated when appropriate x Notes: This is an area for improvement in the course. I could not find much evidence of any of the above points in the online material available. The final test seems to be rather a high stakes activity. Maybe more use of other assessment activities (peer assessment of written assignments etc) as well as quizzes would help. Information on Learning Assessment was given to the students, however there is room for improvement. The reviewers suggest improving the variety of learning assessments and moving away from a final high stakes test.

x 59

Technical support contact information is provided for all x required technology tools Notes: Explanations of the linked activities (Facebook groups, external quizzes and resources) should be included, explaining how they contribute to the course objectives and the students‟ learning. It‟s not enough just to link to something and hope they use it. Technical support seems lacking except for the teacher which leads to a considerable burden on the teacher and vulnerability. Clear instructions for all tools should be available, preferably in different formats (video and text), and the availability of technical support should be clearly stated with contact times and addresses. However, based on students‟ responses they were extremely satisfied with the learning platform (Moodle) and technical support (90 % satisfaction at least).

Ethical / Legal Compliance E A P I M NA Criteria Course materials provide information about how students x with disabilities may receive accommodations. Course materials/activities show evidence of universal design x principles (e.g., video captioning online/face-to-face; “clicker buddies” pairing two students with one personal response device; alternative text for images in online materials, etc.) Course materials contain statements clarifying ownership and x usage rights where appropriate (e.g., “…used with permission…;” “…falls within Fair Use guidelines…;” “…used under the terms of a Creative Commons Attribution 3.0 license…;” etc.) Instructor takes steps to protect students‟ educational x records/privacy rights (e.g., no personal information used in public posting of student grades; encouraging student aliases in online public interactions; etc.) Course materials provide written definitions of and x consequences for student behaviors that constitute plagiarism and/or academic misconduct. Notes: No evidence that disability issues have been addressed. Alternative forms of material should be included for students with less bandwidth (video/audio/text alternatives and downloadable files for offline work). Subtitles for videos are also useful. University should develop common films, texts, FAQs about plagiarism and copyright issues (or link to existing resources from other universities) and this should be available to all students. No information on these issues was found for this course.

No information on Ethical and Legal Compliance was given. The reviewers note that information on grades is public (within the course) and students‟ names are given on-line along with their grades. This could lead to integrity issues.

Implementation of Blended Course E A P I M NA Criteria Instructor is active in guiding students through the course x (e.g., issuing reminders, clarifying instructions, etc.) Instructor ensures her/his accessibility by students in both f2f x and online contexts Instructor solicits feedback from students x Instructor is responsive to student questions/concerns x Instructor works to maintain a consistent integration between x f2f and online contexts Instructor adapts design of the course to meet emergent x needs as appropriate Students are engaged appropriately in both f2f and online x contexts Notes: The instructor has put considerable effort into the design and implementation of this course and has included many promising elements to encourage student interaction. At present the course is highly dependent on the instructor‟s enthusiasm and is there fore very vulnerable if the instructor is ill for any period. Encouraging students to take turns at moderating forum discussions and getting them to produce own learning resources as group work could help to relieve the instructor‟s workload. Due to missing information, it is not possible to give marks to the questions related to f2f and flexibility of the course design. Instructor's responsiveness can be seen from the forum and there is a room for improvement (see students' evaluation). He was good in giving instructions on the clarifying instructions and was able to get students' evaluation.

General notes: The implementation looks promising, however there is a room for improvement of the course. Some general information on the course is missing. The students were more involved in this course than in other courses to attend this semester (85 %) and the materials presented on Moodle platforms have been interesting (100 %).  Information about course design, objectives, structure, study methods etc should be clearly available on Moodle.  Technical support in form of instruction films, easy guides, FAQ should be available on Moodle. 61

ď&#x201A;ˇ

Each module must have learning objectives and information about how they are to be met and what assessment criteria are to be used.

BACK

7. Contemporary English Language 1- Anita Jankovic Self evaluation Course name Short name Instructors

Contact hours ECTS Moodle

Class blog

91110 Contemporary English Language 1 SEJ 1 Nenad Pejic, PhD (Lectures) Snezana Zecevic (Seminar 1) Anita Jankovic (Seminar 2) 3,75 7 https://elearning.rcub.bg.ac.rs/moodle/login/index.php user name: aleksandrajovic1 password: jankovic http://sejcomm.edublogs.org/

1. Course design SEJ 1 course is delivered for the first year students of English philology at the Faculty of Philosophy in Kosovska Mitrovica. It's an integrated language skills course at the level B2 of Common European Framework of Reference. It is designed asa supplemental model of a blended course, which incorporates technology into the instructional approach of the course, but does not alter its basic structure. Students are required to complete online readings or activities, or participate in online discussion and projects. However, there is no reduction in course meeting time. The course combines three learning environments, each dealing with a separate set of language and learning skills, which also support each other in the „presentation-practiceproduction‟ cycle. Face to face:the classes are organized on a biweekly basis; students attend lectures in normative grammar for 90‟ on one week. In the alternate week, they come to seminar classes for 6 hours. The seminars are held by two teaching assistants who work on separate language skills. Teacher Snezana revises grammar points taught in lectures through reading and listening. I work on speaking and writing; as well as vocabulary building through these skills.

Since guest access has great limitations, I provided log in details for a fake student account I created.

Moodle: Moodle acts as supporting learning platform in a sense that it contains the syllabus and the course materials, something to fall back on in case a student misses a class, or is unable to attend classes at all. In addition, it is a home to drill-and-kill grammar work and vocabulary revision. Also, it contains support materials for blogging and preparation materials for term tests and speaking assignments. Class blog:I have looked into different platforms that support class/group blogging and initially opted for Tumblr powered by Disqus add-on for comments, but realized that Edublogs offer a better blogging platform. The class blog is a place for students to share their reflections on learning, findings on English language and culture, resources for learning English, and their musings on the topics we cover in class, as well as post their written assignments for peer review. 2. Course objectives and learning outcomes The primary goal of this course is revision and leveling of the previously acquired language competences to the B2 level of the Common European Framework of Reference for Languages. In addition, the course aims at developing productive and receptive skills through customized content and topical tasks. Elaborated course objectives can be found in the syllabus: http://elearning.rcub.bg.ac.rs/moodle/mod/book/view.php?id=5126&chapterid=32 The learning outcomes in SEJ 1-8 courses at my Department are aligned with the EQUALS/ALTE level descriptors. Detailed can-do-statements for this course can be found here: http://bit.ly/1jEfH96 3. Teaching staff Three people were involved in the delivery of this course. Professor Pejic was supportive of the initiative as long as his workload was not increased. Teacher Snezana was only observing during the piloting stage. She registered for Moodle and was present in the course participant list, but didnâ&#x20AC;&#x;t take any active part. She will take on a more active role in the continuation (SEJ 2 course). I was in charge of setting up the course, creating course materials, providing tech support, assessing student work and moderating discussions. In many aspects I was omnipresent and I needed to take a step back. For that purpose, I created a fake student account, Aleksandra Jovic. I used this account to push forward stalling discussion threads, ask questions my students failed to ask, and generally check whether and how the assignment instructions worked. My students were

Log-in required.

not baffled by this fake account as there were several students who didn‟t come to classes but took active part in Moodle. I was exploring Moodle functionalities long before I started designing this particular course, so I felt comfortable and confident with the interface. BLATT webinars and especially training sessions with Metka were very helpful. Also, a vast number of Youtube tutorials on Moodle came in handy. In addition, I often consulted Moodle Docs. Finally, helping students cope with Moodle, gave me further insight into the inner workings of the platform. I needed help with creating learning materials and activities because the start of the online component of the course was overwhelming. Students had many questions and issues I had to resolve, which was time consuming. Furthermore, it took even more time making them post their questions and issues in the Help Forum where everybody could benefit from the answer. 4. Facilities and learning resources Moodle: The Serbian Academic Network provides free hosted Moodle to all HE institutions, as well as tech support and teacher training. I like that my course on their platform is a part of a wider academic context. Consequently, I am not just taking my students outside their classroom, but also putting them in the middle of the national academic community. The problem with course design and delivery in this particular Moodle site was the untimely response from the administrators and tech support. Additionally, I felt very limited with instructor‟s permissions in the site. Online tools: I made screencasts with Screenr demonstrating the use of every tool I introduced. A really rewarding moment was during a particular activity which asked of the students to complete an external interactive test and share their results and comments. Several students used Screenr to record themselves doing the test. Learning happens even where we didn‟t plan it. Computer/internet access: The Faculty of Philosophy offers a well equipped computer lab; unfortunately the internet access is erratic and unreliable. When we needed to work in Moodle in class for demonstration purposes, I would share my phone‟s 3G internet and that was sufficient. The students had access to high speed internet in the library and could use those two computers there to work on the assignments and activities in their own time. For those who had their own laptops, each dorm offers Wi-Fi access. In addition, the University Library supports Eduroam internet access which is free of charge for students, though complicated to register for. Books: Faculty Library offers complete list of the required readings and course books. Consultations: We do not have official office hours for consultations with students. Usually this happens on the go, or doesn‟t happen at all. Therefore, students are reluctant to ask for one-on65

one time with their instructors. I set up a virtual office on Meeting.io site and allocated four time slots every Friday. Again, not a lot of students asked for consults, they mostly relied on having their issues resolved via Moodle messaging system or the Help forum. 5. Study process and assessment There were seven Moodle modules that covered two weeksâ&#x20AC;&#x; worth of learning activities and study materials. In addition, there were two sections that aimed at helping students prepare for the term tests and speaking assignments. We planned to have students spend 2-3 hours working on a module, which constituted 40 hours of the overall student workload in this course. Therefore, the final division of the workload looked like this: 60 hours of class time, 40 hours in Moodle, and 75 hours of self study.

Module: topic, grammar point, lexical set, language function

Lecture: presentation of grammar point

Seminar 1: practice of grammar point

Seminar 2: language function, presentation and practice of lexical set

Moodle: revision of grammar point and lexical set

Blog: revision of lexical set and language function (production)

The elements of the assessment are as follows: 66

Element Class participation Term tests Moodle Blogging Written exam Oral exam

Percentage 10 20 10 10 40 10

Details of individual assessment methods can be found in the syllabus: http://elearning.rcub.bg.ac.rs/moodle/mod/book/view.php?id=5126&chapterid=63 6. Institutional management and support I‟ve submitted a formal proposal on the transformation of the SEJ 1 course to my Department Head and Faculty Management in September. My Department Head approved the proposal and acted on my behalf with the Faculty Management. They informally approved, but that is as far as their interest in the matter reached. I had no obstructions in delivering the course, had all the technical facilities at my disposal, as well as declarative support in front of the student body. Minor rise in the interest spiked during the EACEA representative‟s visit in November 2013. 7. Internal QA Formative assessment of the course was performed through surveys, exit slip notes in class and conversations with students. The students were firstly surveyed on their digital skills and habits at the beginning of the course through a direct questionnaire and a quiz entitled „Are you ready for this course‟. In the second half of the course, I administered a native Moodle survey, COLLES Actual. And finally, at the end of the course, they completed a student satisfaction questionnaire. 8. Student evaluation results Initial surveying showed that my students spend approximately 2-3 hours a day online. 60% of that time is spent surfing the internet or connecting with friends via social networking sites. One fourth of their online time is spent in studying or any kind of school related activity. As for their perceived digital skills, on average, 50-77 percent declared to be very skilled with e-mailing software, word processors, presentation software and photo editing programs. They were least skilled in audio and video editing software. Finally, asked how they learn to use new programs and applications, only 4 percent answered they would contact tech support, while equal 3

Log-in required.

percentages opted for: watch video tutorials, read instructions, ask a friend, and by themselves through trial and error. The quiz on their readiness for e-learning revealed that 35 percent of them have many characteristics of the successful online student. They seem to realize that online component requires a considerable amount of time. They finish all projects they start and are persistent in reaching their goals. They seem to learn very easily, have good problem-solving skills and feel very comfortable using online communications. All these are very important qualities for successful online learning, and you seem to be fully prepared to take charge of your own learning process. The fact that you consistently do things in advance and keep track of all their assignments also bodes well for their success in online learning. They are very experienced computer users, and didnâ&#x20AC;&#x;t have any problems accessing or interacting with the online component of the course. Half of the students needed to improve some of their technical skills and equipment before taking the course. They seem to be well-organized. They are prepared to pace themselves, figure out things on their own and communicate with people in writing. They generally seem to realize that taking a hybrid course is more time consuming and requires more study discipline. Overall, they performed fairly well in this course, with a little extra effort on their part. Only 15 percent of students underperformed in this quiz. They lacked independence and timemanagement skills. Based on their answers about their technical readiness, it seems that they are not very comfortable using the computer for learning. Complete course evaluation results can be accessed at the links below. COLLES results: http://bit.ly/collesresults Student evaluation results: http://bit.ly/evaluacija 9. Final considerations Overall, I am satisfied with the course design. The three learning environments intertwined and complemented each other, and have proven effective in meeting the course objectives. However, certain elements of the design have failed in the field test. Here are the lessons Iâ&#x20AC;&#x;ve learned: Moodle ď&#x201A;ˇ

Too many modules too soon. My weekly timetable stipulated that registration, adjustment, and the first module should be completed in the first week. This was a highly unrealistic expectation. I had students registering for Moodle throughout October. Therefore, most of the next two modules had a low completion rate resulting in lower 68

grade. I remedied this by repeating major assignments in the Late Assignments section further down the line. For the next semester (SEJ 2 course), I‟ve prepared fewer modules, the first module starts on the third week of the semester, so there‟s plenty of time for adjustment. Not that they need it now, they know what to expect and what is expected of them. Here is the activity log:



   

Discussions in the Q&A forum livened up after I gave the students the assistant roles with permissions to rate each other‟s contributions and posts. Though, by doing that I‟ve negated the basic function of a Q&A forum, because the students could see each other‟s posts and responses before posting their own. Having a fake student account, as cheating as it could be, turned out to be an efficient way of boosting discussions. Generally, giving students more executive role is a powerful motivator. In the next course, they alone will be in charge of the course glossary. FAQ sections should be an integral part of the revised course. Written instructions should always accompany a video tutorial.

Blog 

Blogging activity failed in its primary function for two reasons. The initial idea was for the blog to have versatile posts. Unfortunately, it became a medium for publishing writing assignments only. Secondly, there was very little interaction between students. They would cordially respond to my comment, but there was no conversation going on between themselves. Again, I believe the main culprit to be too many writing assignments. Therefore, in the next course, commenting will get more attention and be 69

ď&#x201A;ˇ

rewarded by badges or symbolical awards in class. Also, there will be fewer writing assignments. However, the blog as a medium turned out to be very valuable as it promoted writing for public, emphasized editing and proof reading skills, and general thoughtfulness inwriting and word choice. There have been instances of plagiarism, but I managed to handle them.

BACK

Course Review – Contemporary English Language 1 Course Name: Reviewer Names: % F2F:

SEJ 1 Alastair Creelman, Izidor Golob

Course Instructor Name: Review Date:

Anita Jankovic 4 April 2014

% Online:

Course Expectations E A P I M NA Criteria Instructions are provided to students on how to start the x course Some possible options: • In-class orientation session • Online orientation materials (e.g., screencasts, diagrams, documents, etc.) The relationship between online and face-to-face x expectations/activities is stated clearly to students Protocols for course communications between students and x instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) Protocols for course communications between students and x instructor are provided (e.g., general “Netiquette,” best venue for questions, procedures for conducting discussions, etc.) All materials required for the course are delineated for x students (e.g., texts, equipment, software, web access, etc.). Notes: Excellent and comprehensive instructions to students using a good mix of text and video. The quality of video and especially audio stream however could be improved. Good that many instructions are both in video and reinforced in the syllabus and other text sources. Principles for class interaction are clearly stated stressing importance of respect and reflection. Clear information about the concept of blended learning and the balance between F2F and online.

Learning Objectives Criteria Course-level learning objectives/outcomes are written clearly in student-friendly wording

Learning objectives/outcomes for each sub-division of the course (e.g., modules, lessons, weeks, chapters, etc.) are written clearly in student-friendly wording

Learning objectives/outcomes align with multiple levels of Bloomâ&#x20AC;&#x;s Taxonomy of Educational Objectives as appropriate

Notes: Objectives are clearly stated in the detailed syllabus and each unit refers clearly to objectives. Good variation of activities that provide different modes of interaction with the material being studied.

Learning Activities / Content E A P I M NA Criteria Learning activities align with stated learning x objectives/outcomes Learning activities contain clear, detailed instructions for x students Learning activities promote student-to-student interaction x Learning activities promote student-to-instructor interaction x Learning activities promote student-to-content interaction x Content presentations support learning activities x Content presentations align with learning x objectives/outcomes All web links and embedded media elements included in x content presentations are functional Notes: Combining Moodle and the course blog is an excellent way of providing different learning arenas and giving students a window to the outside world in that the blog as well as recorded videos are publicly viewable. Student interaction is clearly encouraged in all course elements though the response has not been so great though many took advantage of the help forum to ask questions direct to the teacher. This difficulty in activating student discussion is not unusual since students are clearly accustomed to traditional classroom

teaching and they will become more active in future courses. Having a fake student account seems to be an efficient way of boosting discussions (from the self evaluation report).It is however vital that future courses are designed in a similar way and there is a consistent pedagogical approach from all the teachers they meet. Nice touch to provide a link to another group‟s blog to encourage more interaction between course groups. This can be extended as more blended courses take place.

Learning Assessment E A P I M NA Criteria A variety of learning assessments is provided to students x Learning assessments align with stated learning x objectives/outcomes Course documents state the contributions toward the final x course grade of each graded learning assessment Written scoring criteria are provided to students for each x human-scored learning assessment Care is taken to avoid a “high stakes” testing environment x Course documents include statements detailing when and x where grades will be posted Course documents include statements detailing when and x where grades will be posted Informal assessments are incorporated when appropriate x Notes: All activities are clearly explained in terms of assessment and criteria. Impressive to include peer assessment at this level though this is a skill that students will need a few terms of practice to fully benefit from. It‟s important that once you start with peer assessment that it is enhanced with each course the students take. First attempt may not work so well but they need practice and continued discussion on the merits of peer assessment. Rating each other‟s contributions can be a powerful motivator. Very clear structure and signposting throughout the course.

x 73

Technical support contact information is provided for all x required technology tools Notes: Excellent support but very vulnerable since only one person seems to deal with every aspect of the online course. Support must be more distributed to be sustainable. As noted in the self-evaluation report, written instructions should always accompany a video tutorial.

Ethical / Legal Compliance E A P I M NA Criteria Course materials provide information about how students x with disabilities may receive accommodations. Course materials/activities show evidence of universal design x principles (e.g., video captioning online/face-to-face; “clicker buddies” pairing two students with one personal response device; alternative text for images in online materials, etc.) Course materials contain statements clarifying ownership and x usage rights where appropriate (e.g., “…used with permission…;” “…falls within Fair Use guidelines…;” “…used under the terms of a Creative Commons Attribution 3.0 license…;” etc.) Instructor takes steps to protect students‟ educational x records/privacy rights (e.g., no personal information used in public posting of student grades; encouraging student aliases in online public interactions; etc.) Course materials provide written definitions of and x consequences for student behaviors that constitute plagiarism and/or academic misconduct. Notes: Consider making course material CC-licensed and introducing students to using CC photos instead of copyright protected material (use CC Search http://search.creativecommons.org/ to find photos, diagrams, music etc). Use CC music as background to videos to avoid copyright issues. Students with disabilities need subtitles in videos and always best to avoid music and speech at same time (hard to hear instructions in some course videos due to music interference). Alternative low bandwidth options for important material – eg text alternative to a video. Links to Serbian material on plagiarism would be appropriate – important they fully understand definitions and own language is always best in such cases.

General notes: This course could be presented as a role model for all other courses at the university in terms of structure, clarity, media use, interactivity and support. The implementation looks extremely promising. There are some options for further improvement, such as activity competition feature in LMS. Perhaps grouping of students can have some positive effect. A peer assessment functionality could also have some positive effect (the instructor gave the students the assistant roles with permissions to rate each other‟s contributions and posts Q&A forum for this purpose). A few points that could be addressed:  Share responsibility among 2-3 teachers plus possibly a tech support person and a librarian. This reduces vulnerability if the lead teacher gets ill.  Encourage students to find and assess (source criticism) extra learning resources (incl films, articles etc) – librarian would be a useful extra resource for the course.  Introduce aspects of copyright/CC into the course encouraging the use of open CC images, music and text.  Sound quality in some videos (eg introduction video) is poor due to recording in different environments. Good idea to film outside and inside but better microphones are needed. Hard to hear some parts due to background music.  Consider introducing collaborative writing via Google Drive or similar to encourage students to comment on and actively edit texts. If teacher also has access 75

ď&#x201A;ˇ

to these texts she/he can comment on work in progress and provide formative assessment. Consider letting small groups take turns at being course moderators for a week, taking responsibility for leading discussion activities and then producing a course newsletter / video blog / podcast with a summary of the lessons learned and highlights from the week. This will take the load off the teacher.

BACK