ELSE manual - for instructors

Page 1


Eco/logical Learning and Simulation Environments in Higher Education




Table of Contents FOREWORD: The ELSE partnership..................................................................................................... 5 INTRODUCTION ....................................................................................................................................... 7 ELSE: Where did it all start? .................................................................................................................... 7 The common regulatory frameworks ................................................................................................. 7 Beyond the frameworks - what actions are needed? ......................................................................... 8 The ELSE consortium, its strategy and objectives............................................................................... 9 Before all ELSE - Intellectual Output 1, and scoping the horizon ..................................................... 11 Some reflections on the achievement of the ELSE outcomes - digital strategies and the importance of the human factor .......................................................................................................................... 12 CHAPTER 1 ............................................................................................................................................ 13 ELSE: The pedagogy and the tools ........................................................................................................ 13 Blended, digitally supported learning, and strategies to implement it ............................................ 13 Active learning – peer instruction and the flipped classroom .......................................................... 14 The ELSE tools to effectively support blended, digitally enhanced learning .................................... 17 •

ECORE........................................................................................................................................ 17

EVOLI ......................................................................................................................................... 18

EDASH ....................................................................................................................................... 18

CHAPTER 2 ............................................................................................................................................ 19 ECORE – Serious Games as an interdisciplinary pedagogical tool for experiential learning ................ 19 HOW TO DESIGN A GAME USING ECORE.......................................................................................... 20 HOW WAS ECORE USED BY THE PROJECT PARTNERS? ..................................................................... 29 Application 1: ECORE in the Translation class: Moving House (Author: Alba Graziano, Università della Tuscia, IT).................................................................................................................................. 29 Application 2: ECORE in the Teacher Training class: Un inizio difficile (A difficult start, Author: Patrizia Sibi, Università della Tuscia, IT)............................................................................................ 32 Application 3: ECORE in the Auditing Course (Author: Nieves Gómez Aguilar, Universidad de Cadiz, ES) ..................................................................................................................................................... 36 Application 4: ECORE in the Literary Translation class: Getting It (Author: Aba-Carina Pârlog, West University of Timişoara, RO) ............................................................................................................. 39 CHAPTER 3 ............................................................................................................................................ 40 EVOLI – video tagging to support the flipped learning experience ...................................................... 40 What EVOLI looks like, how it works and how it facilitates the flipped model ................................ 40 How to use EVOLI as a tutor ......................................................................................................... 41 How to use EVOLI as a student ..................................................................................................... 44 How the data from the video viewing exercise support the live session ..................................... 45 EVOLI IN PRACTICE – and some evaluation ...................................................................................... 46 3|Page

Application 1: EVOLI in a BEGINNERS’ LATIN CLASS (Author: Nicoletta Di Ciolla, Manchester Metropolitan University, UK) ............................................................................................................ 47 Application 2: EVOLI in the Translation class (Author: Alba Graziano, Università della Tuscia, IT) .. 50 Application 3: EVOLI and the Translation class – dealing with Equivalence Problems (Author: AbaCarina Pârlog, West University of Timişoara, RO)............................................................................. 52 Application 4: EVOLI and the Stylistics class (Author: Nicoletta Di Ciolla, Manchester Metropolitan University, UK) .................................................................................................................................. 53 Application 5: EVOLI in a Financial Management and a Corporate Finance class (Author: Violeta Madzova, International Balkan University, North Macedonia).......................................................... 55 Application 6: EVOLI in the Physics class (Author: Matteo Passoni, Politecnico di Milano, IT) ........ 56 .......................................................................................................................................................... 57 Application 7: EVOLI in an Auditing course UCA (Author: Nieves Gómez Aguilar, Universidad de Cadiz, ES) ........................................................................................................................................... 58 Application 8: EVOLI and a discussion on teaching methodology. (Author: Marius-Mircea Crișan, West University of Timișoara, RO) .................................................................................................... 59 Application 9: EVOLI in the Management Accounting course (Author: Paulino Silva, Polytechnic of Porto, PT) .......................................................................................................................................... 61 Application 10: EVOLI in the Multimedia Contents course (Authors: Luciana Oliveira & Arminda Sa Sequeira, Polytechnic of Porto, PT) .................................................................................................. 63 CHAPTER 4 ............................................................................................................................................ 65 EDASH – The assessment tool............................................................................................................... 65 EDASH – general overview of the tool .............................................................................................. 66 Viewing Assessment Data ............................................................................................................. 66 Tutor Feedback on the initial versions of EDASH, and improvements made ............................... 67 REFERENCES and SUGGESTED FURTHER READING............................................................................... 69 FOLLOW US AT: ..................................................................................................................................... 72


FOREWORD: The ELSE partnership The partners of the ELSE consortium are: Università della Tuscia, Italy (PI and project coordinator: Alba Graziano) Manchester Metropolitan University, United Kingdom (PI: Nicoletta Di Ciolla) West University of Timişoara, Romania (PI: Aba-Carina Pârlog) International Balkan University, North Macedonia (PI: Violeta Madzova) Politecnico di Milano, Italy (PI: Nicoletta Di Blas) University of Cyprus, Cyprus (PI: George Papadopoulos) Universidad de Cadiz, Spain (PI: Nieves Gómez Aguilar) Instituto Politecnico do Porto, Portugal (PI: Paulino Manuel Leite da Silva) Uniwersytet Im. Adama Mickiewicza W Poznaniu, Poland (PI: Aleksandra Jankowska) Entropy Knowledge Network s.r.l., Italy (PI: Michela Fiorese) Universitatea de Medicină, Farmacie, Științe și Tehnologie Din Tậrgu Mures, Romania (PI: Bianca Han) All the partners have contributed to the content of this manual, which has been put together by Nicoletta Di Ciolla, Huw Bell and Chrissi Nerantzi, of the Manchester Metropolitan University, UK.

The ELSE project has been funded with support of European Commission Agreement no. 2018-1IT02-KA203-048006. The content reflects only the views of the authors and the Commission cannot be held responsible for any use which may be made of the information contained therein.



INTRODUCTION ELSE: Where did it all start? It all started about twenty years after the introduction of University reform laws in many EU countries following the Bologna process, which in its turn had started at the end of the last millennium. As university teachers, we started wondering if all our efforts to design new courses, to introduce new subjects and topics of greater appeal to students, and to increase greater transparency and flexibility in our teaching behaviours and practice, were really achieving the effects expected by the European legislators inspiring the Bologna process. And, most of all, we started wondering if academic staff, often working in an older university system based mainly on research and much less on teaching, were really aware of what the Bologna process implied for facilitating students’ learning.

The common regulatory frameworks A series of European documents indicating the scope, objectives, and keywords of the nascent European Higher Education Area (EHEA) had been available for years:  A Framework for the Qualifications of the European Higher Education Area (EHEA) including the so-called Dublin descriptors (Bologna Working Group, 2005) 1  Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) (Bergen 2005, updated in 2015). 2 These intersect with parallel suggestions for Lifelong learning (LLL) and the underlying pedagogies such as the Recommendation 2008/C

111/01/CE of the European Parliament and Council of Europe for Lifelong Learning, introducing the European Qualifications Framework for lifelong learning (EQF). 3 Over time, references to LLL have gradually disappeared and the qualifications have been extended to the whole education system until alignment with the Bologna process has been established (level 6 = BA; level 7 = MA; level 8 = PhD). This document defines knowledge, abilities and competences, and these definitions lie behind both the Dublin descriptors and the EQF. Although not completely homogenous, these two frameworks were intended primarily to develop learner competences in university curricula and to assist in course design. In order to help design academic curricula, in 2010 the Tuning Project produced  A Tuning Guide to Formulating Degree Programme Profiles Including Programme Competences and Programme Learning Outcomes: 4 besides introducing a very interesting distinction between competences and learning outcomes, this guide offered examples of good practice and of both generic competences and learning outcomes, often following Bloom’s revised taxonomy. Further recommendations and inputs came at the dawn of the current decade (2011) from a European Commission communication with an accompanying document regarding the 2020 “horizon”.  Supporting growth and jobs – An agenda for the modernisation of Europe’s higher

http://ecahe.eu/w/images/7/76/A_Framework_for_Qualifications_for_the_European_Higher_Education_Area.p df 2 https://enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf 3 https://ec.europa.eu/ploteus/sites/eac-eqf/files/journal_en.pdf 4 http://www.core-project.eu/documents/Tuning%20G%20Formulating%20Degree%20PR4.pdf, p. 6. 1


education systems highlights: a strong need for flexible, innovative learning approaches and delivery methods: to improve quality and relevance while expanding student numbers, to widen participation to diverse groups of learners, and to combat drop-out. One key way of achieving this, in line with the EU Digital Agenda, is to exploit the transformational benefits of ICTs and other new technologies to enrich teaching, improve learning experiences, support personalised learning, facilitate access through distance learning, and virtual mobility. 5  The Development of Higher Education Systems in European Countries 6 further emphasised that a high percentage of students dropped out during or after their first year of undergraduate study, and a limited number reached the end of the degree: in consequence, some EU countries have been unable to reach the EU target of 40% of young people attending university by 2020.

Beyond the frameworks - what actions are needed? We knew this already, from experience. Somehow, the reading of all these documents increased the number of questions already looming in our minds as to the efficacy of our actions in the several areas of university reform. We were all aware of the great efforts university systems had already made to try and introduce ICTs in the conviction that technology and digitalization alone could provide the necessary innovations in teaching and learning: blended learning through the use of platforms such as Moodle; full e-learning for distance courses, especially for specialised Master degrees or in

telematic universities; Open Courseware projects, based on short video-lectures with slides and/or internationalised materials; Open Educational Resources; 3-15 minute Knowledge Clips such as TedEd lessons; MOOCs (Massive Open Online Courses), produced by individual universities such as federica.eu, PoliMi’s POK (PoliMi Open Knowledge), or consortiums such as Coursera, EMMA, OpenUpEd (panEuropean), and more recently EduOpen. Yet the observation of substantially disappointing results reinforced our doubts:  Are we teaching/learning competences at university level?  How can we develop pedagogies and teaching methods to improve students’ competence learning?  Does technology systematically equate effective learning?  How can we transform both distance and inpresence learning into student-centred practices?  If, for example, we adopt a flipped classroom approach and use any of the abovementioned digital materials, what precisely are we to do in class?  And what about assessing competences and learning outcomes at all stages (entry tests, assessed coursework, summative exams)? Two more documents appeared between 2015 and 2016 to increase our doubts and the sense of failure:  The Bologna Process Implementation Report on The European Higher Education Area in 2015 confirmed that, in most systems, neither curriculum nor assessment design referred to the intended learning outcomes. 7  The investigation carried out by the Tuning Academy between 2011 and 2016, significantly entitled A LONG WAY TO GO… A Study on the implementation of the learningoutcomes based approach in the EU and the

COM/2011/0567 final at https://ec.europa.eu/assets/eac/education/library/policy/modernisation_en.pdf SEC/2011/1063 final at https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011SC1063&from=EN 7 http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/182EN.pdf, p. 18. 5 6


USA (by Tim Birtwistle and Robert Wagenaar) indicates a very clear correlation between a generalised weakness of the pedagogic and technological backgrounds of academic teaching staff and the delay in reaching anticipated outcomes in the quantity and quality of graduates. Although good practices have been identified, the actual implementation of the student-centred approach is not proceeding beyond a discourse on the paradigm shift and there is no certainty it will be achieved. There has been a failure to engage with and convince academic staff about the necessity and advantages of this paradigm shift. 8

The ELSE consortium, its strategy and objectives A group of researchers from Università della Tuscia (Italy) decided to investigate these topics as part of an Erasmus+ project, involving a large group of partners from the European Higher Education Area in a Strategic Partnership. Even the preparatory stage, involving nine universities from across Europe, represented an opportunity to learn how far the much longed-for Bologna process of harmonization among HE systems and teaching innovation had gone. All the partners involved ‒ besides Tuscia and PoliMi (Italy), WUT and UMFST (Romania), UCA (Spain), ISCAP-Porto (Portugal), UCY (Cyprus), UAM (Poland), IBU (North Macedonia) and even the far more advanced UK partner (MMU) ‒ agreed that much had been achieved in the individual countries: harmonisation of student workloads, comparability of national systems, embedment of employability in the curriculum and creation of links with employers; implementation of quality evaluation processes and greater transparency of and access to information; and foregrounding of the “student voice” through surveys and participation in decision making processes. Yet,

everybody agreed that two fundamental principles of Bologna (echoed in the Lisbon document 9 and the Lifelong Programme for adult education) remain unrealised: 1. students continue to be peripheral to the process of knowledge co-construction; 2. the potential for true pedagogical innovation through new technologies that can enhance the learning experience is underexplored. On the one hand, technology remains an innovative toolkit used to deliver traditional content, rather than a means to drive a process of radical pedagogical innovation. Although embedded to a variable extent in everyday teaching and in curriculum design, technology and digitization alone do not generate innovative teaching/learning. On the contrary, the use of Lower Order Thinking ICTs in class leaves the pedagogical relation exactly on the same terms as before or in some cases even strengthens the “excathedra” lecturing format. On the other hand, the deficit in academic teaching staff can be ascribed to more general pedagogic weaknesses: one can surmise that, possibly for generational reasons, university instructors are on average not fully aware of the general change in the millennials’ sensorium and learning style due to the ongoing digital innovation and traditionally neglect learner-centred pedagogies (holistic, constructivist, connectivist approaches) or ignore how to activate learning processes through the many possible versions of problem-solving, learning-by-doing, gamification, digital information research, etc. made possible by Higher Order Thinking ICTs. For this reason, an IT firm specialised in Technology Enhanced Learning through simulation environments, Entropy Knowledge Network (Italy), was invited to contribute to the development of updated pedagogies and to help create our own simulation environment.

A shorter version is available at http://www.tuningjournal.org/article/view/1064/1256. The Lisbon Agenda urged HEIs to be a driver of a knowledge-based economy through innovative research and by giving students the skills to boost economic growth, as also indicated in the Europe 2020 strategy (European President, 2002). 8 9


Thus, ELSE-Eco/logical Learning and Simulation Environments in Higher Education took shape as a strategy to achieve the fundamental European goal of redesigning Higher Education, facilitating the application of Bologna principles across Europe. The main objective is to be pursued through the design of an integrated curriculum where the delivery of academic content is coterminous with, and enhanced by, the acquisition of competences and transversal skills appropriate for the digital age. After the publication of the European Framework for the Digital Competence of Educators (DigCompEdu) 10 in 2017, which organises educator-specific digital competences in 6 areas, ELSE project chooses this framework as reference and inscribes itself in the objectives of areas 3 and 4 of DigCompEdu, mainly by “managing and orchestrating the use of digital technologies in teaching and learning” and by “using digital technologies and strategies to enhance assessment” at tertiary education level. We chose the Humanities and Social Sciences as our reference disciplinary areas, as these are the fields where the partners work, confident that what ELSE studies and creates can be extended to the hard sciences. In this context, the ELSE consortium has set three sub-goals addressing a number of needs and conducive to five main intellectual outputs (IOs): • in order to satisfy the need to share information on good practice and teaching policies about curriculum development, adopted pedagogies and the use of learning technologies, a survey of current activity and best practice in the teaching of Humanities in participating institutions and countries has been carried out with the creation of a hypertext based on a database (IO1); • the need to increase learner-teacher and learner-learner interaction, remove disciplinary barriers, enhance competence learning and metacognitive awareness while safeguarding the acquisition of academic

knowledge has implied experimenting innovative pedagogies (including narrative approaches, inquiry-based approaches such as Problem-Based Learning, think-aloud, quest-storming, digital scholarship, projectbased learning and simulation) and providing recommendations for the best ICT applications for competence teaching at university level; the specific objective to design digital learning environments that support blended learning and active students’ participation has responded to the need:  to encourage personalised, collaborative and cooperative learning/teaching, aiming at student inclusion, retention and motivation, by building upon consolidated digital experiences (Moodle, Open Courseware, Knowledge Clips, Open Courses and Open Practices as well as Open Educational Resources and MOOCs) and extending the flipped classroom model to university: a specific cloud-based tool to facilitate flipped and situated learning has been designed (IO2);  to equip students with the competences required by the job market, through the creation of Serious Games, based on problemsetting/problem-solving strategies and gamification (IO3);  to provide effective feedback, through an e-assessment method and tool inspired by personalization and context-awareness concepts (IOs 4 and 6).

https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/european-frameworkdigital-competence-educators-digcompedu 10

10 | P a g e

Before all ELSE - Intellectual Output 1, and scoping the horizon When the consortium first convened, for the first of a series of transnational meetings, one thing all partners agreed on: that the transforming potential of some of the core principles of the Bologna process had not been fully applied or harnessed throughout Europe. This being the case, and the playing field not being level, we agreed that before all partner countries were ready to embrace and take advantage of the pedagogical innovations that the project aimed to develop and disseminate, a certain amount of preparatory work was required. To understand the different circumstances in the field of university education in the national contexts represented by the partners, a preliminary action of the consortium was a survey of the main learning and teaching practices and policies in operation in the partners’ individual institutions, which we took to be representative of the respective, wider national HEI contexts. This survey of the European context has been captured in Intellectual Output 1 (IO1), the results of which can be accessed from www.elseproject.eu. As well as considering the individual governance and regulatory policies in the partner countries, the survey reveals attitudes, preparedness, and actual actions towards improving learning and teaching. We wanted to understand whether pedagogical innovation is left to the individual initiative of the more eager amongst the academic staff, or is an integral part of the universities’ vision, incentivised by a strategic institutional commitment to pedagogical enhancement. We wanted to compare the institutions’ investment in staff development, the measures in use for assessing the quality of the provision and the quality of the students’ learning opportunities. And keeping in mind that universities prepare and shape young people’s lives to make them ready for the world of work, we wanted to determine the extent to which European universities fostered their students’ career readiness, and what performance

indicators measured their effectiveness in doing so. So, we considered how the different institutions ensured that students acquired the transferable skills needed to thrive in an uncertain and still unfathomable future world of work; the importance they attributed to links with employers, to bridging the gap between academia and industry; the space they left in curricula for the development of entrepreneurial (as well as critical) skills. IO1 is a mirror held up to (a number of) HE institutions in Europe, showing whether they (are able and willing to) practise what they preach, and how they can enhance the quality of the experience they offer their students through a pervasive strategy of digital literacy.

11 | P a g e

Some reflections on the achievement of the ELSE outcomes - digital strategies and the importance of the human factor The project targeted four main groups: 1. University teaching staff – they have profited from professional development based on experiencing innovative methods and improving digital skills; 2. University students – they have benefited from improved teaching, greater involvement in the learning process, enhanced self-reflection, and preparation for the world of work; 3. Learning technologists – they have experienced best practice from various countries and contrived new technological solutions (including issues connected to privacy regulations); 4. Finally, schoolteachers and final year students – they have experienced ELSE methods/outputs through involvement in the experimental phase. We are confident that we have demonstrated that the systematic adoption of competences and learning outcomes as teaching objectives, of student-centred approaches combined with cooperative learning and the use of digital devices inspired by these methods, can not only enhance the quality and relevance of university students’ results, but also facilitate the development of key civic, entrepreneurial and creative competences. Digital literacy is a cornerstone of any Higher Education Institution’s employability strategy, and the development of digital skills is central to all curricula as an enabler of student learning and future professional success. Whilst students are highly literate in the use of technology in some areas – the pervasiveness of social media activity is incontrovertible evidence of this – they are not necessarily aware of (or accustomed to) the role of digital tools as vehicles for learning, or as tools that can give them skills that can have a positive influence on their lives in a wider context. With these considerations in mind, we developed the ELSE tools not solely with our own – the teachers’ – immediate objectives in mind, but also mindful that getting used to learning through digital resources, seeing how they can be used to convey

innovative, customised, impactful learning experiences can inspire students’ own creativity, show them how the power of digital tools can be harnessed to improve communication, collaboration, critical thinking, and problemsolving skills. Much as our lives are mediated by technology, no digital strategy or tool can be used on its own, replacing the “human element” of the learning experience, the contact with the educators. We became even more acutely aware of this during the second year of experimentation of the ELSE tools, which coincided with the global COVID-19 pandemic and its strict prescription for social distancing and remote teaching. Whilst, having access to the ELSE tools, we were able to develop valuable resources to enrich (and enable) the delivery of our courses, we became even more conscious of a number of core issues: a. that technology is at its best when it is an integrative tool and not a wholesale replacement. And that especially in times of coerced isolation and forced solitude, moments of discussion – of debate, exchange of ideas, briefings and debriefings – be they synchronous or asynchronous, had to maintain a substantial role in the design of our sessions. b. that, as the circumstances created by the pandemic made even more clear, away from the institutional premises we cannot conceive of “students” as a homogeneous entity. Remote (teaching and) learning has revealed profound inequalities amongst the student body – in terms of access to equipment, good connectivity and even (albeit to a more limited extent) skills that enable them to engage fully with digitally supported learning, and that can be significantly detrimental to the quality of the experience. We have all radically reviewed our professional practices, built in resilience and flexibility, and used the digital tools we developed to construct the full hybrid experience that COVID demanded, but that has equipped teachers and students alike with skills that will outlive the contingency of the situation.

12 | P a g e

CHAPTER 1 ELSE: The pedagogy and the tools Notwithstanding the predictable differences amongst the partners’ countries, including access to digital technologies and digital capabilities, what emerged was widespread curiosity and appetite for new stimulating, innovative and effective pedagogies that responded to the changed student approaches to learning, whilst taking advantage of technological innovation and its increased availability. To design the most effective new teaching, learning and assessment tools, we started by keeping our eyes close to home: on ourselves as tutors in Higher Education, and on our colleagues from the secondary sector, who prepare the young men and women who eventually progress to University study. But we also kept a close eye on our students and what makes them rise to the challenge of acquiring new knowledge; on our classroom dynamics and what makes them participatory events; on our assessment methodologies and strategies, and what makes them effective tools for evaluation, self-evaluation and continuous improvement. We reached out to harness the expertise in academic development within our institutions, and collaboration with the University Teaching Academy (UTA) at Manchester Metropolitan University, for example, has provided valuable food for thought and pedagogical support. 11

Blended, digitally supported learning, and strategies to implement it

While there is a large number of learning theories, the three main theories of teaching as articulated by Ramsden (2003) are: 1. Teaching as telling (passive) 2. Teaching as organising and facilitating (active) and 3. Teaching as making learning possible (self-directed). The role of the teacher is to help students become autonomous in their learning so that they can continue learning and developing beyond their university studies and throughout life and contribute in positive ways to local communities and society. Therefore, designing and implementing effective teaching strategies that facilitate learning in a supportive environment is important, an environment that progressively develops independence not dependencies and leads progressively to students’ autonomy. There is still a tendency to focus primarily on teaching as telling and teachers in higher education often still use didactic strategies, perhaps because teachers feel more in control then (Frydenberg, 2012). Teaching as telling was perhaps more relevant and needed when the access to and availability of books and resources was limited. Today, in the information age, this is no longer the case. We are flooded with information. However, we still seem to be talking about lectures and seminars, as if learning can or should be compartmentalised, as if information has to be shared through instruction, instruction by the teacher, through telling in class, even if this is

The UTA website, accessible at https://www.uta.mmu.ac.uk// , offers valuable guidance on how to create stimulating learning experiences underpinned by evidence-based practice, and can inspire creative uses of the ELSE tools. 11

13 | P a g e

not the most effective strategy and the best use of class-time. Momentous events that occurred whilst the ELSE project was live further demonstrated that alternative ways of conceiving the process of learning were an imperative. With the global COVID-19 pandemic forcing everyone to self-isolate, digital pedagogies played a central role worldwide in keeping learning going.

Active learning – peer instruction and the flipped classroom We know that learning can happen anytime, anywhere, under any conditions. We also know that students learn best when they are active and immersed in the learning process, when their curiosity is stimulated, when they can ask questions and debate in and outside the classroom, and when they are supported in this process and feel part of a learning community. There are many things students can learn without needing a teacher. We often forget this and cram our programmes full of content, leaving little room for personal exploration and discovery. Content delivery is a term frequently used and a concept that often directs the energy of the teacher in preparing materials so that they can deliver content. It will be important to remember that there is no such thing as content delivery. Knowledge transfer is another term often used. Why would we need universities for this? There is a plethora of resources already out there in a range of formats and media. Many made available as open educational resources (OER), open educational practise (OEP) and open access publications. These enable re-use under specific open licenses and often adaptation too. A question to ask ourselves is if we make use of what has been made available by the wider academic community and how we can utilise some of it for our own practice.

We are all immersed in the visual world and often watch, share and create videos not just for entertainment but also for learning and development. Technological advancements have enabled us to not just to consume but also to become digital creators, makers and share our artefacts openly on the web. Through such activities we connect, often via social media. We connect ideas and we connect with each other (Gauntlett, 2018). A review of conceptual and empirical pedagogical frameworks conducted by Nerantzi (2017) that have been developed to enable learning and teaching supported by digital technologies in a range in blended, fully online and open settings, revealed the following four key characteristics: 1. activities 2. choice 3. facilitator support 4. community. Recognising the importance of these four characteristics will help teachers redirect their attention from content delivery to planning more interactive and active learning experiences for their students. Creating seamless learning opportunities that connect learning pre- during and post-class supported by technology will be useful to consider (Ehlers, 2020). Active learning enabled through activities and interaction, peer learning and teacher’s presence and participation in the learning process that is inclusive and recognises that diversity can enrich the learning experience as well as feedback and challenge has been recognised as good practice (Chickering & Gamson, 1987; Gibbs, 2010; Gibbs, 2012). Focusing on what the students do will help teachers think about how they can help their students engage and learn. The teacher’s effort on creating active learning approaches and activities, such as inquiry-based learning strategies (Problem-Based Learning, Project14 | P a g e

Based Learning, the flipped classroom) on own and with others (peer instruction, cooperative, collaborative and team-based learning) will be valuable in this process. 12 Peer instruction and flipped learning are explored here as strategies to consider to scaffold learning and create active learning opportunities within and beyond the physical and online classroom. Both approaches can work in campus-based, blended and fully online programme modes. Where the classroom is mentioned below, in fully online programmes or modules, this could be seen as time spent with the teacher, synchronously or asynchronously. Peer instruction (Mazur, 1997) is an active learning strategy that scaffolds learning, promotes deeper engagement, and fosters conceptual understanding through instruction outside the classroom, peer-to-peer interaction, testing using digital technologies and discussion of answers (Mazur, 1997; Schell & Mazur, 2015; Schell & Butler, 2018). 13 In this concept, the teacher provides readings and instructs students to access and engage with them before coming to class, thus helping them prepare for the in-class activities (Schell & Butler, 2018). Students the work together to make sense of questions provided in the form of a quiz and articulate a response to demonstrate their understanding linked to a particular concept. The teacher collects

responses and provides feedback and through this process identifies any common difficulties students have and designs the session around these so that they can help students to overcome any challenges, make sense of difficult concepts and move forward in their learning (Schell & Mazur, 2015). Similarly to peer instruction, flipped learning too moves instruction outside the classroom, mainly through the use of video resources created by the teacher (Tucker, 2012), but also through a range of other activities, such as serious games. 14 The strategy of flipping the sessions, frontloading the “content part” through activities done outside the classroom, means that the live events with the tutor can be used for collaboration, discussion, problem-solving and application of the learning to case studies in class. Moreover, flipping the class helps the teacher provide more targeted support, identifying the aspects of the lesson that students find more difficult to understand and making them clearer, but also encouraging students to take ownership of their learning, to explore particular aspects of the subject being studied, and leading them to a better understanding of their learning styles and tastes. As Tucker (2012) emphasises, using a video accompanied by a simple instruction to students to “watch it”, or a game with the

While there is a recognition that learning is messy and the process complex, Bloom (1956) attempted to make order out of chaos and defined the cognitive, affective, and kinaesthetic domains of learning and created a taxonomy for the development of higher order thinking skills. His taxonomy is widely used across the education sector, especially where outcomes-based education models are used and can help us scaffold learning. 13 Mazur (1997) is known for having developed peer instruction within physics education at Harvard University in the early 90s, when he realised that transmission of information in the classroom through lectures did not help his students learn and engage more deeply with specific physics concepts. In the 1960s Problem-Based Learning was developed to respond unsatisfactory results in Medical Education at McMaster University in Canada and led to a shift of teaching approaches as telling and memorisation towards inquiry and problemsolving (Barrows & Tamblyn, 1980). 14 The concept of flipped learning was developed by Bergmann and Sams (2012) in 2008 in the context of teaching chemistry in a High School in Colorado to respond to the challenge of having to go through previously delivered content that students had missed, being absent from class. Their strategy was to record their lessons and make them available online. They noticed that students started using them for revising classroom sessions and developed the flipped learning approach that can also be found across higher education practices. 12

15 | P a g e

simple instruction to “play it” is not enough to make flipped learning effective. Videos and games must become an integral component of the learning process, accompanied by carefully designed activities that are clearly connected to their content, and that offer opportunities for inquiry and personalisation of learning. Watching must not become a passive task, and playing a serious game has to be the first step to a reflection on the experience. 15 While flipped learning has the potential to engage the student outside class, the opportunities for active and deeper engagement in the learning process that the methodology presents are not exploited in full when the resources and activities are exclusively designed and prepared by the teacher. Therefore, teachers should move away from the concept of students as consumers, and instead consider engaging students independently or in peer groups in the creation or editing of the resources used in flipped learning. By doing this, students can widen and deepen their engagement with the subject. Such an example comes from a study by Nerantzi and Hannaford (2016) in the area of academic development where flipped learning was used with action learning sets tasked to engage in inquiry into specific learning theories through editing particular Wikipedia pages ahead of class. In class the learning theories were discussed and debated that helped students deepen their understanding further and also identify any aspects of the learning theories they struggled with. The results showed that students enjoyed the authentic nature of the task, felt that they were learning and could support each other in their learning and when faced with technological challenges. Class-time was used

to discuss and clarify specific aspects of learning theories that students found difficult to understand. When action learning sets did not work well together, this seems to have a negative impact on engagement and peer learning. What to consider when planning for flipped learning • What you could stop doing. What hasn’t worked? • What is your rationale for using the flipped learning approach? What are the anticipated benefits for your students? • Are you and your students familiar with flipped learning? • Do your students all have access to the required technology? • What do you want your students to achieve? • Think about what will happen before, during and after the session. • What video resources are already available that you can use. Are any Open Educational Resources? • What questions can be linked to the resources you provide in advance? • Could you ask your students to locate/create a video resource to be used? • Could your students work in small groups or in pairs to prepare a video and/or specific questions associated with the topic? • How will you make sure that the students engage with the resources in advance of the class? • Are the videos you use/create inclusive? Think about accessibility. Have you provided the transcript?

Findlay-Thompson and Mombourguette (2014) highlight the importance for the teacher and students to be familiar with flipped learning in order to benefit the most from it and that judgment is required from the teacher to identify if flipped learning would be the most suitable approach to use in a particular situation. Students also need to understand the rationale for using flipped learning and see how it will help them in their learning. 15

16 | P a g e

The ELSE tools to effectively support blended, digitally enhanced learning Keen as all tutors are to devise the most ground-breaking and the most effective teaching method, we are all limited by the time we have available, and by the different digital capabilities and levels of staff development across Europe in this area. Any new development needs a large investment, and often experimenting with didactics leads to very little that can be defined as truly revolutionary or immediately life changing unless we persevere, invest time and effort, try out and discard or adjust – but in the meanwhile there are syllabi to complete, set assessments to administer, marking to turn around, and so forth. The temptation to stay with the tried and tested is compelling. An additional problem could be that strategies to flip the classroom – whatever the digital tools used to deliver them – could end up being far from revolutionary, but instead replicate old teaching styles – learn the content through tasks to be completed at home / come to class to check your homework – except with a new name. What the partners wanted to achieve was a blended, flipped learning environment into a fully digital ecosystem. Our response to this all too familiar predicament is to design a set of tools that: • are adaptive and flexible – they can be scaled to the level of the class, and adapted for a range of disciplines (although we trialled them in humanities subjects) and languages of instruction; • are intuitive and reproductible – we offer exemplars of sessions that clearly show the underlying principles, and


how they relate to specific learning outcomes; do not require complex technology, such as hardware beyond what is standard issue in most schools and universities and software that is not readily available to download for free; together form an ecosystem supporting an integrated system for teaching, learning and assessment.

In all our institutions, as in most learning establishments, various forms of VLEs were in use. In designing a set of tools to facilitate teaching and learning activities supported by a virtual learning environment, the team took Moodle as the platform of reference. It then set as its priority objective the design of a learning experience that also helped students attain and develop a set of transferrable skills – such as resilience, critical and lateral thinking, time management – that are fundamental to making them ready for the challenges of the 21st century job market. We recognise that it is essential that students’ career readiness and employability grow alongside the acquisition of subject knowledge, and the activities that can be set up with our tools contribute to this synergy between knowledge and skills. The ELSE project developed three tools: ECORE, EVOLI and EDASH. •

ECORE is a simulation tool used to design Serious Games that convey subject content, stimulate reflection in students, and promote experiential learning, that is learning by doing. 16 The importance of this pedagogical approach lies in its close resemblance to real world situations, where everything is connected, and where decisions made in one stage have an impact on what comes next.

The father of experiential learning is considered to be John Dewey. See Dewey, J. (1938).

17 | P a g e

Starting from a template that clarifies the overall intended learning outcomes of the exercise and identifies a number of steps through which to achieve them, tutors can use the ECORE editor to create a story narrated through a succession of frames. Each frame presents a scenario with a number of options for students to choose from. Each option carries a number of points, and each choice made by students can be supplemented by additional materials (links to further readings, additional observations), exploiting the pedagogical value of errors, and transforming the assessment in a learning opportunity.

classroom analytics are a valuable instrument to get a proper pulse of the class, and one that helps design targeted interventions when needed, students can also benefit from the opportunity to see their own levels of performance and engagement represented in an objective way. A brief account of the rationale supporting each of the tools is provided in the following chapters, together with some examples of how each of them was used in partner institutions in a range of very different disciplinary areas. The examples show the versatility of the tools and will hopefully inspire tutors to imagine further uses in similar or different subject contexts.

EVOLI is a video tagging tool that can be exploited to support the preliminary stage of the flipped classroom. It uses knowledge clips made available in YouTube videos, which students are encouraged to engage with and respond to in advance of the class, indicating what aspects of the topic presented in the video were clear, what needed further clarification, and any observations that the students wanted to share with tutor and peers. EVOLI makes the students’ responses to the videos available to the tutors, who are able to design wholly customised live sessions, based on the real requirements of the class.

EDASH is an assessment tool that tutors and students can use to monitor engagement, performance, and progress. Whilst tutors universally agree that appropriately presented

18 | P a g e

CHAPTER 2 ECORE – Serious Games as an interdisciplinary pedagogical tool for experiential learning One of the strategies to support the flipped classroom mode is to expose students to experiential learning – a process of learning by doing that enables them to immerse themselves in real-life scenarios, try out actions and observe their outcomes, reflect onand in-action and get constructive feedback, all in a (relatively) risk-free environment. By inviting the activation of a “pattern of inquiry” from the outset, experiential methodologies encourage students to think not only after the experience but “throughout the experience” (Wurdinger, 2005): to examine the various facets of a problem, develop a plan to address it, test their plan against the reality of the experience, and apply what they have learned to create a solution. The ECORE editor for serious games, developed by ELSE partner EntropyKN was used to create a range of activities that were tested in the partners’ institutions, in a wide range of predominantly Humanities courses. Serious games have an important role in pedagogy, where they can be used to engage students in challenging decision-making processes, often in contentious contexts. Some disciplines appear to lend themselves more readily to such a tool: in business studies, for example, serious games can be used to challenge students with complex scenarios – e.g., involving tax avoidance, or tax evasion – and make them work through the most appropriate and effective course of action (Robben et al., 1990). But, as the ELSE partners realized through their experimentation, games can also be used to convey philosophical concepts, such as the intellectual stance that a translator has to maintain when accompanying

a text from one language into another; or to understand the Weltanschauung of ancient Greece and Rome, that inspired the attitudes to the foreign guest in the epic poems of the classical tradition. A well-designed serious game will make the student/player think, challenge their own beliefs, test out possible solutions to problems and understand their relative values and effectiveness. It can translate complex concepts into a simplified, self-contained model that enables tutors to bridge theories taught in class with real life experiences and enables students to make decisions in a relatively “safe” environment, devoid of real risks. It will enable students to learn through a sequential process of cause and effect and learning by doing (Abdullah et al., [2013]), and lead to increased participation and personal investment in the learning process (Farrell, 2005). ELSE partner EntropyKN, a small Italian company specialising in technology-enhanced learning, was tasked with developing a tool for creating Serious Games. The ELSE partners had a twofold objective: to realise virtual learning objects and environments that were flexible – that is to say adoptable and adaptable to different disciplines and different local contexts; and to equip university tutors – who are equally from different disciplines and different cultural contexts, and unlikely to be game designers – with sufficient technical competencies to be able to use said objects and environments in their practice. The end result was the ELSE ECORE editor. The players in our experimentation were invited to run their own simulated firm, make strategic decisions on investment 19 | P a g e

opportunities, or test the processes involved in translating text between languages. They were offered engaging, realistic, and memorable learning experiences that encourage them to

reflect about the outcomes of each of their decisions, as well as about the overall process that led to the overall outcome.

HOW TO DESIGN A GAME USING ECORE The ECORE editor is intuitive, flexible, easy-touse and easy-to-customise. It allows users to select realistic characters and scenarios to create a story, to upload supplementary files in different formats (pdf, word, videos, Web pages), and to provide numeric and qualitative feedback to learners. It is also a valuable tool for the co-design of elements of units, encouraging the involvement of students in the construction of their teaching and learning journey. ECORE can be integrated with any training platform or tool through the use of LTI-Learning Tools Interoperability communication protocols.

The idea of the ECORE serious games is that they enable tutors to create activities that challenge and engage students and help them learn something about a topic. The indispensable preliminary step, before any game is designed, is to establish what the purpose of the game is: what it aims to achieve, what competences students need to possess before they attempt it, what the intended learning outcomes are, and how the learning gain will be measured. To help articulate the pedagogical underpinning of the serious game, and orient the story narrated through the game, it is useful to map out the general principles at the outset.

The editor can be accessed at https://else.entropylearningplatform.it/demoeditor/ The tables below help visualise these, working backwards from the intended outcomes: 20 | P a g e

1. START FROM THE INTENDED LEARNING OUTCOMES The learning outcomes (what the students will have learnt) The Assessment (how the competences acquired can be measured) The competences and skills developed through the game 2. DEFINE THE PROCESS What is the student’s goal in the game? What are the priorities? What to consider in developing a strategy Define the most effective strategy to get to the final goal. The riskiest strategy The least effective strategy 3. PROVIDE SOME GENERAL INFORMATION ON THE GAME TITLE DESCRIPTION PLAYER’S GOAL INTENDED LEARNING OUTCOMES Here is an example of a game designed to enable students to experiment with point of view, to understand how it is mobilised in literature, and to stretch their intercultural awareness in the process. It was created for students on a unit called Understanding the Style of Literary Texts, which is offered to 2nd year undergraduates in a degree in Modern Languages and Linguistics, in a UK university.

21 | P a g e


The aims the game intends to achieve (what the students will have learnt) Learning outcomes

Assessment How the competences acquired can be measured

The competences and skills developed through the game

UNDERSTAND HOW POINT OF VIEW OPERATES IN A NARRATIVE: identify the rhetorical strategies deployed by the narrator to achieve a set effect/provoke a certain response in the reader critically appraise the centrality of point of view and how it can be manipulated to produce meaning and a response. identify the cultural significance of choices made by characters, and assess the difference with their own system of values and beliefs Critical thinking, reflection on appropriateness of certain choices in relation to the cultural context in which they operate Being able to detect and analyse the choices made by the narrator/author to make sense of these choices, in terms of the message they convey

2. DEFINE THE PROCESS What is the student’s goal?

Ensure that the protagonist achieves the desired outcome – returning home after two decades - with the fewest possible obstacles or setbacks.

What are the priorities?

Understand the rationale behind each turn of the plot (=hero's choices) so that the ultimate goal can be achieved

Understand and align with the hero's desired outcome; Understand and apply the choices that will get the hero closer to that outcome. This means: What to consider, in developing a strategy a. understanding the context and the moves made by other characters; b. reflecting on the cultural implications of each decision made.

22 | P a g e

Define the most effective strategy

remain aware of the historical and cultural environment in which the hero operates and make choices that are historically appropriate

The riskiest strategy

keep eyes fixed on the desired outcome and pay insufficient attention to the steps required to get there.

The least effective strategy

Use an ahistorical set of values to inform decisions

Step 3 completes the definition of the overall structure and goals of the activity.

23 | P a g e

Once the fundamentals (the pedagogical requirements, the learning outcomes, and the assessment strategies) are established, the steps to create a new game using the ECORE editor are outlined below:

After you have completed all the steps for your game, you can prepare the feedback comments you want students to see:

24 | P a g e

25 | P a g e

26 | P a g e

Insights enable tutors to get an overview of student performance in a game.

27 | P a g e

The tables in the following pages show some examples of the process that tutors followed to structure the Serious Games that were created during the project, comments about the use of the games in class (including ideas for follow up activities, and suggestions for future use).

28 | P a g e

HOW WAS ECORE USED BY THE PROJECT PARTNERS? The ECORE editor was used to create serious games that were tested on students studying languages, literature, translation, business studies, accounting and finance. Below are a few examples of the games that were created, with a few reflections from the tutors. The demo game, OTHERS, can be accessed at https://else.entropylearningplatform.it/demogame/

Application 1: ECORE in the Translation class: Moving House (Author: Alba Graziano, Università della Tuscia, IT) Description Moving House is a serious game that puts the translation student in the role of a musician who is looking for a place in a band much larger than any he has ever played in. As part of the process of pursuing this professional move upwards, the musician is faced with a number of situations and choices related to his personal circumstances, represented in the game by the experience of moving to a new house: changing city, context, adjusting to a novel set of situations. The musician’s goal in the game is to persuade the audition panel to take him on. The panel’s decision is also influenced by the skill set that the musician can demonstrate during the game: decisionmaking, organizational and relational skills, flexibility, adaptability and a little improvisation, too.

• •

Purpose of the game Through playing the game, students should: understand a translator’s attitude and behaviour, and understand the object of translation. Observations The underscoring principle of the game – which is discussed with the students – is that translation is about transforming a text written in a language (and culture) into ‘the same-but-not-quite’ text in

a different language (and culture): just like moving furniture, adapting it to the new environment, or re-purposing it where possible, or discarding it altogether and replacing for something more suitable! So, adaptation is suggested as a good synonymic keyword to all kinds of translation.

   

Process Before introducing the game, the same objectives were pursued through a session on critical thinking, brief research, and class debate. Besides the game and subsequent debrief, the following is also available on Moodle: two discussion forums on related keywords and topics; a pdf outlining the pedagogical rationale of the serious game; an audio-lecture of one hour to recap the outcomes of this introductory chapter; a spidergram to support the audio-lecture and serve as model for the student task. The follow-up activities include the debriefing phase. During in-person classes this would consist of a discussion of the steps in the SG plot, the options given and their significance. Once the fundamental points of translator competences and translation object have been agreed, a series of brief research topics, keywords and suggestions are shared with students, to be developed through collaborative or individual work. The students’ research/opinions/reflections were collated in two discussion forum threads and later commented on by the tutor in an audio-lecture. 29 | P a g e

Moving House – here is how the serious game was announced to students in Moodle

Screenshots from the game

30 | P a g e

31 | P a g e

Application 2: ECORE in the Teacher Training class: Un inizio difficile (A difficult start, Author: Patrizia Sibi, Università della Tuscia, IT) Description The game features three different teacher behaviours reflecting ‘accepted’ modes within the current school environment – although implying very different soft skills levels. These are: the regulatory teacher: focussed on organisation and procedures, with a rigorous approach towards the syllabus, reliable as s/he executes the principal and the school bodies’ decisions, capable of a good relationship with students, provided that rules, timings, deliveries and commitments are followed and met. The empathetic teacher: always attentive towards the students’ needs, difficulties, and issues, s/he tries to form alliances and establish agreements with students in order to reach her teaching objectives, follow the syllabus schedule and meet the organisation’s expectations. The divergent teacher: skilled at effective communication, purposeful and proactive towards the organisation and the school community, s/he uses contextual data and contingent situation data to adopt strategies and behavioural postures to reach objectives. About the course The game was designed to be delivered on a Training Course for Special Needs Teachers. The general aim was to mobilize and further develop essential soft skills, such as relational and communicative abilities, cooperative abilities and the ability to work in a team, conflict mediation, listening and data inference abilities in a specific context, pro-activeness in difficult situations as well as the awareness of one’s role. The game also had the function of promoting knowledge and practice of a digital tool designed for experiential learning through problem-solving and role-playing activities. Italian teachers rarely use this type of tool to improve their students' learning.

• • •

Purpose of the game The game presents players with a typical school situation. A young, newly qualified teacher of Italian, who is spending her probation year in a high school, is tasked with solving a complex range of issues: students are complaining about the introduction of a new and strict set of school rules senior colleagues are being difficult with her and want to put her to test some disgruntled parents are upset at the disruption that the protests are bringing to their children’s learning the newly-appointed and uncompromising principal is mainly concern to demonstrate how good he is at managing a school. Observations The highest scores are given to the divergent teacher profile. However, many other answer combinations produce a positive outcome with points awarded which fall in the ‘winner’ range, thus opening up to debate among teachers. The game’s timeline is 15 days, and the story has 5 ‘sub-situations’: teacher vs. principal, teacher vs. students, teacher vs. tutor teacher, teacher vs. colleagues and teacher vs. parents. By the end of the first two weeks of school, the newly appointed teacher will have to gain her principal and her colleagues’ respect and appreciation whilst also passing her probation year with honours. The situations are realistic. They are deliberately exaggerated to make the possibility of wrong answers more acceptable to the players. The game can be played individually, and it will provide a personal performance evaluation. However, it is also a situated learning tool to reflect on one’s own didactic practice cooperatively.

32 | P a g e

Follow-up activities The de-briefing phase was carried out through an online questionnaire:

1. Have you played any other digital training games (Serious Games, role-playing games, simulation games and so on)? 2. Do you think that training in a simulation environment is useful in order to develop competences and abilities to apply in the real world? 3. Which abilities/competences can be developed by playing Un inizio difficile? 4. Which situations seemed unreal or not very useful to you? 5. Do you agree with the feedback you received? 6. Do you think you could improve your score by playing once again or more?

 

the game guarantees the teacher-player’s involvement, as it allows for a simulated performance that is protected from external scrutiny and emotional-relational dynamics; the game has good internal consistency levels between experiential situations presented and soft skills that need activating/developing (see item 3); the game offers a functional experiential environment for the learning by doing approach; the game needs a classroom debriefing integrated session, with the presence of the facilitator, in order to: clarify and process the emotional reactions produced by the game outcomes; analyse and share the meaning of the problem situations offered by the SG; think and exchange views about the problem-solving solutions adopted; evaluate learning and competence transferability in a real context.

The analysis of the answers suggests the following in respect of the use of Un inizio difficile in teacher training programmes:

33 | P a g e

screenshots of the main scenes The principal summons the young teacher of Italian and asks her to help him solve the complicated situation with the class that is protesting about the new school rules.

Protesting students refuse to carry out any teaching activity so long as the rules remain in place.

The teacher of Italian is assigned a mentor – as is usual in the probation year – to support her and monitor her teaching activities.

34 | P a g e

The teachers’ council also counts on the teacher of Italian to solve the problem.

The parents, summoned a few days after the start of school, are irritated and not very confident in the teacher.

15 days into the new school year, the principal asks the teacher to report on the solutions adopted.

Will the teacher be able to gain her principal and her colleagues’ respect?

35 | P a g e

Application 3: ECORE in the Auditing Course (Author: Nieves Gómez Aguilar, Universidad de Cadiz, ES) Description This game is an attractive way to review core subject concepts apply them in real time. It puts the student in the position of having to give quick answers to the demands of the audit partner, so he/she understands the pressure at work and commitment to quality About the course students learn to deal with the individual stages of the auditing process. They learn to work under pressure and to appreciate the centrality of thoroughness and quality. Purpose of the game This game allows students to put themselves in realistic situation, where they must get

information about an audit engagement in the best possible time but ensuring that this information is useful and appropriate for the audit partner. The game is intended for the end of the course because it allows students to review all the concepts in a fun and enjoyable way. Observations Due to COVID-19, the class was delivered online. While the students were playing the game, the teacher was in a virtual room guiding them and collecting their impressions. The students were encouraged to repeat the game as many time as were needed to achieve top score – this is because repetition led to understanding. At the end of the game, the students filled in a questionnaire about the tool, its usefulness and clarity.


36 | P a g e




37 | P a g e



38 | P a g e

Application 4: ECORE in the Literary Translation class: Getting It (Author: Aba-Carina Pârlog, West University of Timişoara, RO) Description The game is presented as a storyboard illustrating the stages of the translation of a text. Students are presented with a series of lexical, structural, and stylistic choices. About the course This is a course on literary translation, addressed to MA students. I start with a storyboard on general translation issues and continue with a set of literary texts that students must translate either from Romanian into English or vice versa, bearing in mind what they learnt throughout the semester. Purpose of the game The game is used in the context of a revision class. The Theory and Practice of Translation and the storyboard was created as a relatively easy exercise so that the rationale may be more easily grasped by students regardless of their level of translation knowledge. Observations The storyboard produced focuses on ways of tackling text translation with an emphasis on methods of translation, on the slight differences between the connotations of a word, on intratextual and cultural translation. The game is intended to be played from start to finish, and

every interruption wipes up the points achieved previously. However, students can interrupt the game for the time required to understand the implications of their choices Still, the exercise can be stopped at each step so that the avatar’s input and options given may be clarified, if necessary.

Follow up activities Student feedback was sought through a questionnaire at the end of the class in order to clarify how useful, interesting, easy, or clear the exercise was for them. They also gave answers regarding their overall score and their satisfaction with it. The theory tested in the exercise was applied to the texts given for translation. The problems were raised again so that they may be clear to everyone in the class. Most students enjoyed solving the ECORE game and understood its purpose as a digital learning tool.

39 | P a g e

CHAPTER 3 EVOLI – video tagging to support the flipped learning experience Recent studies find that undergraduate students in classes with traditional stand-anddeliver lectures are 1.5 times more likely to fail than students in classes that use more stimulating, so-called interactive learning methods- see for example the study by Freeman et al on STEM students (Freeman, Eddy et al, 2014). Many tutors want to make a more efficient use of contact time, but they also want to increase their students’ mastery of conceptual reasoning and levels of self-reliance and self- esteem. This means cutting down on the time spent introducing and explaining core concepts whilst ensuring that all students reach a similar level of understanding and are able to progress and do well in the unit. It also means establishing an interactive relationship with students that makes them active and confident participants in the learning experience. The EVOLI video-tagging tool, developed at the Politecnico di Milano, helps achieve this, marrying easy technology with the flipped classroom model. It is designed to enhance the use of educational videos in online courses, specifically but not exclusively within a flipped-classroom approach, and to facilitate the communication between teachers and students, by collecting students' feedback on the videos. EVOLI is an easy to use stand-alone web-app, 17 compatible with VLE systems and integrated in MOODLE that enables tutors and students to work with YouTube videoclips that deliver subject content, in advance of a taught session. As such, it is an effective tool to support flipped classroom methodology. 17

The main feature of EVOLI is that it allows students to “interact” with the videoclips provided by the tutors by tagging them – indicating what parts/concepts that they found difficult, what parts were clear, and what questions they would like to pose in the follow up session. By examining the students’ responses to the videoclips, the tutor can therefore prepare the follow up session with a clear understanding of what concepts need to be explained further, which aspects need to be clarified to students, and which instead were unproblematic. The analytics feature available in EVOLI also enables the tutor to identify trends and clusters – for example the percentage of students that found specific points/ concepts difficult to grasp, or groups of similar questions.

What EVOLI looks like, how it works and how it facilitates the flipped model Watch a video demonstration of EVOLI at ELSE_Evoli .

Written in: html, javascript, firebase for the database, flutter as backend javascript.

40 | P a g e

The EVOLI app can be accessed at https://www.evoli.polimi.it. On access, the welcome screen offers the option to log in as a tutor or as a student:

How to use EVOLI as a tutor 1. CREATE AN ACCOUNT On first accessing the tool, tutors create an account which will be saved for future use in their browsers:

41 | P a g e

2. CREATE /USE VIDEO CLIPS As soon as you are in the tool, you are asked to either a. upload a new video from YouTube, or b. use a video you have already uploaded.

a. Uploading




You can: a. Upload relevant YouTube videos; or b. Record your own video, and upload it to EVOLI. Give it a title that you and your students will recognise, for ease of access. Save it to a specific folder

b. Using a video that you had previously uploaded The videos you have uploaded will appear listed here, with the title you have given them, the code that EVOLI has assigned to them (and that you will give your students to access), any notes you want to add, and details of statistics on use. More about the latter, a very useful function, later.


42 | P a g e

To use the video in your flipped classroom, you share it with students in advance of the live class. a. Identify the video you want to share, click on “copy” to grab the code

b. Then paste it in your chosen area of your VLE page, with instructions to students:

43 | P a g e

How to use EVOLI as a student 1. ACCESS EVOLI THROUGH THE URL PROVIDED – which they will save to favourites for subsequent access – and log in as STUDENT:


3. Watch and tag the video:

44 | P a g e



How the data from the video viewing exercise support the live session The tutor can analyse the students’ responses to the set video clip to prepare the follow up class. EVOLI offers useful statistics on use, which provide an indication of what students found clear (and possibly requires no further elaboration), what they require more help with, and any comments, observations, or further questions they have. Tutors can access the statistics on number of viewings and student reactions by clicking on the Stats button.

45 | P a g e

As all feedback and comments are connected to a specific time in the video, tutors have a clear idea of what particular points in the videos attracted what reaction, and can plan the session accordingly:

EVOLI IN PRACTICE – and some evaluation The following pages present some practical examples of how the EVOLI tool has been used to support the flipped classroom experience.

46 | P a g e

Application 1: EVOLI in a BEGINNERS’ LATIN CLASS (Author: Nicoletta Di Ciolla, Manchester Metropolitan University, UK) Description The class had been reading an excerpt from a famous Phaedrus fable that presented instances of relative pronouns. Purpose and rationale for use 1. efficiency – less class time devoted to explanation of grammar/syntax means more time for practice and access to authentic materials 2. inclusivity – enabling all students to learn at their own pace 3. “personalised” T&L – focussing on the aspects that really mattered (interesting to students, aligned with the unit’s Learning Outcomes) and customised scaffolding.

Twenty-four hours before the class, the tutor checked the statistics on use – to customise the live session. The graph looked like this:

Observations The tutor noticed that: all students were able to translate the story intuitively, with a partial understanding of the function and meaning of those “words” in the Latin context. all students found the concept of “relative pronoun” complex, even when presented with actual examples of use from English. Process In preparation for the subsequent week – and for more reading about Roman wisdom – students were asked to watch a short “knowledge clip” on relative pronouns in Latin. The clip is available on YouTube (https://magistrafischer.weebly.com/relativepronouns.html), although students were only provided with the EVOLI code. They all received the following message on Moodle:

The peaks on the above graph show the following:  The “I get it” happen at the moments when Latin sentences are presented – confirming that the students are able to translate this level of Latin intuitively;  The “I don’t get it” happen when the theory is introduced – confirming that students struggle to understand concepts., and to apply deductive reasoning The prominent spike at minute 9 corresponded to the grammar recap – which was a little worrying, given that the clip was mostly explaining grammar rules.

47 | P a g e

The follow up activities The tutor decided, for the live session, to go for a hands-on approach, where the rule (enabling students to understand general mechanisms) was not presented as an abstraction, but as an obvious conclusion derived from the evidence.

The picture below highlights the rationale for the live class, showing (left to right) the tutor’s plan for the steps to follow in the session

The experiment with EVOLI in the Latin class suggested a number of considerations, which were shared with colleagues and students:

48 | P a g e

49 | P a g e

Application 2: EVOLI in the Translation class (Author: Alba Graziano, Università della Tuscia, IT) Description Translating: from theory to practice is a course provides students with the main notions to analyse English textualities for specific communicative purposes. It introduces translation theories but mainly it is aimed at practicing the translation of some genres and registers, e.g., newspaper articles, scientific/academic essays, info-marketing texts, and audio-visuals. The final exam is a project work including text analysis and translation and oral discussion of the translation choices. A number of very varied language activities and exercises are included in the moodle course: they are preparatory for the exam project and thus compulsory. Journalistic translation is also practiced and commented upon during classes. About the course The debate around the (un)translatability of languages, where the reason WHY we need to translate is approached from the biblical foundation myths and proceeding to a brief survey of the main related linguistic theories of the modern age about the ideal of the perfect language (Bacon vs. Locke) and the debates on languages’ culture-boundedness (Saussure’s radical arbitrariness) as opposed to linguistic universals (Chomsky’s generative linguistics and post-Chomskyan cognitive linguistics). Roman Jakobson’s contribution to a communicative/pragmatic approach to translation: from translating languages to translating texts. The texts read and commented were - Genesis 2 ad 11 - F. Bacon, “The Idols of the Market” - J. Locke, Of words, chaps in B. III of Essay on Human Understanding

- The strange case of the Opi language (SapirWhorf vs. Gibbs) - R. Jakobson, “On Linguistic Aspects of Translation”. EVOLI was used in the following two sessions - from Unit 1 to 2: a transition from the inductive learning stage to the more theoretical one in order to get an idea of the modern branch of Translation Studies and its vastity. - at the end of Unit 2: after using written texts as theoretical inputs, in order to expand the concept of equivalence met in Jakobson’s essay. Rationale for selecting specific clips The three clips selected were amongst the few available on the topic on YouTube that are suitable for an academic audience. The lack of useful and adequate videos online is a big problem. I would not like to resort to my own video lectures since they might be too easy or in case they might be used as debriefing. 1. Jeremy Munday’s presentation of his own manual Introducing Translation Studies (included in the course bibliography): https://www.youtube.com/watch?v=iffkVwa9lno &feature=youtu.be 2. Anthony Pym’s conference on theories of natural equivalence as intrinsic to linguistic systems: https://www.youtube.com/watch?v=_G5oAMWf ObI&feature=youtu.be 3. Anthony Pym’s conference, which emphasizes equivalence as the fruit of meaning negotiation and translator's choice and as irreversible: https://www.youtube.com/watch?v=ZP9PcjIMkV U&feature=youtu.be

50 | P a g e

• •

Follow-up activities The planned follow-up activities and debriefing sessions (originally scheduled to take place once the viewing of the three videos had been completed) had to be cancelled because of the changes in delivery forced by the CoViD-19 lockdown. Consequently, too few students completed the work required by this activity, and it was not considered useful to set up a ZOOM session to discuss it. There may be a series of reasons why these Ss did not particularly enjoy the videos (possibly not the tool): the videos were not particularly interesting. Number of ESL students with general English competence who were not up to the standard, particularly for Pym’s videos (a couple of comments confirm it is the language, not the concepts, that they don’t understand). Unless supported by really challenging contents and a clear learning finality, the tool can appear as quite mechanical.

from home, so they must have done it uninterruptedly. One big lesson learnt from this exercise is that the tool works, and works well, only if incorporated into a fully blended teaching plan, with pre- and post- activity exchanges with the tutor. Provided one manages to find more adequate videos on the specific course topic, it will be worth trying to use EVOLI again in a blended learning environment which would realize the flipped class approach more consistently. Yet, for language courses there remains a problem: that is an overlap of content and language understanding. It would be useful to try to combine the tool with a CLIL/EMI approach.

Observations Students watch and tag the clip (viewing in one go). In the first pilot they had to do it individually

51 | P a g e

Application 3: EVOLI and the Translation class – dealing with Equivalence Problems (Author: Aba-Carina Pârlog, West University of Timişoara, RO) Description This session introduces the concept of equivalence based on references to Mona Baker and Anthony Pym and the tutor’s own observations on the topic. It focuses on learning the tutor’s method for error analysis. Purpose and Rationale for use We discuss the issue of equivalence, Mona Baker’s classification, the tutor’s own classification and various cases of equivalence vs non-equivalence errors. The assessment strategy for this unit is an instant translation of sample sentences, identification and correction of wrong translations, and error analysis.

Follow-up activities The analysis of the EVOLI data enables the tutor to prepare a bespoke live session, by highlighting

the areas that need to be explained, explored, and tested with the class (see below).

The YouTube clip selected contains part of a speech delivered by Anthony Pym who explains the concept of equivalence by making links to Jean-Paul Vinay and Jean Darbelnet, and Eugene Nida. Pym uses examples to support the theories presented. URL: https://www.youtube.com/watch?v=_G5oAMWf ObI&feature=youtu.be Observations Students watched the video without interruption. If the viewing had been part of a classroom activity, the tutor would have provided intercalated exercises, for example to go over parts of the theories that the speaker refers to, or to clarify there and then parts that students found difficult. In the live sessions the students are presented with examples of bad translations, which the tutor divided into categories. Referring to Pym’s theory, students are asked to analyse the translations, identify any errors, and suggest better alternatives. 52 | P a g e

Application 4: EVOLI and the Stylistics class (Author: Nicoletta Di Ciolla, Manchester Metropolitan University, UK) Description This session introduces the concept of point of view, with reference to the theories formulated by Genette, Stanzel, Leech and Short, Simpson. Purpose and Rationale for use The course, Stylistics in narrative texts, introduces elements of narratology and literary linguistics, through a set of theories and the attendant tools that students apply to the interpretation of a narrative text. The assessment is a stylistic analysis of an excerpt from a selection provided by the tutor.

• •

Process In the absence of clips freely available, the tutor recorded her own. In preparing the clip, the tutor: started with the presentation that had already been written for the 2-hour session. identified a small number of core elements – what are normally referred to as the session’s “take home message”, the essential concepts that the session delivers and that the students need to understand before moving on. Introduced each of these core elements in a 3–4minute knowledge clip to be presented to the students in advance of the class. For this course short clips from films that deploy the strategies introduced in the class are presented to students. An example of these is the incipit of the film Possession (LaBute, 2002) for a discussion of anachronies – students know both actors and film, and tackle the tagging exercise with enthusiasm (here it is no longer a question of “I get it”/”I don’t get it”, but most of the work goes towards commenting the strategies used to achieve the temporal split whilst keeping the viewer engaged and in the know). Students are asked to watch and tag the clip (viewing uninterrupted).

Afterwards, they complete a crib sheet (testing/consolidating/ articulating the questions for the follow-up session). They can watch the clip again to help complete the crib sheet- this enables them to set a learning pace they are comfortable with. Intercalating the viewing with exercises (filling a crib sheet, but also a group discussion), students engage with the video – and with the pedagogical aims of using it – more actively: they feel that they need to get information that is necessary to answer questions, instead of just waiting for the “next instalment”, i.e. the live class, to be told by the tutor. In this case, the questions asked them to reflect on how the transitions from one chronological plane to another (=the anachronies) were enacted in the film, on what strategies were used to ensure that these shifts kept the attention – and understanding – of the viewer, and on how time shifts conveyed corresponding shifts in point of view. Follow-up activities The tutor delivered the full 2-hour session, which includes an expansion on theories (so that students linked the concepts learnt in the clip with the wider context, following an inductive method) and textual application of the toolkit. Observations. Making their own notes during the viewing – as well as pressing the tagging buttons – focussed the students’ attention, and made them see clearly the theory put into practice. The hands-on approach required by this tool means that students do engage with the content more fully, generally come to class more prepared, and curious to check their responses against those of the rest of the class. 53 | P a g e

The use of clips in teaching is widespread in the tutor’s university. In the School of Law, for example, clips from courtroom dramas (especially from the British soaps) are used to discuss how accurately (or conversely how badly) the workings of the legal system are portrayed in prime-time TV.

54 | P a g e

Application 5: EVOLI in a Financial Management and a Corporate Finance class (Author: Violeta Madzova, International Balkan University, North Macedonia) Context The tool was used in three units, as follows: Unit 1: Stock Pricing - Learning outcomes: Recognise and apply models of pricing of different types of stock Unit 2: Calculation of VAT (two approaches) - Learning outcomes: apply two approaches to VAT calculation Unit 3: Cash Management - Learning outcomes: Apply the principles of cash management planning Resources All the clips used for these activities were available in YouTube. A number of students have difficulties in understanding the methods of calculation (calculation of the values of stocks, / VAT calculation or cash management) hence multiple explanation of certain procedures are normally required. Students can watch clips as many times as they need to understand the calculations. If problems remain, they can ask for clarification in class. Financial Management -40 students https://www.youtube.com/watch?v=7T8sgTotDU Public Finance – 40 students https://www.youtube.com/watch?v=a6RB4rIxWq I

Corporate Finance -15 students https://www.youtube.com/watch?v=HBYwpqn1F Mk

Process Students watch the clip on their own, and once again in class. Follow-up activities Discussion in class of the topics introduced in the clips, and any difficulties experienced by students. Learning to articulate the difficulties encountered, and to explain to others core concepts is a useful skill, and used in this context was an effective example of peer learning. A bank of examples can be used to answer students’ questions. Observations 95 students on 3 courses were asked to watch the videos. Students understood most of the videos after watching 2-3 times. The questions asked were very helpful for further clarification.

55 | P a g e

Application 6: EVOLI in the Physics class (Author: Matteo Passoni, Politecnico di Milano, IT) Description Politecnico di Milano performed a test with three courses: 1.“Physics of the Nucleus”, an elective course within the Nuclear Engineering track at Master level. Circa 70 students enrolled, more than 40 videos shared. 2. “Particles’ physics”, an elective course, following “Physics of the Nucleus” and was taught by the same instructor, within the Nuclear Engineering track at Master level. 15 students circa enrolled, more than 20 videos shared. 3.“Introduction to the mathematical formalism of quantum mechanics” (an elective course falling into the category of “Passion in Actions” initiatives launched by Politecnico di Milano in 2018 to which students take part on a voluntary basis, to expand their views and foster their passions). Again, at master level, 20 students enrolled, 8 videos shared. Purpose and rationale for use EVOLI was used to support the learning of all the elements of the two courses mentioned. Assessment will be performed (starting June-July 2020) via traditional oral examinations. Although ELSE’s main mission is to support the use of innovative technologies in humanities teaching, Politecnico di Milano tested EVOLI on science courses (on a large-scale with two courses). The way EVOLI was used in these two cases helped highlight further possible scenarios of use. EVOLI was used to cover the whole content of both courses. More than 40 videos (equivalent to sessions) for the first, 8 for the second. The videos were 1 hour and a half long, on average. The “Physics of the Nucleus” course was offered in the second semester of academic year 2019-20

completely online (due to CoViD-19), on Microsoft Teams platform. Process All lessons were recorded and put online in Microsoft Stream immediately after the lesson was over. The instructor decided to use this recording to reinforce the students’ learning and to make sure that all the (quite complex) notions were clear before moving on to the next topic. He therefore uploaded the videos on EVOLI and asked students to re-watch them and tag them between one lesson and the other. The second course also makes streamed lessons available again, asking students to tag them to steer the following. For the second course, the instructor made it compulsory for the students to re-watch the videos and add at least one comment in order for him to be sure that they all attended the course and be able to grant them a digital badge. In this way the EVOLI tool is used as a virtual attendance checker (an unintended but useful benefit). Observations Due to the special conditions of the use of EVOLI (the videos were used to reinforce learning, but their content had actually already been accessed during the synchronous sessions), only a fraction (20% on average) of the students actually performed the task. Their comments were found quite useful to steer the lessons’ content, which usually started with an explanation of the unclear parts of the previous. EVOLI was used to supporting the whole course, no lesson excluded: more than 40 videos were shared with the students. The instructor plans to ask the students to use the videos through EVOLI also to prepare for the exam: he will check their comments and 56 | P a g e

plan some sort of follow up (either a live session or a video where he answers the questions). The main issue, as may be expected, was to have the students actually perform this cumbersome task, considering that they had already followed the lessons in streaming. However, this unorthodox experience with the tool allowed the development team to identify interesting new possibilities for future expansions.

57 | P a g e

Application 7: EVOLI in an Auditing course UCA (Author: Nieves Gómez Aguilar, Universidad de Cadiz, ES) Description The behaviour of the auditor (COURSE: AUDITING). In the lesson on audit regulation, an important aspect is to work on the audit code of ethics. The objective is for the student to understand the importance of the auditor's maintaining appropriate professional behaviour in his/her relationships with the client.

Follow-up activities Finally, we held a 30-minute debate on the importance of ethical behaviour and what threats might arise. Those aspects marked by the students as understood and difficult to understand were used to structure the debate. See image 2.

The trainee should understand that the client may put pressure on the auditor to issue a more favourable audit opinion than he/she deserves.

Observations The video was watched by 16 students, although some of them did not understand well the use of the tool and did not interact with it. During the discussion afterwards, they showed that they found the tool very attractive and that they would be willing to use it more. See image 1.

Resources Once the lesson is over, the student is told that the relationship between the auditor and the client is based on incentives and pressures that are difficult to understand from outside the audit relationship. The link to the video is sent to the student showing a dramatized situation. The student is asked to watch and analyse the video using the EVOLI tool. Link https://www.youtube.com/watch?v=chjhfkKMJr U&feature=youtu.be Process Students watch and tag the clip (viewing uninterrupted) In the next class, they have been asked questions about the principles of ethics (if they have seen them reflected in the video, the reaction of the auditor, what kind of pressures and incentives are observed...).



58 | P a g e

Application 8: EVOLI and a discussion on teaching methodology. (Author: Marius-Mircea Crișan, West University of Timișoara, RO) Description This session introduces the concept of teaching methods, approaches, and techniques, and starts with establishing a distinction between method and approach.

3. Developing their behaviours: learning as a whole 4. Storytelling 5. Getting students to ask the questions

Rationale for use The students analyse some of the methods and techniques that are used in English language teaching, in preparation for an assessment which is on the effectiveness of each of the methods and techniques discussed.

Students watch and tag the clip 3 times: 1. they tag what they understand and what they don’t; 2. after each method, they comment on whether their teachers used it or not; 3. they express their preferences for the techniques presented.

Resources The clip used is Teaching Tips: Best classroom techniques for teachers of English (https://www.youtube.com/watch?v=KCI7R96EKQ)

51 students participated in this activity, and the EVOLI analytics enabled the tutor to observe when the students understood everything, and when conversely, they had problems understanding the content.

The process Students are encouraged to reflect on the way in which they prepare for the teaching career and on the use of the computer assisted instruction in the English lesson. They watch the 4-minute video, which presents 5 teaching approaches to the teaching of English, which are: 1. Teachers need to know how to use whatever presentation material they want to use. Knowing how to correct mistakes. 2. Dictogloss

The graphs also showed at what points of the video the students had left comments, and the type of observations they made while viewing:

59 | P a g e

It was clear from the EVOLI analytics that the students understood the individual methods presented by each teacher: there is a peak in the I get it curve at the end of each teacher’s presentation, and the I don’t get it points are very few. The tutor concluded that the next session on didactic methods could be structured in an inductive manner: beginning from real examples, to later move on to articulate a conceptual definition of teaching methods. The follow up activities Following the exercise, the group commented on the prevalence of each of the techniques shown in the video in the classes that they themselves attended. The class was also given the opportunity to reflect on their own preferred teaching technique, and to formulate a rationale for their choice.

60 | P a g e

Application 9: EVOLI in the Management Accounting course (Author: Paulino Silva, Polytechnic of Porto, PT) Description The activity was implemented in the course ‘Management Accounting’, of the undergraduate degree in Accounting and Administration during the year 2019/20. The teaching/learning process combined flipped classroom and active learning supported by practical exercises. The process To carry out the activity, the tutor made eight videos, one about the topic “Cost-Volume-Profit Analysis”, and the other seven explaining the resolution of some practical exercises. All videos were made available on the website of the EVOLI tool: https://www.evoli.polimi.it. Step 1 – the teacher created the videos and informed about the access code. The students accessed the platform, chose the video, entered the password and watched the video. To watch the video, the student used the Evoli tool. This task was performed outside the classroom (outclass) and when the student considered most appropriate. Step 2 – in class, the students discussed with the teacher the visualized contents. Observations Video “Análise Custo-Volume-Resultado” Results The duration of the video was 2:44 minutes. The average understanding is 94/100 and the average appreciation is 94/100. The video collected 17 reactions: 15 I GET IT, 1 I DO NOT GET IT and 1 comment. Comment to the video (translated from Portuguese):

Very enlightening video about CVR analysis. Focus on the important points, well elaborated chart that allows the consolidation of the CVR analysis. The video is an important study tool for the student, since it can be viewed at any time, and in case of doubt it can be viewed as many times as necessary. It works almost like an individualized class. The EVOLI tool, in my point of view, is an asset both for the student and the teacher, as it allows the student during the video, to transmit whether he understood the content or not, allowing the teacher to see what his difficulties in understanding the various contents.

61 | P a g e

Conclusions The EVOLI tool demonstrated to be a useful and effective alternative educational tool for flipped classroom. Students appreciated its use and

demonstrated a better preparation in solving the practical exercises provided in class.

62 | P a g e

Application 10: EVOLI in the Multimedia Contents course (Authors: Luciana Oliveira & Arminda Sa Sequeira, Polytechnic of Porto, PT) Description This activity is part of a series of activities of a master course named “Multimedia Contents”, which is part of the master's degree in Digital Communication Rationale Cognitive Objectives: Recognize fundamentals of audio editing and apply them in the development of project. Soft Skills: Learn to learn; Learn to do Learning Outcomes: Digital audio radio commercial Resources A video on fundamentals of digital audio The process Phase 1 – Students watch the theoretical video (lecture format) (out class) Phase 2 – Students watch the practical video, while implementing the same procedures on demo audio files (out class) Phase 3 – Problem based exercise in two stages: (1) designing a radio commercial storyboard in class and (2) executing the commercial with the acquired knowledge out class. Phase 4 – Presentation, discussion, and evaluation of radio commercials (in class), and small quiz about theoretical audio concepts (in class) Observations General description (nature and type):

The teaching / learning process combines Flipped Classroom methodologies (using affirmative, interrogative methods) and active learning supported by the development of projects. Analytics: Duration of the video: 22 minutes

The video has been scored 10 times. The average understanding is 88 and the average appreciation is 91. The video collected 105 reactions: 100 I GET IT, 4 I DO NOT GET IT and 1 comment.

Figure 1: Recap for the first video of the activity – all reactions (lecture) There was only one comment at the beginning of the video, sta�ng that this ac�vity requires very high technical skills to be implemented, and it could be a challenge for students with no background in the field. The main four moments with I DO NOT GET IT are illustrated bellow, and it’s possible to conclude that they consist of two of the most complex concepts explained throughout the video.

63 | P a g e

Figure 2: Screens for the I DO NOT GET IT moments of the first video It is interesting to note that there are several GET IT interactions overlapped, which might be an indicator of the end of an idea that was being presented, since the video does not pose questions to students. However, by using EVOLI, this could be a strategy to implement in the future: design videos with questions, requiring students’ confirmation of understanding or even game-based questions, where the buttons GET IT and I DO NOT GET IT would be used as functions with specific meanings. A small survey was conducted by the end of the course, to evaluate the students’ perception regarding activities and resources. In relation to the use of videos, students commented that they were quite long. Therefore, in the future, it would be worth considering what would be the optimal duration of the videos.

64 | P a g e

CHAPTER 4 EDASH – The assessment tool Assessment is the principal instrument with which tutors recognise and reward student progress and provide motivation for further achievement. Assessment also enables tutors and students to identify areas for development in both teaching and learning, monitor levels of achievement, maintain academic standards, and help to prepare students for professional life. The fact that good performance in assessment strongly correlates with sustained engagement has been demonstrated by research (e.g., Fredericks et al., 2004; Lee and Shute, 2010; Schaufeli et al., 2002): engaged students are “motivated to invest in learning, attend classes, and participate in study activities” (Bakker et al., 2014: 49). They show intellectual curiosity, are inclined to participate in class, and rise to the challenges that the subjects pose, and as a result they are also successful in their studies (Salanova et al., 2010). Whilst the partners at the Politecnico di Milano and at Entropy were developing the teaching and learning tools EVOLI and ECORE respectively, colleagues at the University of Cyprus were working on the development of an equally fundamental tool, this time focussing on assessment of student engagement and performance. The partners’ purpose in designing EDASH was to ensure that all the activities that tutors set up and administer through Moodle – the standard Moodle activities, plus those carried out via EVOLI and ECORE – could be captured through an integrated dashboard that returned all the

information needed to monitor students’ engagement and level of success in a unit. 18 The resulting tool – EDASH – is a comprehensive dashboard that combines data from different sources, all converging to return an accurate picture of how students are doing: the extent to which they are engaging with the teaching and learning materials, their engagement with the assessment, their level of performance in absolute terms and in relation to the rest of the class. These data assist tutors to gain an overall picture of their students’ performance, grades, and whether the students have tried the various activities or not. The dashboard can be used to get an overview the entire cohort on a course, with the option to zoom in on individual students, drilling down on individual performances and trends. The tool is invaluable for students too, as the synoptic view of the learning data for each of the courses that they take constitutes an important instrument for self-evaluation: it contributes to developing a sense of self-efficacy and selfconcordance (Sheldon et al., 2004), making students feel that stronger engagement and better assessment results are goals they pursue because they are in line with their own interests and values, and not because they are imposed by some external authority. The tool enables the tutor to group all learning activities into separate categories according to whether the activity: (i) is “mandatory”, “optional” or “recommended”, and (ii) is used as “flipping”,

18 The term “activity” here refers to the Moodle standard terminology and indicates work that a student does in a unit and that involves interaction with other students and/or with the tutor. The term “unit” is used to indicate an academic module that forms part of a course of study, and that carries a certain number of credits towards that course. Some institutions use different words to define these elements of a programme of study – for instance “modules” or “courses”.

65 | P a g e

“during” (the class), or “after” (the class). By selecting an activity, teachers and students have access to a visual aggregated overview of the accomplishments, in different formats depending on the data produced by each activity/tool. In supplementing the enhancements to the quality of the teaching and learning opportunities provided by EVOLI and ECORE with data on engagement and performance that are visible to tutors and students, EDASH ensures that enhancements to academic standards can also be monitored and measured. To achieve this comprehensive integration of standard and ELSE-specific activities, the new external flipped classroom tools EVOLI and ECORE had to be integrated into Moodle via Moodle’s LTI. Once that was done, tutors would provide students with links enabling them to use these external tools.

EDASH – general overview of the tool The EDASH home screen (see Fig. 1) offers the option to access data as a tutor or as a student.

a choice is made, a further option is to “View Assessment” or “Edit Activities” (Fig. 2)

Fig. 4. The teachers’ main menu

The student view enables students to see data for each unit on which they are enrolled. Users are prompted to select a unit, and then select the “View Assessment” option. “View assessment” offers a combination of several suitable forms of data visualisation such as graphs, statistics, text, and representation of individual and aggregated student learning data. All learning activities are categorized based on: (i) whether the activity is “mandatory”, “optional” or “recommended” (see Fig. 3), and (ii) whether the activity is used prior to the class named as “flipping”, “during” (the class), or “after” (the class).

Fig. 5. The teachers’ “Edit Activity” board

Viewing Assessment Data Fig. 3. The tool home screen

The tutor view gives tutors the option to view data by unit (an overview of the entire class) or by student (looking at individual performance). Once

Tutors have access to a visual overview of the students’ activities, while students have access to their own profile only (Fig. 4).

66 | P a g e

A demonstration of the functionality of the EDASH tool is illustrated in this video.

Tutor Feedback on the initial versions of EDASH, and improvements made

Fig. 6. Overview of selected students’ activities

The activities are also displayed based on their categorisation (Fig. 5).

Fig. 7. “View Assessment” by category

By selecting an activity, teachers and students have access to a visual aggregated overview of the performance, in different formats depending on the data produced by each activity/tool. Fig. 6 presents a graph, combining the “Total time played for the 1st try” of an instance of the ECORE tool, for all the students of a specific course.

The suite of ELSE tools was presented to an audience of tutors from all over Europe at an Intensive Study Programme event organised at the Politecnico in Milan in February 2019. As well as an exercise in dissemination of the project outcomes, the presentations at the Milan event also had the fundamental aim of collecting practitioners’ feedback on design, functionality and usefulness that could inform the finalisation of the tools prior to their general release. In relation to EDASH, there was consensus amongst the participants on the importance of analytics, hence on the usefulness of EDASH as an instrument to collate important data that had to be made available both to tutors and to students. The response from the audience, and the suggestions for improvements to the appearance of the dashboard (data has to be presented in a manner which “speaks to the user” – its significance has to be understood for it to point to necessary interventions) inspired the changes that the Cyprus team made to the tool and resulted in the version of the tool presented here.

Fig. 8. Aggregated visualization of students’ data for ECORE

67 | P a g e

68 | P a g e

REFERENCES and SUGGESTED FURTHER READING Abdullah, N., Hanafiah, M. & Hashim, N. (2013). Developing creative teaching module: Business simulation in teaching strategic management. International Education Studies 6(6): 95-107. Altemueller, L. & Lindquist, C. (2017). Flipped classroom instruction for inclusive learning. British Journal of Special Education 44: 341-358. Anderson, L. W. & Krathwohl D. R. (eds.) (2001). A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman. Bakker, A. B., Vergel, A. I. S. & Kuntze, J. (2015). Student engagement and performance: A weekly diary study on the role of openness. Motivation and Emotion 39(1): 49-62. Barrows, H. S. & Tamblyn, R. M. (1980). Problem-Based Learning. An Approach to Medical Education. New York: Springer. Bateson, G. (1972). Steps to an Ecology of Mind. New York: Ballantine Books. Bellotti, F. et al. (2012). Designing a course for stimulating entrepreneurship in Higher Education through Serious Games. Procedia Computer Science 15: 174-186. Biro, G. I. (2013). Ready, Study, Share: An inquiry into the didactic approach of gamification with a special view to the possible application in Higher Education. European Scientific Journal 9(19): 341-345. Bergmann, J. & Sams, A. (2012). Flip Your Classroom: Talk to Every Student In Every Class Every Day. Alexandria: ISTE-ASCD. Bevilacqua, A. (2018). Inverted approach. A review of the scientific literature. Available at https://flipnet.it/wpcontent/uploads/2019/05/FL_review.pdf Biggs J. (2003). Teaching for Quality Learning at University (2nd ed). Buckingham, UK: Society for Research into Higher Education and Open University Press. Boller, S. (2014). Games vs simulations: Choosing the right approach for learning. March 26. Available at http://www.theknowledgeguru.com/games-vs-simulations-choosing-right-approach/ Butt, A. (2014). Student views on the use of a flipped classroom approach: Evidence from Australia. Business Education & Accreditation 6(1): 33-43. Chickering, A. W. & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin March: 3-7. Cinganotto, L. & Cuccurullo, D. (2015). The role of videos in the teaching and learning of content in a foreign language. Journal of e-Learning and Knowledge Society 11(2): 49-62. Clark, D. (2016). Peer instruction. University of Kent (16 March). Available at https://blogs.kent.ac.uk/learntech/2016/03/16/peer-instruction/ Darvas, P. (1999). Higher education development in transitional societies. In J. Brennan et al. (eds.). What Kind of University?. Buckingham: Open University Press. De Miguel, M. (ed.) (2006). Metodologia de enseñanza aprendizaje y para el desarrollo de competencias: Orientaciones para el profesorado universitario ante el espacio de Europeo educación superiore. Madrid: Alianza. Dewey, J. (2015). Experience and Education. New York: Macmillan Company. Di Blas, N., Calchi, V., Lorini, R., Peruselli, D. & Torrebruno, A. (2021). A video-tagging tool to support the Flipped Classroom. In Langran, E. & Archambault, L. (eds.). Proceedings of Society for Information Technology & Teacher Education International Conference Online. United States: Association for the Advancement of Computing in Education (AACE): 474-479. Ehlers, U-D. (2020). Future Skills. The future of learning and higher education. Springer: Skills on Demand. Available at https://nextskills.org/library/future-skills/ Epasto, A. A. (2015). La formazione professionale dei docenti universitari: Analisi e prospettive. Quaderni di Intercultura 7: 49-66. Farrell, C. (2005). Perceived effectiveness of simulations in international business pedagogy: An exploratory analysis. Journal of Teaching in International Business 16(3): 71-88. Findlay-Thompson, S. & Mombourguette, P. (2014). Evaluation of a flipped classroom in an undergraduate business course. Business Education & Accreditation 6(1): 63-71. 69 | P a g e

Finnegan, F., Merrill, B. & Thunburg, C. (2014) (eds.). Student Voices on Inequalities in European Higher Education. London: Routledge. Formenti, L. & West, L. (2018). Transforming Perspectives in Lifelong Learning and Adult Education. A Dialogue. London: Palgrave Macmillan. Fredericks, J. A., Blumenfeld, P. C. & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research 74: 59-109. Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H. & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences 111(23): 8410-8415. Frydenberg, M. (2012). The Flipped Classroom: It’s got to be done right. Huffpost, 14 December. Available at https://www.huffpost.com/entry/the-flipped-classroomits_b_2300988?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer _sig=AQAAACJULA1TVVrzq5iJe7bw24I9u1swVc4d4HZ6UPFYhVTia5B0DA5Q6HY53JLm7_N4MlKaJFl5FL9MHExXvGs1a30hAsRped6c7mpv5FsSetJEKsnNbu1O8B6H-rnY0zKtJPJQYYNu4iyIpzGjTf7dDNHzarhsMBilWmJU9dVqUt Galliani, L., Zaggia, C. & Serbati, A. (eds.) (2011). Apprendere e valutare competenze all’università. Progettazione e sperimentazione di strumenti nelle lauree magistrali. Lecce, Pensa MultiMedia. Gauntlett, D. (2018). Making is Connecting. The social power of creativity, from craft and knitting to digital everything. Cambridge: Polity Press. Gibbs, G. (2010). Dimensions of Quality. York: The Higher Education Academy. Gibbs, G. (2012). Implications of ‘Dimensions of Quality’ in a Market Environment. York: The Higher Education Academy. Graziano, A. & Sibi, P. (2020). Playing Serious Games in Higher Education. In Chiorean, L. & Nicolae, C. (eds). Humanities in the Spotlight. The Role of Humanities in Pandemic Times. Beau Bassin, LAP Lambert Academic Publishing: 358-382. Kasbi, Y. (2013). Les Serious Games. Une Révolution. Liège: Edipro. Lave, J. & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: CUP. Lee, J. & Shute, V. J. (2010). Personal and social-contextual factors in K-12 academic performance: An integrative perspective on student learning. Educational Psychologist 45: 185-202. Lewis, L. H. & Williams, C. J. (1994). Experiential learning: Past and present. In Jackson, L. & Caffarella, R. S. (eds.). Experiential Learning: A New Approach. San Francisco: Jossey-Bass: 5-16. Love, B., Hodge, A., Corritore, C. & Ernst, D. (2015). Inquiry-based learning and the flipped classroom model. Primus 25(8): 745-762. Mazur, E. (1997). Peer Instruction: A User's Manual. Upper Saddle River, NJ: Prentice Hall. Mazur, E. (2013). The flipped classroom will redefine the role of educators. Retrieved from: https://www.seas.harvard.edu/news/2013/03/flipped-classroom-will-redefine-role-educators Mezirow, J. (1991). Transformative Dimensions of Adult Learning. New York: Wiley & Sons. Moodle Learning Management System. Available at https://moodle.org/. Moon, J. (2002). The Module and Programme Development Handbook. London: Kogan Page. Nerantzi, C. & Hannaford, L. (2016). Flipping the classroom using teams. A case study from academic development. In Whatley, J. & Nerantzi, C. (eds.). Teaching with Team Projects in Higher Education. Santa Rosa, CA: Informing Science Press: 119-130. Nerantzi, C. (2017). Towards a Framework for Cross-boundary Collaborative Open Learning in Crossinstitutional Academic Development. PhD thesis. Edinburgh: Edinburgh Napier University. Nerantzi, C. (2020). The use of peer instruction and flipped learning to support flexible blended learning during and after the COVID-19 pandemic. International Journal of Management and Applied Research 7(2): 184-195. Pancucci, V. (2017). Upside-down education and history: Tips for starting an experimentation. Retrieved from: https://it.pearson.com/content/dam/region-core/italy/pearson italy / pdf / storia / ITALY-DOCENTYSTORIALIVE-2017-Noticeboard% 20della% 20didattica Didattica% 20capovolta% 20e% 20Storia .pdf Ramsden, P. (1992). Learning to Teach in Higher Education. London: Routledge. 70 | P a g e

Robben, H. S. J., Webley, P., Elffers, H., & Hessing, D. J. (1990). Decision frames, opportunity and tax evasion: An experimental approach. Journal of Economic Behavior & Organization 14(3): 353-361. Robertson, S. (2009). Metaphoric imaginings: Re-/Visions on the idea of a university. In R. Barnett et al. (eds.). Rethinking the University after Bologna. Antwerp: Belgium. Salanova, M., Schaufeli, W. B., Martínez, I., & Bresó, E. (2010). How obstacles and facilitators predict academic performance: The mediating role of study burnout and engagement. Anxiety, Stress & Coping 23(1): 53-70. Schaufeli, W. B., Martínez, I. M., Marqués-Pinto, A. M., Salanova, M., & Bakker, A. B. (2002). Burnout and engagement in university students: A cross-national study. Journal of Cross-Cultural Psychology 33(5): 464-481. Schell, J. A. & Butler, A. C. (2018). Insights from the science of learning can inform evidence-based implementation of peer instruction. Frontiers in Education 28 May. DOI: https://doi.org/10.3389/feduc.2018.00033 Schell, J. A. & Mazur, E. (2015). Flipping the chemistry classroom with Peer Instruction. In García-Martínex, J. & Serrano-Torregrosa, E. (eds.), Chemistry Education: Best Practices, Opportunities and Trends. Weinheim: Wiley-VCH: 319-334. Sheldon, K. M., Elliot, A. J., Ryan, R. M. et al. (2004). Self-concordance and subjective well-being in four cultures. Journal of Cross-Cultural Psychology 35(2): 209-223. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning 2(10): 123-142. Smith, C. E. (2017). The flipped classroom. Nursing 47(4): 20-22. Tsai, C. W., Shen, P. D., and Lu, Y. J. (2017). The effects of problem-based learning with flipped classroom on elementary students’ computing skills: A case study of the production of Ebooks. Research and Practice in Technology Enhanced Learning 12(4): 836-838. Tucker, B. (2012). The flipped classroom. Education Next, Winter. Available at http://www.msuedtechsandbox.com/MAETELy2-2015/wpcontent/uploads/2015/07/the_flipped_classroom_article_2.pdf Villa Sanchez, A. & Poblete Ruiz, M. (2008). Competence-Based Learning. Bilbao: University of Deusto. Warren, K., (1995). The student-directed classroom: A model for teaching experiential education theory. In Warren, K. (ed.). The Theory of Experiential Education. Dubuque: Kendall Hunt Publishing: 249-258. Wurdinger, S. D. (2005). Using Experiential Learning in the Classroom. Lanham: Scarecrow Education. Zagal, J. P. & Deterding, S. (2018). Role-Playing Game Studies: Transmedia Foundations. New York: Routledge. Zaggia, C. (2008). L’università delle competenze. Milano: Franco Angeli. Zhonggen, Y. (2019). A meta-analysis of use of Serious Games in education over a decade. International Journal of Computer Games Technology. https://doi.org/10.1155/2019/4797032. Online resources 12 resources on flipped learning https://inservice.ascd.org/12-resources-on-flipped-learning/ Flipped learning https://flippedlearning.org/ Flipped learning ideas https://flippedlearningideas.wordpress.com/ Flipped learning simplified with Jon Bergman, Research http://www.jonbergmann.com/research/ Resources for teaching: Flipped classroom https://sp-sg.libguides.com/c.php?g=377363&p=2553416 Turn to your neighbour, The official peer instruction blog https://peerinstruction.wordpress.com/ Zotero Collection on flipped learning Instructional Video | Zotero

71 | P a g e

FOLLOW US AT: www.elseproject.eu

72 | P a g e